Spectral Condition Numbers of Orthogonal Projections and Full Rank Linear Least Squares Residuals

Abstract

A simple formula is proved to be a tight estimate for the condition number of the full rank linear least squares residual with respect to the matrix of least squares coefficients and scaled 2-norms. The tight estimate reveals that the condition number depends on three quantities, two of which can cause ill-conditioning. The numerical linear algebra literature presents several estimates of various instances of these condition numbers. All the prior values exceed the formula introduced here, sometimes by large factors.