A New Identity for the Least-square Solution of Overdetermined Set of Linear Equations

Abstract

In this paper, we prove a new identity for the least-square solution of an over-determined set of linear equation $Ax=b$, where $A$ is an $m\times n$ full-rank matrix, $b$ is a column-vector of dimension $m$, and $m$ (the number of equations) is larger than or equal to $n$ (the dimension of the unknown vector $x$). Generally, the equations are inconsistent and there is no feasible solution for $x$ unless $b$ belongs to the column-span of $A$. In the least-square approach, a candidate solution is found as the unique $x$ that minimizes the error function $\|Ax-b\|_2$. We propose a more general approach that consist in considering all the consistent subset of the equations, finding their solutions, and taking a weighted average of them to build a candidate solution. In particular, we show that by weighting the solutions with the squared determinant of their coefficient matrix, the resulting candidate solution coincides with the least square solution.