Taking all positive eigenvectors is suboptimal in classical multidimensional scaling

Abstract

It is hard to overstate the importance of multidimensional scaling as an analysis technique in the broad sciences. Classical, or Torgerson multidimensional scaling is one of the main variants, with the advantage that it has a closed-form analytic solution. However, this solution is exact if and only if the distances are Euclidean. Conversely, there has been comparatively little discussion on what to do in the presence of negative eigenvalues: the intuitive solution, prima facie justifiable in least-squares terms, is to take every positive eigenvector as a dimension. We show that this, minimizing least-squares to the centred distances instead of the true distances, is suboptimal - throwing away positive eigenvectors can decrease the error even as we project to fewer dimensions. We provide provably better methods for handling this common case.