Optimal Rescaling and the Mahalanobis Distance

Abstract

One of the basic problems in data analysis lies in choosing the optimal rescaling (change of coordinate system) to study properties of a given data-set $Y$. The classical Mahalanobis approach has its basis in the classical normalization/rescaling formula $Y \ni y \to \Sigma_Y^{-1/2} \cdot (y-\mathrm{m}_Y)$, where $\mathrm{m}_Y$ denotes the mean of $Y$ and $\Sigma_Y$ the covariance matrix . Based on the cross-entropy we generalize this approach and define the parameter which measures the fit of a given affine rescaling of $Y$ compared to the Mahalanobis one. This allows in particular to find an optimal change of coordinate system which satisfies some additional conditions. In particular we show that in the case when we put origin of coordinate system in $ \mathrm{m} $ the optimal choice is given by the transformation $Y \ni y \to \Sigma_Y^{-1/2} \cdot (y-\mathrm{m}_Y)$, where $$ \Sigma=\Sigma_Y(\Sigma_Y-\frac{(\mathrm{m}-\mathrm{m}_Y)(\mathrm{m}-\mathrm{m}_Y)^T}{1+\|\mathrm{m}-\mathrm{m}_Y\|_{\Sigma_Y}^2})^{-1}\Sigma_Y. $$