A Statistical Distance Derived From The Kolmogorov-Smirnov Test: specification, reference measures (benchmarks) and example uses

Abstract

Statistical distances quantifies the difference between two statistical constructs. In this article, we describe reference values for a distance between samples derived from the Kolmogorov-Smirnov statistic $D_{F,F'}$. Each measure of the $D_{F,F'}$ is a measure of difference between two samples. This distance is normalized by the number of observations in each sample to yield the $c'=D_{F,F'}\sqrt{\frac{n n'}{n+n'}}$ statistic, for which high levels favor the rejection of the null hypothesis (that the samples are drawn from the same distribution). One great feature of $c'$ is that it inherits the robustness of $D_{F,F'}$ and is thus suitable for use in settings where the underlying distributions are not known. Benchmarks are obtained by comparing samples derived from standard distributions. The supplied example applications of the $c'$ statistic for the distinction of samples in real data enables further insights about the robustness and power of such statistical distance.