On the T-test

Abstract

The $T$-test is probably the most popular statistical test; it is routinely recommended by the textbooks. The applicability of the test relies upon the validity of normal or Student's approximation to the distribution of Student's statistic $\,t_n$. However, the latter assumption is not valid as often as assumed. We show that normal or Student's approximation to $\,\L(t_n)\,$ does not hold uniformly even in the class $\,{\cal P}_n\,$ of samples from zero-mean unit-variance bounded distributions. We present lower bounds to the corresponding error. The fact that a non-parametric test is not applicable uniformly to samples from the class $\,{\cal P}_n\,$ seems to be established for the first time. It means the $T$-test can be misleading, and should not be recommended in its present form. We suggest a generalisation of the test that allows for variability of possible limiting/approximating distributions to $\,\L(t_n)$.