Meager Success: A Theory of the Unlearnable for Hypothesis Testing

Abstract

When the standard of pointwise consistency for statistical inference -- convergence to the truth in every possible state of the world -- is provably unachievable, the usual responses are to change the inferential target or to strengthen background assumptions. This paper pursues a third: hold the inference problem fixed and identify the highest standard that remains achievable. I define a hierarchy of standards weaker than pointwise consistency, cast in topological terms, requiring convergence to the truth not everywhere but on a ``large'' set of probability measures. The main result is an impossibility theorem: for finite-precision tests, converging to the truth densely within each hypothesis already forces inconsistency on a comeager -- ``topologically almost all'' -- set of measures, whenever the two hypotheses are dense in their union. Distribution-free testing of conditional independence is one such case. Two further theorems characterize, in purely topological terms, exactly when each weaker standard is achievable, complementing Boeken et al.'s (2026) analysis of pointwise consistency.