Tainted Evidence: Cosmological Model Selection vs. Fitting

Abstract

Interpretation of cosmological data to determine the number and values of parameters describing the universe must not rely solely on statistics but involve physical insight. When statistical techniques such as "model selection" or "integrated survey optimization" blindly apply Occam's Razor, this can lead to painful results. We emphasize that the sensitivity to prior probabilities and to the number of models compared can lead to "prior selection" rather than robust model selection. A concrete example demonstrates that Information Criteria can in fact misinform over a large region of parameter space.