Entropy and inference, revisited
Ilya Nemenman, Fariel Shafee, William Bialek
Abstract
We study properties of popular near-uniform (Dirichlet) priors for learning undersampled probability distributions on discrete nonmetric spaces and show that they lead to disastrous results. However, an Occam-style phase space argument expands the priors into their infinite mixture and resolves most of the observed problems. This leads to a surprisingly good estimator of entropies of discrete distributions.
Create a lesson
Related papers
Reduced latent leakage does not reliably predict lower likelihood bias in collider inference
Tong Pan
The Greedy Bump Bias: Local Profiling Geometry and the Look-Elsewhere Effect
Tommaso Dorigo
Multi-fidelity Monte Carlo estimation of floor response spectra under combined seismic and structural parameter uncertainties
Nils Baillie, Baptiste Kerleguer, Cyril Feau et al.
Parameter inference from a non-stationary unknown process using statistical feature-based slow feature analysis
Kieran S. Owens, Masako Tamaki, Ben D. Fulcher
A Probability Model for Pentagonal Prism Dice Rolls
Paul R. Hurst, J. Naleo Hyde
Geometry-native machine learning reconstruction of DSMC moment fields with support monitoring
Ehsan Roohi