Geometric Information Decomposition for Weighted Empirical Measures on the Sphere

Abstract

We study directional uncertainty when the data already represent a weighted probability measure on the unit sphere, as in importance samples, quadrature rules, or attention-weighted embeddings. A standard approach fits a von Mises-Fisher distribution and reports its concentration or entropy. This is principled but incomplete because vMF uses only mean-direction information and can miss antipodal, axial, girdle-like, or multimodal structure. We introduce the geometric information decomposition (GID), which fits a nested sequence of maximum-entropy projections using spherical features and reports the entropy gap added at each level. The first gap recovers vMF information, the second captures Fisher-Bingham/Bingham-type anisotropy, and later gaps capture finer angular structure. We prove invariance, consistency, asymptotic normality away from zero gaps, and a quadratic-form null calibration for deciding whether a new level carries information. Experiments on circular and spherical examples, calibration studies, and a query-weighted digit projection show when vMF uncertainty suffices and when higher-order gaps reveal hidden structure.