Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity

Abstract

We study latent geometry as an explicit component of representation quality in data-scarce learning. For an encoder (ϕ), we define (Q_{β,γ}(ϕ)=I(ϕ(X);Y)-β\mathcal C(ϕ)-γd_{\mathrm{int}}(ϕ)), combining task-relevant information with penalties for curvature and intrinsic latent dimension. Thus geometry becomes part of the bottleneck criterion, not only a post hoc diagnostic. Under smooth-manifold, loss-transfer, and estimator-concentration assumptions, we derive non-asymptotic low-label generalization bounds where intrinsic dimension and covering complexity enter explicitly. We characterize the information--geometry frontier and prove empirical-surrogate consistency. The analysis links encoder geometry to learning through latent covering numbers, loss-class entropy, and uniform deviation. We instantiate the theory as \texttt{V-GIB}, adding curvature and dimension penalties to variational bottleneck training. Real low-label benchmarks compare \texttt{V-GIB} with ERM, VIB, and ablations across (1%)--(20%) label fractions. Results show improved performance and reduced geometric complexity in several regimes, especially FashionMNIST and CIFAR-10, while confirming that no fixed regularizer is universally dominant.