Closed-Form Information-Theoretic Roughness Measures for Mixture Densities

Abstract

We calculate the smoothest mixture density under a variety of prescribed specifications. This includes constraints on certain moments, specifications on density values and/or its derivatives, and prescribed probability masses in certain regions. As a roughness measure, we use Fisher Information (FI) in the space of mixtures $\cal M$. For mixtures, FI cannot be calculated in closed form. We define the space $\cal R$ of root mixtures (RMs) living on the Hilbert sphere. A transformation of FI to $\cal R$ admits a closed-form solution and yields the desired result in $\cal M$. This naturally leads to a tandem processing with two density representations maintained simultaneously in $\cal R$ and $\cal M$. FI is calculated in RM space $\cal R$ while the constraints are evaluated in mixture space $\cal M$.