Least-Squares Problem Over Probability Measure Space

Abstract

In this work, we investigate the variational problem $$\rho_x^\ast = \text{argmin}_{\rho_x} D(G\#\rho_x, \rho_y)\,, $$ where $D$ quantifies the difference between two probability measures, and ${G}$ is a forward operator that maps a variable $x$ to $y=G(x)$. This problem can be regarded as an analogue of its counterpart in linear spaces (e.g., Euclidean spaces), $\text{argmin}_x \|G(x) - y\|^2$. Similar to how the choice of norm $\|\cdot\|$ influences the optimizer in $\mathbb R^d$ or other linear spaces, the minimizer in the probabilistic variational problem also depends on the choice of $D$. Our findings reveal that using a $\phi$-divergence for $D$ leads to the recovery of a conditional distribution of $\rho_y$, while employing the Wasserstein distance results in the recovery of a marginal distribution.