On the optimal estimation of probability measures in weak and strong topologies

Abstract

Given random samples drawn i.i.d. from a probability measure $\mathbb{P}$ (defined on say, $\mathbb{R}^d$), it is well-known that the empirical estimator is an optimal estimator of $\mathbb{P}$ in weak topology but not even a consistent estimator of its density (if it exists) in the strong topology (induced by the total variation distance). On the other hand, various popular density estimators such as kernel and wavelet density estimators are optimal in the strong topology in the sense of achieving the minimax rate over all estimators for a Sobolev ball of densities. Recently, it has been shown in a series of papers by Gin\'{e} and Nickl that these density estimators on $\mathbb{R}$ that are optimal in strong topology are also optimal in $\|\cdot\|_{\mathcal{F}}$ for certain choices of $\mathcal{F}$ such that $\|\cdot\|_{\mathcal{F}}$ metrizes the weak topology, where $\|\mathbb{P}\|_{\mathcal{F}}:=\sup\{\int f\,\mathrm{d}\mathbb{P}: f\in\mathcal{F}\}$. In this paper, we investigate this problem of optimal estimation in weak and strong topologies by choosing $\mathcal{F}$ to be a unit ball in a reproducing kernel Hilbert space (say $\mathcal{F}_H$ defined over $\mathbb{R}^d$), where this choice is both of theoretical and computational interest. Under some mild conditions on the reproducing kernel, we show that $\|\cdot\|_{\mathcal{F}_H}$ metrizes the weak topology and the kernel density estimator (with $L^1$ optimal bandwidth) estimates $\mathbb{P}$ at dimension independent optimal rate of $n^{-1/2}$ in $\|\cdot\|_{\mathcal{F}_H}$ along with providing a uniform central limit theorem for the kernel density estimator.