Estimation of Smooth Functionals in Normal Models: Bias Reduction and Asymptotic Efficiency

Abstract

Let $X_1,\dots, X_n$ be i.i.d. random variables sampled from a normal distribution $N(\mu,\Sigma)$ in ${\mathbb R}^d$ with unknown parameter $\theta=(\mu,\Sigma)\in \Theta:={\mathbb R}^d\times {\mathcal C}_+^d,$ where ${\mathcal C}_+^d$ is the cone of positively definite covariance operators in ${\mathbb R}^d.$ Given a smooth functional $f:\Theta \mapsto {\mathbb R}^1,$ the goal is to estimate $f(\theta)$ based on $X_1,\dots, X_n.$ Let $$ \Theta(a;d):={\mathbb R}^d\times \Bigl\{\Sigma\in {\mathcal C}_+^d: \sigma(\Sigma)\subset [1/a, a]\Bigr\}, a\geq 1, $$ where $\sigma(\Sigma)$ is the spectrum of covariance $\Sigma.$ Let $\hat \theta:=(\hat \mu, \hat \Sigma),$ where $\hat \mu$ is the sample mean and $\hat \Sigma$ is the sample covariance, based on the observations $X_1,\dots, X_n.$ For an arbitrary functional $f\in C^s(\Theta),$ $s=k+1+\rho, k\geq 0, \rho\in (0,1],$ we define a functional $f_k:\Theta \mapsto {\mathbb R}$ such that \begin{align*} & \sup_{\theta\in \Theta(a;d)}\|f_k(\hat \theta)-f(\theta)\|_{L_2({\mathbb P}_{\theta})} \lesssim_{s, \beta} \|f\|_{C^{s}(\Theta)} \biggr[\biggl(\frac{a}{\sqrt{n}} \bigvee a^{\beta s}\biggl(\sqrt{\frac{d}{n}}\biggr)^{s} \biggr)\wedge 1\biggr], \end{align*} where $\beta =1$ for $k=0$ and $\beta>s-1$ is arbitrary for $k\geq 1.$ This error rate is minimax optimal and similar bounds hold for more general loss functions. If $d=d_n\leq n^{\alpha}$ for some $\alpha\in (0,1)$ and $s\geq \frac{1}{1-\alpha},$ the rate becomes $O(n^{-1/2}).$ Moreover, for $s>\frac{1}{1-\alpha},$ the estimators $f_k(\hat \theta)$ is shown to be asymptotically efficient. The crucial part of the construction of estimator $f_k(\hat \theta)$ is a bias reduction method studied in the paper for more general statistical models than normal.