Partial exchangeability of the prior via shuffling

Abstract

In inference problems involving a multi-dimensional parameter $\theta$, it is often natural to consider decision rules that have a risk which is invariant under some group $G$ of permutations of $\theta$. We show that this implies that the Bayes risk of the rule is {\em as if} the prior distribution of the parameter is partially exchangeable with respect to $G$. We provide a symmetrization technique for incorporating partial exchangeability of $\theta$ into a statistical model, without assuming any other prior information. We refer to this technique as {\em shuffling}. Shuffling can be viewed as an instance of empirical Bayes, where we estimate the (unordered) multiset of parameter values $\{\theta_1,\theta_2,\dots,\theta_p\}$ while using a uniform prior on $G$ for their ordering. Estimation of the multiset is a missing data problem which can be tackled with a stochastic EM algorithm. We show that in the special case of estimating the mean-value parameter in a regular exponential family model, shuffling leads to an estimator that is a weighted average of permuted versions of the usual maximum likelihood estimator. This is a novel form of shrinkage.