Characterising variation of nonparametric random probability measures using the Kullback-Leibler divergence

Abstract

This work studies the variation in Kullback-Leibler divergence between random draws from some popular nonparametric processes and their baseline measure. In particular we focus on the Dirichlet process, the P\'olya tree and the frequentist and Bayesian bootstrap. The results shed light on the support of these nonparametric processes. Of particular note are results for finite P\'olya trees that are used to model continuous random probability measures. Our results provide guidance for specifying the parameterisation of the P\'olya tree process that allows for greater understanding while highlighting limitations of the standard canonical choice of parameter settings.