Consistency of Nonparametric Density Estimators in CAT(0) Orthant Space

Abstract

The inference of evolutionary histories is a central problem in evolutionary biology. The analysis of a sample of phylogenetic trees can be conducted in Billera-Holmes-Vogtmann tree space, which is a CAT(0) metric space of phylogenetic trees. The globally non-positively curved (CAT(0)) property of this space enables the extension of various statistical techniques. In the problem of nonparametric density estimation, two primary methods, kernel density estimation and log-concave maximum likelihood estimation, have been proposed, yet their theoretical properties remain largely unexplored. In this paper, we address this gap by proving the consistency of these estimators in a more general setting$\unicode{x2014}$CAT(0) orthant spaces, which include BHV tree space. We extend log-concave approximation techniques to this setting and establish consistency via the continuity of the log-concave projection map. We also modify the kernel density estimator to correct boundary bias and establish uniform consistency using empirical process theory.