Beyond Smoothness: Incorporating Low-Rank Analysis into Nonparametric Density Estimation
Robert A. Vandermeulen, Antoine Ledent
Abstract
The construction and theoretical analysis of the most popular universally consistent nonparametric density estimators hinge on one functional property: smoothness. In this paper we investigate the theoretical implications of incorporating a multi-view latent variable model, a type of low-rank model, into nonparametric density estimation. To do this we perform extensive analysis on histogram-style estimators that integrate a multi-view model. Our analysis culminates in showing that there exists a universally consistent histogram-style estimator that converges to any multi-view model with a finite number of Lipschitz continuous components at a rate of O(1/[3]n) in L1 error. In contrast, the standard histogram estimator can converge at a rate slower than 1/[d]n on the same class of densities. We also introduce a new nonparametric latent variable model based on the Tucker decomposition. A rudimentary implementation of our estimators experimentally demonstrates a considerable performance improvement over the standard histogram estimator. We also provide a thorough analysis of the sample complexity of our Tucker decomposition-based model and a variety of other results. Thus, our paper provides solid theoretical foundations for extending low-rank techniques to the nonparametric setting
Create a lesson
Related papers
Species sampling priors through exact augmentation
Ramsés H. Mena, Christos Merkatas, Theodoros Nicoleris et al.
Adaptive local Fréchet regression with polyspherical predictors
Chang Jun Im
Spectral-Domain Information Geometry of Stationary Covariance Models: Gaussian and Elliptical Extensions
Shu Yang, Dongying Li, Wenxian Yu
A Strict Upper Bound for Three-Point Weighted Selection in Planar Vector-Valued Regression
Sergey Volkov
Linear Functional Estimation under Multiplicative Measurement Error with Unknown Error Distribution
Jan Johannes, Maximilian Siebel
Minimum cumulative power divergence estimation with applications to copulas
Ostap Okhrin, Konstantinos Zografos