PAC Classification based on PAC Estimates of Label Class Distributions
Nick Palmer, Paul W. Goldberg
Abstract
A standard approach in pattern classification is to estimate the distributions of the label classes, and then to apply the Bayes classifier to the estimates of the distributions in order to classify unlabeled examples. As one might expect, the better our estimates of the label class distributions, the better the resulting classifier will be. In this paper we make this observation precise by identifying risk bounds of a classifier in terms of the quality of the estimates of the label class distributions. We show how PAC learnability relates to estimates of the distributions that have a PAC guarantee on their L1 distance from the true distribution, and we bound the increase in negative log likelihood risk in terms of PAC bounds on the KL-divergence. We give an inefficient but general-purpose smoothing method for converting an estimated distribution that is good under the L1 metric into a distribution that is good under the KL-divergence.
Create a lesson
Related papers
TopGQ: Fast GNN Post-Training Quantization Leveraging Topology Information
Dain Kwon, Kanghyun Choi, Hyeyoon Lee et al.
Foundation Models Meet Agriculture: Challenges Beyond Pretraining
Vishal Nedungadi, Xingguo Xiong, Marc Rußwurm et al.
Uncertainty of Vision Medical Foundation Models
Haoxu Huang, Narges Razavian
DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving
Yanqi Yu, Pingwei Sun, Jianchao Tan et al.
RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search
Rastislav Lenhardt, Teodora Dobos, Thomas Vecchiato et al.
Convergence rates for the RMSprop optimizer with full control of the hyperparameters
Steffen Dereich, Arnulf Jentzen