An Optimal Agnostic PAC Algorithm
Markus Engelund Mathiasen, Jian Qian, Nikita Zhivotovskiy
Abstract
Let H⊂eq\-1,+1\X be a class of finite VC dimension d1. Writing L for the binary risk and L*=h∈ HL(h), we construct a learner achieving the statistically optimal risk bound: from an i.i.d.\ sample of size n, for every 0<δ 1/2, with probability at least 1-δ, \[ L( h) L*+ 7·108( L*(d+(1/δ))n +d+(1/δ)n ). \] This settles the sample complexity of agnostic PAC learning up to universal constants at every fixed L*, matching the lower bounds of Devroye, Györfi, and Lugosi [A Probabilistic Theory of Pattern Recognition, Springer, 1996].
Create a lesson
Related papers
CalibForge: Adversarial Solver Calibration for Scaling Learnable Terminal Tasks
Fanzhe Meng, Guoxin Chen, Jiale Zhao et al.
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction
Chenglong Wang, Ziming Zhu, Yifu Huo et al.
On-Policy Self-Distillation without Any Supervision
Yijiang Li, Bingyang Wang, Yijun Liang et al.
BaKron: Efficient Quantization with Kronecker-Factored Hessians
Johann Birnick, Rayan Saab
Surv-IPTB: An Attention-Based Model for Estimating Individual Probability of Treatment Benefit with Survival Data
Lev V. Utkin, Stanislav K. Kogan, Andrei V. Konstantinov
The Tamed Subgradient Unadjusted Langevin Algorithm beyond Convexity
Iosif Lytras, Nikolaos Makras, Sotirios Sabanis