Optimal Transport PAC-Bayes Bounds
Armin Beck, Peter Ochs, Jalal Fadili
Abstract
PAC-Bayesian generalization bounds are among the strongest distribution-dependent guarantees in statistical learning and are typically expressed in terms of the Kullback-Leibler (KL) divergence between a prior and posterior distribution. While analytically convenient, the KL divergence requires the posterior to be absolutely continuous w.r.t. the prior, imposing support constraints that can substantially restrict the choice of posterior. Moreover, the KL divergence does not account for the underlying geometry of the probability space. In this work, we derive new PAC-Bayesian bounds replacing the KL divergence with general optimal transport (OT) distances, including the total variation and Wasserstein-p distances.
Create a lesson
Related papers
SPLICE: Sparse Physics Library Identification and Compositional Expansion
Lucas Thil, Jesse Read, Guillaume Doquet et al.
LLM Evaluation is a Forecasting Problem
Rahul Thomas, Adam Block, Micah Goldblum
ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertainty
Yuyang Dai, Yuting Tang, Futoshi Futami et al.
MAGNET: Manifold-Aware Graph Diffusion Network for Connectome Generation
Protyay Dey, Ayush Roy, Hyuna Cho et al.
Mixture of GFlowNets
Omar G. Younis, Joseph D. Viviano, Chirayu Anant Haryan et al.
A physics-coupled graph neural operator for mid-plane six-component stress tensor field prediction in concrete penetration
Wenpu Du, Peng Zhou, Yunlong Xia et al.