Tight Sequential Confidence Intervals for Online Kernel Regression
Haichen Hu, David Simchi-Levi
Abstract
We characterize, in the worst case over kernels, the sharp minimax width of online confidence intervals for elements of a reproducing kernel Hilbert space (RKHS). This resolves the open problem posed by Vakili et al. (2021) in its original discrete-time form. A learner observes yi=f(xi)+εi at adaptively chosen points, with \|f\| Hk B and i.i.d. R-sub-Gaussian noise, and reports the interval |f(x)-μn(x)|ρn(δ)σn(x), where μn and σn are the posterior mean and standard deviation of a surrogate Gaussian process model with regularization parameter λ. Writing ρn*(δ) for the minimax confidence width for a fixed kernel, we show that, for the sum of a linear kernel and a periodic Sobolev kernel, all sufficiently large n, and all δ∈(0,1/4], ρn*(δ) B+(R/λ)γn+(1/δ), where γn is the maximal information gain and grows polynomially in n. Building on Lattimore (2023), we construct an adaptive design on a fixed two-dimensional domain for which, under Gaussian noise, the normalized error at a prespecified point is Ω(B+(R/λ)γn) with constant probability. This rules out the conjectured polylogarithmic width in the worst case over kernels and online designs.
Create a lesson
Related papers
Bayesian optimization on amortized search space
Tristan Fauvel, Federico Pavone, Titouan Guillou et al.
Exact Bayesian layers via characteristic functions
William H. W. Thompson, Guillaume St-Onge, Laurent Hébert-Dufresne et al.
Flow Matching from Non-Gaussian Sources: A Theory of Efficient Sampling
Yuchen Zhou, Lili Zheng, Yuxin Chen et al.
Conformal Selection for Hierarchical Streaming Data
Zhenyuan Sun, Chengyao Yu, Ruixing Ming et al.
A Statistical Framework for Training Data Auditing of Large Language Models
Shengjie Niu, Yeheng Ge, Jian Huang
Hellinger Decision Trees for Density Ratio Estimation: Finite-Sample Guarantees
Ren-Rui Liu, Zheng-Chu Guo