Skip to content

Tight Sequential Confidence Intervals for Online Kernel Regression

Haichen Hu, David Simchi-Levi

stat.MLarXiv:2611.19772

Abstract

We characterize, in the worst case over kernels, the sharp minimax width of online confidence intervals for elements of a reproducing kernel Hilbert space (RKHS). This resolves the open problem posed by Vakili et al. (2021) in its original discrete-time form. A learner observes yi=f(xi)+εi at adaptively chosen points, with \|f\| Hk B and i.i.d. R-sub-Gaussian noise, and reports the interval |f(x)-μn(x)|ρn(δ)σn(x), where μn and σn are the posterior mean and standard deviation of a surrogate Gaussian process model with regularization parameter λ. Writing ρn*(δ) for the minimax confidence width for a fixed kernel, we show that, for the sum of a linear kernel and a periodic Sobolev kernel, all sufficiently large n, and all δ∈(0,1/4], ρn*(δ) B+(R/λ)γn+(1/δ), where γn is the maximal information gain and grows polynomially in n. Building on Lattimore (2023), we construct an adaptive design on a fixed two-dimensional domain for which, under Gaussian noise, the normalized error at a prespecified point is Ω(B+(R/λ)γn) with constant probability. This rules out the conjectured polylogarithmic width in the worst case over kernels and online designs.

Create a lesson