Batch learning equals online learning in Bayesian supervised learning

Abstract

In this paper we study Bayesian supervised learning models proposed by Lê in \cite{Le2025}. We show the existence of Bayesian inversions on universal Bayesian supervised learning models $(\mathcal{P}(\mathcal{Y})^{\mathcal{X}}, μ, \mathrm{Id}_{\mathcal{P}(\mathcal{Y})^{\mathcal{X}}}, \mathcal{P}(\mathcal{Y})^{\mathcal{X}}$ for arbitrary input space $\mathcal{X}$, Souslin label space $\mathcal{Y}$, and prior probability measure $μ\in \mathcal{P}( \mathcal{P}(\mathcal{Y})^{\mathcal{X}})$. Using functoriality of probabilistic morphisms, we prove that sequential and batch Bayesian inversions coincide in supervised learning models with conditionally independent (possibly non-i.i.d.) data \cite{Le2025}. This equivalence holds without domination or discreteness assumptions on sampling operators. We derive a recursive formula for posterior predictive distributions, which reduces to the Kalman filter in Gaussian process regression. For Souslin label spaces $\mathcal{Y}$ and arbitrary input sets $\mathcal{X}$, we characterize probability measures on $\mathcal{P}(\mathcal{Y})^{\mathcal{X}}$ via projective systems, generalizing Orbanz \cite{Orbanz2011}. We revisit MacEachern's Dependent Dirichlet Processes (DDP) \cite{MacEachern2000} using copula-based constructions \cite{BJQ2012} and show how to compute posterior predictive distributions in universal Bayesian supervised models with DDP priors.