Density of Neural Network Classes on Compact Subsets of Topological Vector Spaces

Abstract

We prove density results for neural-network classes on compact sets \(K\subset X\), where \(X\) is a topological vector space whose continuous dual \(X^*\) separates points. Let \(Ψ:\mathbb R\to\mathbb R\) be a continuous squashing function. We show that the class \[ Σ_X(Ψ) = \left\{ \sum_{j=1}^{N}ω_jΨ(f_j(x)+b_j): N\in\mathbb N,\ ω_j,b_j\in\mathbb R,\ f_j\in X^* \right\} \] is dense in \(C(K)\) with respect to the uniform norm. As a consequence, if \(μ\) is a Radon probability measure supported on \(K\), then \(Σ_X(Ψ)\) is dense in \(L^p(K,μ)\) for every \(1\le p<\infty\).