Optimal Sobolev Approximation by Deterministic and Random Shallow Sigmoidal Networks

Abstract

Shallow networks with prescribed or randomly sampled hidden parameters are widely used as numerical trial spaces, yet their optimal Sobolev approximation power with standard smooth sigmoidal activations in general dimension remains unresolved. We establish the corresponding optimal rates for a class of smooth sigmoidal activations with Schwartz-class derivative decay, including $\tanh$, the logistic sigmoid, and the error function $erf$. We first construct deterministic direction--offset dictionaries with $M$ features such that every $u\in H^k(Ω)$ can be approximated with error of order $M^{-(k-m)/d}$ in $H^m(Ω)$ for all $0\le m\le k$. This rate is optimal in the sense of Kolmogorov widths for Sobolev balls. We further prove that dictionaries obtained by independent parameter sampling from any prescribed density bounded away from zero attain the same approximation exponent with high probability, up to logarithmic oversampling. The analysis develops a sigmoidal ridge representation and combines it with deterministic or probabilistic quadrature in direction--offset space while retaining polynomial control of the output coefficients. Numerical experiments across a broad range of dimensions, target regularities, and Sobolev error norms recover the predicted algebraic rates for both deterministic and random feature dictionaries.