The role of parameter Jacobians in the stability of network outputs

Abstract

In the framework of network dynamics, learning models, and neural tangent kernels (NTK), we show that the corresponding linearized dynamics leads naturally to a semigroup formulation. More precisely, in our analysis of input/output models, the time-dynamics is presented via special semigroups of linear operators on Hilbert spaces, together with an associated class of semigroup perturbations. In this context, we then present new and explicit a priori perturbation-bound results: for the fixed-kernel linearization constructions arising in the NTK setting, we prove norm-bounds on the corresponding semigroup perturbations, in the form of explicit finite-time perturbation estimates. We further present refinements on prescribed task spaces, Cesàro-averaged (ergodic) comparisons estimates, and versions in which the lower spectral edge assumption is replaced by a spectral-distribution condition. We also extend the comparison to nonautonomous NTK evolutions through piecewise-frozen approximations, record a corresponding discrete Euler specialization, and offer worked examples in order to illustrate our perturbation-bound estimates.