A perturbative approach to non-linearities in the information carried by a two layer neural network
E. Korutcheva, V. Del Prete, J. -P. Nadal
Abstract
We evaluate the mutual information between the input and the output of a two layer network in the case of a noisy and non-linear analogue channel. In the case where the non-linearity is small with respect to the variability in the noise, we derive an exact expression for the contribution to the mutual information given by the non-linear term in first order of perturbation theory. Finally we show how the calculation can be simplified by means of a diagrammatic expansion. Our results suggest that the use of perturbation theories applied to neural systems might give an insight on the contribution of non-linearities to the information transmission and in general to the neuronal dynamics.
Create a lesson
Related papers
Absolute area under a scaled Brownian motion with resetting
Vicenç Méndez, Javier Cristín
Understanding the Mpemba Effect through a Rule of Thumb
Ran Yaacoby
Beyond the lemon: Spectra of random Lindblad operators with a Hamiltonian component
Mykhailo Hontarenko, Wojciech Tarnowski, Sergey Denisov et al.
Towards synergy between theory and experiments: First-passage processes under resetting with stochastic return protocols
Ashutosh Dubey, Arnab Pal
Emergent Loop Geometry in Gauge-Equivariant Graph Neural Networks
Ali Rayat, Gia-Wei Chern
Extending the mean first-passage-time formalism to the reconstruction of high free-energy barriers: A top-start approach
Ying Jiang, Ivan Saika-Voivod