Mean-field theory of learning: from dynamics to statics
K. Y. Michael Wong, S. Li, Peixun Luo
Abstract
Using the cavity method and diagrammatic methods, we model the dynamics of batch learning of restricted sets of examples. Simulations of the Green's function and the cavity activation distributions support the theory well. The learning dynamics approaches a steady state in agreement with the static version of the cavity method. The picture of the rough energy landscape is reviewed.
Create a lesson
Related papers
The Stone-Wales trefoil: a new topological defect in graphene
M. V. Kondrin, Y. B. Lebed, V. V. Brazhkin
Short-Range Vector Spin Glasses: Parisi Hierarchies and Real-Space Ultrametricity
Akash Vijay, Jaewon Kim, Jong Yeon Lee
Lyapunov spectrum of random neural networks
David G. Clark
Revealing more on complex energy landscapes by passing less local information: Cavity approach with trust region in non-convex optimization problems
Zhiqi Xu, Chi Ho Yeung
Predicting activation-barrier and plasticity-onset statistics in a model of glasses
Makoto Suda, Edan Lerner, Eran Bouchbinder
Classifications in modular restricted Boltzmann machines
Elena Agliari, Andrea Lepre, Edoardo Roscani