Mean-field theory of learning: from dynamics to statics

Abstract

Using the cavity method and diagrammatic methods, we model the dynamics of batch learning of restricted sets of examples. Simulations of the Green's function and the cavity activation distributions support the theory well. The learning dynamics approaches a steady state in agreement with the static version of the cavity method. The picture of the rough energy landscape is reviewed.