A mathematical analysis of hierarchical Hopfield models

Abstract

The central question that we address is: How can structured information be stored in a hierarchical Hopfield model involving hidden layers? To this end, we develop a formalism of strokes and concepts that allows us to appropriately structure information: initial features are first classified into strokes, which in a second step are aggregated into concepts. We rigorously derive criteria under which concepts can be retrieved from noisy input data. A remarkable effect is that we do not require a perfect retrieval at the level of strokes, as the second-layer retrieval procedure compensates for first-layer errors. We treat separately the cases of fixed and variable-sized concepts.