Inference from correlated patterns: a unified theory for perceptron learning and linear vector channels

Abstract

A framework to analyze inference performance in densely connected single-layer feed-forward networks is developed for situations where a given data set is composed of correlated patterns. The framework is based on the assumption that the left and right singular value bases of the given pattern matrix are generated independently and uniformly from Haar measures. This assumption makes it possible to characterize the objective system by a single function of two variables which is determined by the eigenvalue spectrum of the cross-correlation matrix of the pattern matrix. Links to existing methods for analysis of perceptron learning and Gaussian linear vector channels and an application to a simple but nontrivial problem are also shown.