On-line learning in a discrete state space

Abstract

On-line learning of a rule given by an N-dimensional Ising perceptron, is considered for the case when the student is constrained to take values in a discrete state space of size $L^N$. For L=2 no on-line algorithm can achieve a finite overlap with the teacher in the thermodynamic limit. However, if $L$ is on the order of $\sqrt{N}$, Hebbian learning does achieve a finite overlap.