A Stochastic Gradient Descent Theorem and the Back-Propagation Algorithm

Abstract

We establish a convergence theorem for a certain type of stochastic gradient descent, which leads to a convergent variant of the back-propagation algorithm