A fast memoryless predictive algorithm in a chain of recurrent neural networks

Abstract

In the recent publication (arxiv:2007.08063v2 [cs.LG]) a fast prediction algorithm for a single recurrent network (RN) was suggested. In this manuscript we generalize this approach to a chain of RNs and show that it can be implemented in natural neural systems. When the network is used recursively to predict sequence of values the proposed algorithm does not require to store the original input sequence. It increases robustness of the new approach compared to the standard moving/expanding window predictive procedure. We consider requirements on trained networks that allow to implement the proposed algorithm and discuss them in the neuroscience context.