Structural determinants of soft memory in recurrent biological networks

Abstract

Recurrent neural networks are frequently studied in terms of their information-processing capabilities. The structural properties of these networks are seldom considered, beyond those emerging from the connectivity tuning necessary for network training. However, real biological networks have non-contingent architectures that have been shaped by evolution over eons, constrained partly by information-processing criteria, but more generally by fitness maximization requirements. Here we examine the topological properties of existing biological networks, focusing in particular on gene regulatory networks in bacteria. We identify structural features, both local and global, that dictate the ability of recurrent networks to store information on the fly and process complex time-dependent inputs.