Noisy deep networks: chaos, multistationarity, and eternal evolution

Abstract

We study time-recurrent hierarchical networks that model complex systems in biology, economics, and ecology. These networks resemble real-world topologies, with strongly connected hubs (centers) and weakly connected nodes (satellites). Under natural structural assumptions, we develop a mean-field approach that reduces network dynamics to the central nodes alone. Even in the two-layer case, we establish universal dynamical approximation, demonstrating that these networks can replicate virtually any dynamical behavior by tuning center-satellite interactions. In multilayered networks, this property extends further, enabling the approximation of families of structurally stable systems and the emergence of complex bifurcations, such as pitchfork bifurcations under strong inter-satellite interactions. We also show that internal noise within nodes moderates bifurcations, leading to noise-induced phase transitions. A striking effect emerges where central nodes may lose control over satellites, akin to transitions observed in perceptrons studied by E. Gardner-relevant in complex combinatorial problems. Finally, we examine the networks’ responses to stress, demonstrating that increasing complexity during evolution is crucial for long-term viability