Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Among the various architectures of Recurrent Neural Networks, Echo State Networks (ESNs) emerged due to their simplified and inexpensive training procedure. These networks are known to be sensitive to the setting of hyper-parameters, which critically affect their behaviour. Results show that their performance is usually maximized in a narrow region of hyper-parameter space called edge of chaos. Finding such a region requires searching in hyper-parameter space in a sensible way: hyper-parameter configurations marginally outside such a region might yield networks exhibiting fully developed chaos, hence producing unreliable computations. The performance gain due to optimizing hyper-parameters can be studied by considering the memory--nonlinearity trade-off, i.e., the fact that increasing the nonlinear behavior of the network degrades its ability to remember past inputs, and vice-versa. In this paper, we propose a model of ESNs that eliminates critical dependence on hyper-parameters, resulting in networks that provably cannot enter a chaotic regime and, at the same time, denotes nonlinear behaviour in phase space characterised by a large memory of past inputs, comparable to the one of linear networks. Our contribution is supported by experiments corroborating our theoretical findings, showing that the proposed model displays dynamics that are rich-enough to approximate many common nonlinear systems used for benchmarking

Physics-Guided Deep Learning for Dynamical Systems: A survey

Modeling complex physical dynamics is a fundamental task in science and engineering. Traditional physics-based models are interpretable but rely on rigid assumptions. And the direct numerical approximation is usually computationally intensive, requiring significant computational resources and expertise. While deep learning (DL) provides novel alternatives for efficiently recognizing complex patterns and emulating nonlinear dynamics, it does not necessarily obey the governing laws of physical systems, nor do they generalize well across different systems. Thus, the study of physics-guided DL emerged and has gained great progress. It aims to take the best from both physics-based modeling and state-of-the-art DL models to better solve scientific problems. In this paper, we provide a structured overview of existing methodologies of integrating prior physical knowledge or physics-based modeling into DL and discuss the emerging opportunities

Control of subsonic cavity flows by neural networks -Analytical models and experimental validation

Flow control is attracting an increasing attention of researchers from a wide spectrum of specialties because of its interdisciplinary nature and the associated challenges. One of the main goals of The Collaborative Center of Control Science at The Ohio State University is to bring together researchers from different disciplines to advance the science and technology of flow control. This paper approaches the control of subsonic cavity flow, a study case we have selected, from a computational intelligence point of view, and offers a solution that displays an interconnected neural architecture. The structures of identification and control, together with the experimental implementation are discussed. The model and the controller have very simple structural configurations indicating that a significant saving on computation is possible. Experimental testing of a neural emulator and of a directly-synthesized neurocontroller indicates that the emulator can accurately reproduce a reference signal measured in the cavity floor under different operating conditions. Based on preliminary results, the neurocontroller appears to be marginally effective and produces spectral peak reductions analogous to those previously observed by the authors using linearcontrol techniques. The current research will continue to improve the capability of the neural emulator and of the neurocontroller