Unveiling the role of plasticity rules in reservoir computing

Reservoir Computing (RC) is an appealing approach in Machine Learning that combines the high computational capabilities of Recurrent Neural Networks with a fast and easy training method. Likewise, successful implementation of neuro-inspired plasticity rules into RC artificial networks has boosted the performance of the original models. In this manuscript, we analyze the role that plasticity rules play on the changes that lead to a better performance of RC. To this end, we implement synaptic and non-synaptic plasticity rules in a paradigmatic example of RC model: the Echo State Network. Testing on nonlinear time series prediction tasks, we show evidence that improved performance in all plastic models are linked to a decrease of the pair-wise correlations in the reservoir, as well as a significant increase of individual neurons ability to separate similar inputs in their activity space. Here we provide new insights on this observed improvement through the study of different stages on the plastic learning. From the perspective of the reservoir dynamics, optimal performance is found to occur close to the so-called edge of instability. Our results also show that it is possible to combine different forms of plasticity (namely synaptic and non-synaptic rules) to further improve the performance on prediction tasks, obtaining better results than those achieved with single-plasticity models

Nanophotonic reservoir computing with photonic crystal cavities to generate periodic patterns

Reservoir computing (RC) is a technique in machine learning inspired by neural systems. RC has been used successfully to solve complex problems such as signal classification and signal generation. These systems are mainly implemented in software, and thereby they are limited in speed and power efficiency. Several optical and optoelectronic implementations have been demonstrated, in which the system has signals with an amplitude and phase. It is proven that these enrich the dynamics of the system, which is beneficial for the performance. In this paper, we introduce a novel optical architecture based on nanophotonic crystal cavities. This allows us to integrate many neurons on one chip, which, compared with other photonic solutions, closest resembles a classical neural network. Furthermore, the components are passive, which simplifies the design and reduces the power consumption. To assess the performance of this network, we train a photonic network to generate periodic patterns, using an alternative online learning rule called first-order reduced and corrected error. For this, we first train a classical hyperbolic tangent reservoir, but then we vary some of the properties to incorporate typical aspects of a photonics reservoir, such as the use of continuous-time versus discrete-time signals and the use of complex-valued versus real-valued signals. Then, the nanophotonic reservoir is simulated and we explore the role of relevant parameters such as the topology, the phases between the resonators, the number of nodes that are biased and the delay between the resonators. It is important that these parameters are chosen such that no strong self-oscillations occur. Finally, our results show that for a signal generation task a complex-valued, continuous-time nanophotonic reservoir outperforms a classical (i.e., discrete-time, real-valued) leaky hyperbolic tangent reservoir (normalized root-mean-square errors = 0.030 versus NRMSE = 0.127)

Comparative evaluation of approaches in T.4.1-4.3 and working definition of adaptive module

The goal of this deliverable is two-fold: (1) to present and compare different approaches towards learning and encoding movements us- ing dynamical systems that have been developed by the AMARSi partners (in the past during the first 6 months of the project), and (2) to analyze their suitability to be used as adaptive modules, i.e. as building blocks for the complete architecture that will be devel- oped in the project. The document presents a total of eight approaches, in two groups: modules for discrete movements (i.e. with a clear goal where the movement stops) and for rhythmic movements (i.e. which exhibit periodicity). The basic formulation of each approach is presented together with some illustrative simulation results. Key character- istics such as the type of dynamical behavior, learning algorithm, generalization properties, stability analysis are then discussed for each approach. We then make a comparative analysis of the different approaches by comparing these characteristics and discussing their suitability for the AMARSi project

ESTIMATION AND CONTROL OF NONLINEAR SYSTEMS: MODEL-BASED AND MODEL-FREE APPROACHES

State estimation and subsequent controller design for a general nonlinear system is an important problem that have been studied over the past decades. Many applications, e.g., atmospheric and oceanic sampling or lift control of an airfoil, display strongly nonlinear dynamics with very high dimensionality. Some of these applications use smaller underwater or aerial sensing platforms with insufficient on-board computation power to use a Monte-Carlo approach of particle filters. Hence, they need a computationally efficient filtering method for state-estimation without a severe penalty on the performance. On the other hand, the difficulty of obtaining a reliable model of the underlying system, e.g., a high-dimensional fluid dynamical environment or vehicle flow in a complex traffic network, calls for the design of a data-driven estimation and controller when abundant measurements are present from a variety of sensors. This dissertation places these problems in two broad categories: model-based and model-free estimation and output feedback. In the first part of the dissertation, a semi-parametric method with Gaussian mixture model (GMM) is used to approximate the unknown density of states. Then a Kalman filter and its nonlinear variants are employed to propagate and update each Gaussian mode with a Bayesian update rule. The linear observation model permits a Kalman filter covariance update for each Gaussian mode. The estimation error is shown to be stochastically bounded and this is illustrated numerically. The estimate is used in an observer-based feedback control to stabilize a general closed-loop system. A transferoperator- based approach is then proposed for the motion update for Bayesian filtering of a nonlinear system. A finite-dimensional approximation of the Perron-Frobenius (PF) operator yields a method called constrained Ulam dynamic mode decomposition (CUDMD). This algorithm is applied for output feedback of a pitching airfoil in unsteady flow. For the second part, an echo-state network (ESN) based approach equipped with an ensemble Kalman filter is proposed for data-driven estimation of a nonlinear system from a time series. A random reservoir of recurrent neural connections with the echo-state property (ESP) is trained from a time-series data. It is then used as a model-predictor for an ensemble Kalman filter for sparse estimation. The proposed data-driven estimation method is applied to predict the traffic flow from a set of mobility data of the UMD campus. A data-driven model-identification and controller design is also developed for control-affine nonlinear systems that are ubiquitous in several aerospace applications. We seek to find an approximate linear/bilinear representation of these nonlinear systems from data using the extended dynamic mode decomposition algorithm (EDMD) and apply Liealgebraic methods to analyze the controllability and design a controller. The proposed method utilizes the Koopman canonical transform (KCT) to approximate the dynamics into a bilinear system (Koopman bilinear form) under certain assumptions. The accuracy of this approximation is then analytically justified with the universal approximation property of the Koopman eigenfunctions. The resulting bilinear system is then subjected to controllability analysis using the Myhill semigroup and Lie algebraic structures, and a fixed endpoint optimal controller is designed using the Pontryagin’s principle

Learning with precise spike times : a new decoding algorithm for liquid state machines

There is extensive evidence that biological neural networks encode information in the precise timing of the spikes generated and transmitted by neurons, which offers several advantages over rate-based codes. Here we adopt a vector space formulation of spike train sequences and introduce a new liquid state machine (LSM) network architecture and a new forward orthogonal regression algorithm to learn an input-output signal mapping or to decode the brain activity. The proposed algorithm uses precise spike timing to select the presynaptic neurons relevant to each learning task. We show that using precise spike timing to train the LSM and selecting the readout presynaptic neurons leads to a significant increase in performance on binary classification tasks, in decoding neural activity from multielectrode array recordings, as well as in a speech recognition task, compared with what is achieved using the standard architecture and training methods

Dynamical Systems in Spiking Neuromorphic Hardware

Dynamical systems are universal computers. They can perceive stimuli, remember, learn from feedback, plan sequences of actions, and coordinate complex behavioural responses. The Neural Engineering Framework (NEF) provides a general recipe to formulate models of such systems as coupled sets of nonlinear differential equations and compile them onto recurrently connected spiking neural networks – akin to a programming language for spiking models of computation. The Nengo software ecosystem supports the NEF and compiles such models onto neuromorphic hardware. In this thesis, we analyze the theory driving the success of the NEF, and expose several core principles underpinning its correctness, scalability, completeness, robustness, and extensibility. We also derive novel theoretical extensions to the framework that enable it to far more effectively leverage a wide variety of dynamics in digital hardware, and to exploit the device-level physics in analog hardware. At the same time, we propose a novel set of spiking algorithms that recruit an optimal nonlinear encoding of time, which we call the Delay Network (DN). Backpropagation across stacked layers of DNs dramatically outperforms stacked Long Short-Term Memory (LSTM) networks—a state-of-the-art deep recurrent architecture—in accuracy and training time, on a continuous-time memory task, and a chaotic time-series prediction benchmark. The basic component of this network is shown to function on state-of-the-art spiking neuromorphic hardware including Braindrop and Loihi. This implementation approaches the energy-efficiency of the human brain in the former case, and the precision of conventional computation in the latter case