Optimal Control and Synchronization of Dynamic Ensemble Systems

Ensemble control involves the manipulation of an uncountably infinite collection of structurally identical or similar dynamical systems, which are indexed by a parameter set, by applying a common control without using feedback. This subject is motivated by compelling problems in quantum control, sensorless robotic manipulation, and neural engineering, which involve ensembles of linear, bilinear, or nonlinear oscillating systems, for which analytical control laws are infeasible or absent. The focus of this dissertation is on novel analytical paradigms and constructive control design methods for practical ensemble control problems. The first result is a computational method %based on the singular value decomposition (SVD) for the synthesis of minimum-norm ensemble controls for time-varying linear systems. This method is extended to iterative techniques to accommodate bounds on the control amplitude, and to synthesize ensemble controls for bilinear systems. Example ensemble systems include harmonic oscillators, quantum transport, and quantum spin transfers on the Bloch system. To move towards the control of complex ensembles of nonlinear oscillators, which occur in neuroscience, circadian biology, electrochemistry, and many other fields, ideas from synchronization engineering are incorporated. The focus is placed on the phenomenon of entrainment, which refers to the dynamic synchronization of an oscillating system to a periodic input. Phase coordinate transformation, formal averaging, and the calculus of variations are used to derive minimum energy and minimum mean time controls that entrain ensembles of non-interacting oscillators to a harmonic or subharmonic target frequency. In addition, a novel technique for taking advantage of nonlinearity and heterogeneity to establish desired dynamical structures in collections of inhomogeneous rhythmic systems is derived

Optimal Subharmonic Entrainment

For many natural and engineered systems, a central function or design goal is the synchronization of one or more rhythmic or oscillating processes to an external forcing signal, which may be periodic on a different time-scale from the actuated process. Such subharmonic synchrony, which is dynamically established when N control cycles occur for every M cycles of a forced oscillator, is referred to as N:M entrainment. In many applications, entrainment must be established in an optimal manner, for example by minimizing control energy or the transient time to phase locking. We present a theory for deriving inputs that establish subharmonic N:M entrainment of general nonlinear oscillators, or of collections of rhythmic dynamical units, while optimizing such objectives. Ordinary differential equation models of oscillating systems are reduced to phase variable representations, each of which consists of a natural frequency and phase response curve. Formal averaging and the calculus of variations are then applied to such reduced models in order to derive optimal subharmonic entrainment waveforms. The optimal entrainment of a canonical model for a spiking neuron is used to illustrate this approach, which is readily extended to arbitrary oscillating systems

Training Echo State Networks with Regularization through Dimensionality Reduction

In this paper we introduce a new framework to train an Echo State Network to predict real valued time-series. The method consists in projecting the output of the internal layer of the network on a space with lower dimensionality, before training the output layer to learn the target task. Notably, we enforce a regularization constraint that leads to better generalization capabilities. We evaluate the performances of our approach on several benchmark tests, using different techniques to train the readout of the network, achieving superior predictive performance when using the proposed framework. Finally, we provide an insight on the effectiveness of the implemented mechanics through a visualization of the trajectory in the phase space and relying on the methodologies of nonlinear time-series analysis. By applying our method on well known chaotic systems, we provide evidence that the lower dimensional embedding retains the dynamical properties of the underlying system better than the full-dimensional internal states of the network

Intrinsic adaptation in autonomous recurrent neural networks

A massively recurrent neural network responds on one side to input stimuli and is autonomously active, on the other side, in the absence of sensory inputs. Stimuli and information processing depends crucially on the qualia of the autonomous-state dynamics of the ongoing neural activity. This default neural activity may be dynamically structured in time and space, showing regular, synchronized, bursting or chaotic activity patterns. We study the influence of non-synaptic plasticity on the default dynamical state of recurrent neural networks. The non-synaptic adaption considered acts on intrinsic neural parameters, such as the threshold and the gain, and is driven by the optimization of the information entropy. We observe, in the presence of the intrinsic adaptation processes, three distinct and globally attracting dynamical regimes, a regular synchronized, an overall chaotic and an intermittent bursting regime. The intermittent bursting regime is characterized by intervals of regular flows, which are quite insensitive to external stimuli, interseeded by chaotic bursts which respond sensitively to input signals. We discuss these finding in the context of self-organized information processing and critical brain dynamics.Comment: 24 pages, 8 figure

Rhythmic dynamics and synchronization via dimensionality reduction : application to human gait

Reliable characterization of locomotor dynamics of human walking is vital to understanding the neuromuscular control of human locomotion and disease diagnosis. However, the inherent oscillation and ubiquity of noise in such non-strictly periodic signals pose great challenges to current methodologies. To this end, we exploit the state-of-the-art technology in pattern recognition and, specifically, dimensionality reduction techniques, and propose to reconstruct and characterize the dynamics accurately on the cycle scale of the signal. This is achieved by deriving a low-dimensional representation of the cycles through global optimization, which effectively preserves the topology of the cycles that are embedded in a high-dimensional Euclidian space. Our approach demonstrates a clear advantage in capturing the intrinsic dynamics and probing the subtle synchronization patterns from uni/bivariate oscillatory signals over traditional methods. Application to human gait data for healthy subjects and diabetics reveals a significant difference in the dynamics of ankle movements and ankle-knee coordination, but not in knee movements. These results indicate that the impaired sensory feedback from the feet due to diabetes does not influence the knee movement in general, and that normal human walking is not critically dependent on the feedback from the peripheral nervous system

Control and Synchronization of Neuron Ensembles

Synchronization of oscillations is a phenomenon prevalent in natural, social, and engineering systems. Controlling synchronization of oscillating systems is motivated by a wide range of applications from neurological treatment of Parkinson's disease to the design of neurocomputers. In this article, we study the control of an ensemble of uncoupled neuron oscillators described by phase models. We examine controllability of such a neuron ensemble for various phase models and, furthermore, study the related optimal control problems. In particular, by employing Pontryagin's maximum principle, we analytically derive optimal controls for spiking single- and two-neuron systems, and analyze the applicability of the latter to an ensemble system. Finally, we present a robust computational method for optimal control of spiking neurons based on pseudospectral approximations. The methodology developed here is universal to the control of general nonlinear phase oscillators.Comment: 29 pages, 6 figure

Multi-Robot Transfer Learning: A Dynamical System Perspective

Multi-robot transfer learning allows a robot to use data generated by a second, similar robot to improve its own behavior. The potential advantages are reducing the time of training and the unavoidable risks that exist during the training phase. Transfer learning algorithms aim to find an optimal transfer map between different robots. In this paper, we investigate, through a theoretical study of single-input single-output (SISO) systems, the properties of such optimal transfer maps. We first show that the optimal transfer learning map is, in general, a dynamic system. The main contribution of the paper is to provide an algorithm for determining the properties of this optimal dynamic map including its order and regressors (i.e., the variables it depends on). The proposed algorithm does not require detailed knowledge of the robots' dynamics, but relies on basic system properties easily obtainable through simple experimental tests. We validate the proposed algorithm experimentally through an example of transfer learning between two different quadrotor platforms. Experimental results show that an optimal dynamic map, with correct properties obtained from our proposed algorithm, achieves 60-70% reduction of transfer learning error compared to the cases when the data is directly transferred or transferred using an optimal static map.Comment: 7 pages, 6 figures, accepted at the 2017 IEEE/RSJ International Conference on Intelligent Robots and System

Comparative evaluation of approaches in T.4.1-4.3 and working definition of adaptive module

The goal of this deliverable is two-fold: (1) to present and compare different approaches towards learning and encoding movements us- ing dynamical systems that have been developed by the AMARSi partners (in the past during the first 6 months of the project), and (2) to analyze their suitability to be used as adaptive modules, i.e. as building blocks for the complete architecture that will be devel- oped in the project. The document presents a total of eight approaches, in two groups: modules for discrete movements (i.e. with a clear goal where the movement stops) and for rhythmic movements (i.e. which exhibit periodicity). The basic formulation of each approach is presented together with some illustrative simulation results. Key character- istics such as the type of dynamical behavior, learning algorithm, generalization properties, stability analysis are then discussed for each approach. We then make a comparative analysis of the different approaches by comparing these characteristics and discussing their suitability for the AMARSi project