A Study of Machine Learning Techniques for Dynamical System Prediction

Dynamical Systems are ubiquitous in mathematics and science and have been used to model many important application problems such as population dynamics, fluid flow, and control systems. However, some of them are challenging to construct from the traditional mathematical techniques. To combat such problems, various machine learning techniques exist that attempt to use collected data to form predictions that can approximate the dynamical system of interest. This thesis will study some basic machine learning techniques for predicting system dynamics from the data generated by test systems. In particular, the methods of Dynamic Mode Decomposition (DMD), Sparse Identification of Nonlinear Dynamics (SINDy), Singular Value Decomposition (SVD), and Deep Neural Network (DNN) regression will be studied. Such techniques provide alternatives to determine the dynamics of a system of interest without needing to resort to the computationally expensive elementary methods. From numerically testing a few linear and nonlinear systems of ordinary differential equations, it was observed that the methods of DMD and SVD could approximate linear systems effectively but performed poorly against nonlinear systems. The approach of DNN regression proved effective for both linear and nonlinear dynamical systems

Koopman Operator Theory and The Applied Perspective of Modern Data-Driven Systems

Recent theoretical developments in dynamical systems and machine learning have allowed researchers to re-evaluate how dynamical systems are modeled and controlled. In this thesis, Koopman operator theory is used to model dynamical systems and obtain optimal control solutions for nonlinear systems using sampled system data. The Koopman operator is obtained using data generated from a real physical system or from an analytical model which describes the physical system under nominal conditions. One of the critical advantages of the Koopman operator is that the response of the nonlinear system can be obtained from an equivalent infinite dimensional linear system. This is achieved by exploiting the topological structure associated with the spectrum of the Koopman operator and the Koopman eigenfunctions. The main contributions of this thesis are threefold. First, we provide a data-driven approach for system identification, and a model-based approach for obtaining an analytic change of coordinates associated with the principle Koopman eigenfunctions for systems with hyperbolic equilibrium points. A new derivation of the Hamilton-Jacobi equations associated with the infinite time horizon nonlinear optimal control problem is obtained using the Koopman generator. Then, a learning algorithm called Koopman Policy Iteration is used to obtain the solution to the infinite horizon nonlinear optimal fixed point regulation problem without state and input constraints. Finally, the finite time nonlinear optimal control problem with state and input constraints is solved using a receding horizon optimization approach called dual mode model predictive control using Koopman eigenfunctions. Evidence supporting the convergence of these methods are provided using analytical examples

Learning Stable Koopman Models for Identification and Control of Dynamical Systems

Learning models of dynamical systems from data is a widely-studied problem in control theory and machine learning. One recent approach for modelling nonlinear systems considers the class of Koopman models, which embeds the nonlinear dynamics in a higher-dimensional linear subspace. Learning a Koopman embedding would allow for the analysis and control of nonlinear systems using tools from linear systems theory. Many recent methods have been proposed for data-driven learning of such Koopman embeddings, but most of these methods do not consider the stability of the Koopman model. Stability is an important and desirable property for models of dynamical systems. Unstable models tend to be non-robust to input perturbations and can produce unbounded outputs, which are both undesirable when the model is used for prediction and control. In addition, recent work has shown that stability guarantees may act as a regularizer for model fitting. As such, a natural direction would be to construct Koopman models with inherent stability guarantees. Two new classes of Koopman models are proposed that bridge the gap between Koopman-based methods and learning stable nonlinear models. The first model class is guaranteed to be stable, while the second is guaranteed to be stabilizable with an explicit stabilizing controller that renders the model stable in closed-loop. Furthermore, these models are unconstrained in their parameter sets, thereby enabling efficient optimization via gradient-based methods. Theoretical connections between the stability of Koopman models and forms of nonlinear stability such as contraction are established. To demonstrate the effect of the stability guarantees, the stable Koopman model is applied to a system identification problem, while the stabilizable model is applied to an imitation learning problem. Experimental results show empirically that the proposed models achieve better performance over prior methods without stability guarantees

Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation

The control of nonlinear dynamical systems remains a major challenge for autonomous agents. Current trends in reinforcement learning (RL) focus on complex representations of dynamics and policies, which have yielded impressive results in solving a variety of hard control tasks. However, this new sophistication and extremely over-parameterized models have come with the cost of an overall reduction in our ability to interpret the resulting policies. In this paper, we take inspiration from the control community and apply the principles of hybrid switching systems in order to break down complex dynamics into simpler components. We exploit the rich representational power of probabilistic graphical models and derive an expectation-maximization (EM) algorithm for learning a sequence model to capture the temporal structure of the data and automatically decompose nonlinear dynamics into stochastic switching linear dynamical systems. Moreover, we show how this framework of switching models enables extracting hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts in an imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro