Learning Over All Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems

This paper presents a policy parameterization for learning-based control on nonlinear, partially-observed dynamical systems. The parameterization is based on a nonlinear version of the Youla parameterization and the recently proposed Recurrent Equilibrium Network (REN) class of models. We prove that the resulting Youla-REN parameterization automatically satisfies stability (contraction) and user-tunable robustness (Lipschitz) conditions on the closed-loop system. This means it can be used for safe learning-based control with no additional constraints or projections required to enforce stability or robustness. We test the new policy class in simulation on two reinforcement learning tasks: 1) magnetic suspension, and 2) inverting a rotary-arm pendulum. We find that the Youla-REN performs similarly to existing learning-based and optimal control methods while also ensuring stability and exhibiting improved robustness to adversarial disturbances

Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear System

This paper proposes a nonlinear policy architecture for control of partially-observed linear dynamical systems providing built-in closed-loop stability guarantees. The policy is based on a nonlinear version of the Youla parameterization, and augments a known stabilizing linear controller with a nonlinear operator from a recently developed class of dynamic neural network models called the recurrent equilibrium network (REN). We prove that RENs are universal approximators of contracting and Lipschitz nonlinear systems, and subsequently show that the the proposed Youla-REN architecture is a universal approximator of stabilizing nonlinear controllers. The REN architecture simplifies learning since unconstrained optimization can be applied, and we consider both a model-based case where exact gradients are available and reinforcement learning using random search with zeroth-order oracles. In simulation examples our method converges faster to better controllers and is more scalable than existing methods, while guaranteeing stability during learning transients

Guaranteed safe switching for switching adaptive control

Adaptive control algorithms may not behave well in practice due to discrepancies between the theory and actual practice. The proposed results in this manuscript constitute an effort in providing algorithms which assure more reliable operation in practice. Our emphasis is on algorithms that will be safe in the sense of not permitting destabilizing controllers to be switched in the closed-loop and to prevent wild signal fluctuations to occur. Coping with the connection or possible connection of destabilizing controllers is indeed a daunting task. One of the most intuitive forms of adaptive control, gain scheduling, is an approach to control of non-linear systems which utilizes a family of linear controllers, each of which provides satisfactory control for a different operating point of the system. We provide a mechanism for guaranteeing closed-loop stability over rapid switching between controllers. Our proposed design provides a simplification using only finite number of pre-determined values for the controller gain, where the observer gain is computed via a table look-up method. In comparison to the original gain scheduling design which our procedure builds on, our design achieves similar performance but with much less computational burden. Many multi-controller adaptive switching algorithms do not explicitly rule out the possibility of switching a destabilizing controller into the closed-loop. Even if the new controller is ensured to be stabilizing, performance verification with the new controller is not straightforward. The importance of this arises in iterative identification and control algorithms and multiple model adaptive control (MMAC). We utilize a limited amount of experimental and possibly noisy data obtained from a closed-loop consisting of an existing known stabilizing controller connected to an unknown plant-to infer if the introduction of a prospective controller will stabilize the unknown plant. We propose analysis results in a nonlinear setting and provide data-based tests for verifying the closed-loop stability with the introduction of a new nonlinear controller to replace a linear controller. We also propose verification tools for the closed-loop performance with the introduction of a new stabilizing controller using a limited amount of data obtained from the existing stable closed-loop. The simulation results in different practical scenarios demonstrate efficacy and versatility of our results, and illustrate practicality of our novel data-based tests in addressing an instability problem in adaptive control algorithms

Robust Control

The need to be tolerant to changes in the control systems or in the operational environment of systems subject to unknown disturbances has generated new control methods that are able to deal with the non-parametrized disturbances of systems, without adapting itself to the system uncertainty but rather providing stability in the presence of errors bound in a model. With this approach in mind and with the intention to exemplify robust control applications, this book includes selected chapters that describe models of H-infinity loop, robust stability and uncertainty, among others. Each robust control method and model discussed in this book is illustrated by a relevant example that serves as an overview of the theoretical and practical method in robust control