Differential Dynamic Programming for time-delayed systems

Trajectory optimization considers the problem of deciding how to control a dynamical system to move along a trajectory which minimizes some cost function. Differential Dynamic Programming (DDP) is an optimal control method which utilizes a second-order approximation of the problem to find the control. It is fast enough to allow real-time control and has been shown to work well for trajectory optimization in robotic systems. Here we extend classic DDP to systems with multiple time-delays in the state. Being able to find optimal trajectories for time-delayed systems with DDP opens up the possibility to use richer models for system identification and control, including recurrent neural networks with multiple timesteps in the state. We demonstrate the algorithm on a two-tank continuous stirred tank reactor. We also demonstrate the algorithm on a recurrent neural network trained to model an inverted pendulum with position information only.Comment: 7 pages, 6 figures, conference, Decision and Control (CDC), 2016 IEEE 55th Conference o

Delay-dependent stabilization of stochastic interval delay systems with nonlinear disturbances

This is the post print version of the article. The official published version can be obtained from the link below - Copyright 2007 Elsevier Ltd.In this paper, a delay-dependent approach is developed to deal with the robust stabilization problem for a class of stochastic time-delay interval systems with nonlinear disturbances. The system matrices are assumed to be uncertain within given intervals, the time delays appear in both the system states and the nonlinear disturbances, and the stochastic perturbation is in the form of a Brownian motion. The purpose of the addressed stochastic stabilization problem is to design a memoryless state feedback controller such that, for all admissible interval uncertainties and nonlinear disturbances, the closed-loop system is asymptotically stable in the mean square, where the stability criteria are dependent on the length of the time delay and therefore less conservative. By using Itô's differential formula and the Lyapunov stability theory, sufficient conditions are first derived for ensuring the stability of the stochastic interval delay systems. Then, the controller gain is characterized in terms of the solution to a delay-dependent linear matrix inequality (LMI), which can be easily solved by using available software packages. A numerical example is exploited to demonstrate the effectiveness of the proposed design procedure.This work was supported in part by the Engineering and Physical Sciences Research Council (EPSRC) of the UK under Grant GR/S27658/01, the Nuffield Foundation of the UK under Grant NAL/00630/G, and the Alexander von Humboldt Foundation of Germany

Feedback control by online learning an inverse model

A model, predictor, or error estimator is often used by a feedback controller to control a plant. Creating such a model is difficult when the plant exhibits nonlinear behavior. In this paper, a novel online learning control framework is proposed that does not require explicit knowledge about the plant. This framework uses two learning modules, one for creating an inverse model, and the other for actually controlling the plant. Except for their inputs, they are identical. The inverse model learns by the exploration performed by the not yet fully trained controller, while the actual controller is based on the currently learned model. The proposed framework allows fast online learning of an accurate controller. The controller can be applied on a broad range of tasks with different dynamic characteristics. We validate this claim by applying our control framework on several control tasks: 1) the heating tank problem (slow nonlinear dynamics); 2) flight pitch control (slow linear dynamics); and 3) the balancing problem of a double inverted pendulum (fast linear and nonlinear dynamics). The results of these experiments show that fast learning and accurate control can be achieved. Furthermore, a comparison is made with some classical control approaches, and observations concerning convergence and stability are made

On general systems with network-enhanced complexities

In recent years, the study of networked control systems (NCSs) has gradually become an active research area due to the advantages of using networked media in many aspects such as the ease of maintenance and installation, the large flexibility and the low cost. It is well known that the devices in networks are mutually connected via communication cables that are of limited capacity. Therefore, some network-induced phenomena have inevitably emerged in the areas of signal processing and control engineering. These phenomena include, but are not limited to, network-induced communication delays, missing data, signal quantization, saturations, and channel fading. It is of great importance to understand how these phenomena influence the closed-loop stability and performance properties

Model predictive control based on LPV models with parameter-varying delays

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper presents a Model Predictive Control (MPC) strategy based on Linear Parameter Varying (LPV) models with varying delays affecting states and inputs. The proposed control approach allows the controller to accommodate the scheduling parameters and delay change. By computing the prediction of the state variables and delay along a prediction time horizon, the system model can be modified according to the evaluation of the estimated state and delay at each time instant. Moreover, the solution of the optimization problem associated with the MPC design is achieved by solving a series of Quadratic Programming (QP) problem at each time instant. This iterative approach reduces the computational burden compared to the solution of a non-linear optimization problem. A pasteurization plant system is used as a case study to demonstrate the effectiveness of the proposed approach.Peer ReviewedPostprint (author's final draft

Modeling and control of complex dynamic systems: Applied mathematical aspects

The concept of complex dynamic systems arises in many varieties, including the areas of energy generation, storage and distribution, ecosystems, gene regulation and health delivery, safety and security systems, telecommunications, transportation networks, and the rapidly emerging research topics seeking to understand and analyse. Such systems are often concurrent and distributed, because they have to react to various kinds of events, signals, and conditions. They may be characterized by a system with uncertainties, time delays, stochastic perturbations, hybrid dynamics, distributed dynamics, chaotic dynamics, and a large number of algebraic loops. This special issue provides a platform for researchers to report their recent results on various mathematical methods and techniques for modelling and control of complex dynamic systems and identifying critical issues and challenges for future investigation in this field. This special issue amazingly attracted one-hundred-and eighteen submissions, and twenty-eight of them are selected through a rigorous review procedure

Time-delayed models of gene regulatory networks

We discuss different mathematical models of gene regulatory networks as relevant to the onset and development of cancer. After discussion of alternativemodelling approaches, we use a paradigmatic two-gene network to focus on the role played by time delays in the dynamics of gene regulatory networks. We contrast the dynamics of the reduced model arising in the limit of fast mRNA dynamics with that of the full model. The review concludes with the discussion of some open problems