Stochastic model predictive control for constrained networked control systems with random time delay

In this paper the continuous time stochastic constrained optimal control problem is formulated for the class of networked control systems assuming that time delays follow a discrete-time, finite Markov chain . Polytopic overapproximations of the system's trajectories are employed to produce a polyhedral inner approximation of the non-convex constraint set resulting from imposing the constraints in continuous time. The problem is cast in a Markov jump linear systems (MJLS) framework and a stochastic MPC controller is calculated explicitly, oine, coupling dynamic programming with parametric piecewise quadratic (PWQ) optimization. The calculated control law leads to stochastic stability of the closed loop system, in the mean square sense and respects the state and input constraints in continuous time

Approximate Dynamic Programming for Constrained Piecewise Affine Systems with Stability and Safety Guarantees

Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this paper, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate non-convex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the sub-optimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer linear programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time

Approximate Explicit MPC and Closed-loop Stability: Analysis based on PWA Lyapunov Functions

Model Predictive Control (MPC) is the de facto standard in advanced industrial automation systems. There are two main formulations of the MPC algorithm: an implicit one and an explicit MPC one. The first requires an optimization problem to be solved on-line, which is the main limitation when dealing with hard real-time applications. As the implicit MPC algorithm cannot be guaran- teed in terms of execution time, in many applications the explicit MPC solution is preferable. In order to deal with systems integrating mixed logic and dynam- ics, the class of the hybrid and piecewise affine models (PWA) were introduced and tackled by the explicit MPC strategy. However, the resulting controller complexity leads to a requirement on the CPU/memory combination which is as strict as the number of states, inputs and outputs increases. To reduce drasti- cally the complexity of the explicit controller while preserving the controller’s performance, a strategy combining switched MPC with discontinuous simpli- cial PWA models is introduced in this thesis. The latter is proven to be circuit implementable, e.g., in FPGA. To ensure that closed-loop stability properties are guaranteed, a stability analysis tool is proposed which exploits suitable and possibly discontinuous PWA Lyapunov-like functions. The tool requires solving offline a linear programming problem. Moreover, the tool is able to compute an invariant set for the closed-loop system, as well as ultimate boundedness and input-to-state stability properties

Event-triggered control for piecewise affine discrete-time systems

In the present work, we study the problems of stability analysis of piecewise-affine (PWA) discrete-time systems, and trigger-function design for discrete-time event-triggered control systems. We propose a representation for piecewise-affine systems in terms of ramp functions, and we rely on Lyapunov theory for the stability analysis. The proposed implicit piecewise-affine representation prevents the shortcomings of the existing stability analysis approaches of PWA systems. Namely, the need to enumerate regions and allowed transitions of the explicit representations. In this context, we can emphasize two benefits of the proposed approach: first, it makes possible the analysis of uncertainty in the partition and, thus, the transitions. Secondly, it enables the analysis of event-triggered control systems for the class of PWA systems since, for ETC, the transitions cannot be determined as a function of the state variables. The proposed representation, on the other hand, implicitly encodes the partition and the transitions. The stability analysis is performed with Lyapunov theory techniques. We then present conditions for exponential stability. Thanks to the implicit representation, the use of piecewise quadratic Lyapunov functions candidates becomes simple. These conditions can be solved numerically using a linear matrix inequality formulation. The numerical analysis exploits quadratic expressions that describe ramp functions to verify the positiveness of extended quadratic forms. For ETC, a piecewise quadratic trigger function defines the event generator. We find suitable parameters for the trigger function with an optimization procedure. As a result, this function uses the information on the partition to reduce the number of events, achieving better results than the standard quadratic trigger functions found in the literature. We provide numerical examples to illustrate the application of the proposed representation and methods.Ce manuscrit présente des résultats sur l’analyse de stabilité des systèmes affines par morceaux en temps discret et sur le projet de fonctions de déclenchement pour des stratégies de commande par événements. Nous proposons une représentation pour des systèmes affines par morceaux et l’on utilise la théorie de stabilité de Lyapunov pour effectuer l’analyse de stabilité globale de l’origine. La nouvelle représentation implicite que nous proposons rend plus simple l’analyse de stabilité car elle évite l’énumération des régions et des transitions entre régions tel que c’est fait dans le cas des représentations explicites. Dans ce contexte nous pouvons souligner deux avantages principaux, à savoir I) la possibilité de traiter des incertitudes dans la partition qui définit le système et, par conséquent des incertitudes dans les transitions, II) l’analyse des stratégies de commande par événements pour des systèmes affines par morceaux. En effet, dans ces stratégies les transitions ne peuvent pas être définies comme des fonctions des variables d’état. La théorie de stabilité de Lyapunov est utilisée pour établir des conditions pour la stabilité exponentielle de l’origine. Grâce à la représentation implicite des partitions nous utilisons des fonctions de Lyapunov quadratique par morceaux. Ces conditions sont données par des inégalités dont la solution numérique est possible avec une formulation par des inégalités matricielles linéaires. Ces formulations numériques se basent sur des expressions quadratiques décrivant des fonctions rampe. Pour des stratégies par événement, une fonctions quadratique par morceaux est utilisée pour le générateur d’événements. Nous calculons les paramètres de ces fonctions de déclenchement a partir de solutions de problèmes d’optimisation. Cette fonction de déclenchement quadratique par morceaux permet de réduire le nombre de d’événementsen comparaison avec les fonctions quadratiques utilisées dans la littérature. Nous utilisons des exemples numériques pour illustrer les méthodes proposées.No presente trabalho, são estudados os problemas de análise de estabilidade de sistemas afins por partes e o projeto da função de disparo para sistemas de controle baseado em eventos em tempo discreto. É proposta uma representação para sistemas afins por partes em termos de funções rampa, e é utilizada a teoria de Lyapunov para a análise de estabilidade. A representação afim por partes implícita proposta evita algumas das deficiências das abordagens de análise de estabilidade de sistemas afins por partes existentes. Em particular, a necessidade de anumerar regiões e transições admissíveis das representações explícitas. Neste contexto, dois benefícios da abordagem proposta podem ser enfatizados: primeiro, ela torna possível a análise de incertezas na partição, e, assim, nas transições. Segundo, ela permite a análise de sistemas de controle baseado em eventos para a classe de sistemas afins por partes, já que, para o controle baseado em eventos, as transições não podem ser determinadas como uma função das variáveis de estado. A representação proposta, por outro lado, codifica implicitamente a partição e as transições. A análise de estabilidade é realizada com técnicas da teoria de Lyapunov. Condi- ções para a estabilidade exponencial são então apresentadas. Graças à representação implícita, o uso de funções candidatas de Lyapunov se torna simples. Essas condições podem ser resolvidas numéricamente usando uma formulação de desigualdades matriciais lineares. A análise numérica explora expressões quadráticas que descrevem funções de rampa para verificar a postivividade de formas quadráticas extendidas. Para o controle baseado em eventos, uma função de disparo quadrática por partes define o gerador de eventos. Parâmetros adequados para a função de disparo sãoencontrados com um procedimento de otimização. Como resultado, esta função usa informação da partição para reduzir o número de eventos, obtendo resultados melhores do que as funções de disparo quadráticas encontradas na literatura. Exemplos numéricos são fornecidos para ilustrar a aplicação da representação e mé- todos propostos

Stability of hybrid model predictive control

In this paper we investigate the stability of hybrid systems in closed-loop with Model Predictive Controllers (MPC) and we derive a priori sufficient conditions for Lyapunov asymptotic stability and exponential stability. A general theory is presented which proves that Lyapunov stability is achieved for both terminal cost and constraint set and terminal equality constraint hybrid MPC, even though the considered Lyapunov function and the system dynamics may be discontinuous. For particular choices of MPC criteria and constrained Piecewise Affine (PWA) systems as the prediction models we develop novel algorithms for computing the terminal cost and the terminal constraint set. For a quadratic MPC cost, the stabilization conditions translate into a linear matrix inequality while, for an 1-norm based MPC cost, they are obtained as 1-norm inequalities. It is shown that by using 1-norms, the terminal constraint set is automatically obtained as a polyhedron or a finite union of polyhedra by taking a sublevel set of the calculated terminal cost function. New algorithms are developed for calculating polyhedral or piecewise polyhedral positively invariant sets for PWA systems. In this manner, the on-line optimization problem leads to a mixed integer quadratic programming problem or to a mixed integer linear programming problem, which can be solved by standard optimization tools. Several examples illustrate the effectiveness of the developed methodology

Non-smooth model predictive control: stability and applications to hybrid systems

In this report we investigate the stability of hybrid systems in closed-loop with Model Predictive Controllers (MPC) and we derive a priori sufficient conditions for Lyapunov asymptotic stability and exponential stability. A general theory is presented which proves that Lyapunov stability is achieved for both terminal cost and constraint set and terminal equality constraint hybrid MPC, even though the considered Lyapunov function and the system dynamics may be discontinuous. For particular choices of MPC criteria and constrained Piecewise Affine (PWA) systems as the prediction models we develop novel algorithms for computing the terminal cost and the terminal constraint set. For a quadratic MPC cost, the stabilization conditions translate into a linear matrix inequality while, for an ∞-norm based MPC cost, they are obtained as ∞-norm inequalities. It is shown that by using ∞-norms, the terminal constraint set is automatically obtained as a polyhedron or a finite union of polyhedra by taking a sublevel set of the calculated terminal cost function. New algorithms are developed for calculating polyhedral or piecewise polyhedral positively invariant sets for PWA systems. In this manner, the on-line optimization problem leads to a mixed integer quadratic programming problem or to a mixed integer linear programming problem, which can be solved by standard optimization tools. Several examples illustrate the effectiveness of the developed methodology

Data-driven computation of invariant sets of discrete time-invariant black-box systems

We consider the problem of computing the maximal invariant set of discrete-time black-box nonlinear systems without analytic dynamical models. Under the assumption that the system is asymptotically stable, the maximal invariant set coincides with the domain of attraction. A data-driven framework relying on the observation of trajectories is proposed to compute almost-invariant sets, which are invariant almost everywhere except a small subset. Based on these observations, scenario optimization problems are formulated and solved. We show that probabilistic invariance guarantees on the almost-invariant sets can be established. To get explicit expressions of such sets, a set identification procedure is designed with a verification step that provides inner and outer approximations in a probabilistic sense. The proposed data-driven framework is illustrated by several numerical examples.Comment: A shorter version with the title "Scenario-based set invariance verification for black-box nonlinear systems" is published in the IEEE Control Systems Letters (L-CSS

Formal methods for resilient control

Many systems operate in uncertain, possibly adversarial environments, and their successful operation is contingent upon satisfying specific requirements, optimal performance, and ability to recover from unexpected situations. Examples are prevalent in many engineering disciplines such as transportation, robotics, energy, and biological systems. This thesis studies designing correct, resilient, and optimal controllers for discrete-time complex systems from elaborate, possibly vague, specifications. The first part of the contributions of this thesis is a framework for optimal control of non-deterministic hybrid systems from specifications described by signal temporal logic (STL), which can express a broad spectrum of interesting properties. The method is optimization-based and has several advantages over the existing techniques. When satisfying the specification is impossible, the degree of violation - characterized by STL quantitative semantics - is minimized. The computational limitations are discussed. The focus of second part is on specific types of systems and specifications for which controllers are synthesized efficiently. A class of monotone systems is introduced for which formal synthesis is scalable and almost complete. It is shown that hybrid macroscopic traffic models fall into this class. Novel techniques in modular verification and synthesis are employed for distributed optimal control, and their usefulness is shown for large-scale traffic management. Apart from monotone systems, a method is introduced for robust constrained control of networked linear systems with communication constraints. Case studies on longitudinal control of vehicular platoons are presented. The third part is about learning-based control with formal guarantees. Two approaches are studied. First, a formal perspective on adaptive control is provided in which the model is represented by a parametric transition system, and the specification is captured by an automaton. A correct-by-construction framework is developed such that the controller infers the actual parameters and plans accordingly for all possible future transitions and inferences. The second approach is based on hybrid model identification using input-output data. By assuming some limited knowledge of the range of system behaviors, theoretical performance guarantees are provided on implementing the controller designed for the identified model on the original unknown system