Stability analysis and stabilization of linear aperiodic sampled-data systems subject to input constraints

Motivados pelo crescente uso de controladores embarcados em diferentes aplicações, onde um protocolo de comunicação é responsável pela transmissão de dados entre algoritmos computacionais, atuadores e sensores, a análise e o controle de sistemas amostrados foram abordados em muitos trabalhos. Nesse contexto, a amostragem aperiódica pode ser vista como uma abstração matemática empregada para representar, na teoria, o efeito de imperfeições no canal de comunicação, como instabilidades, flutuações e, em alguns casos, perda de pacotes de dados. Além disso, devido a limitações físicas dos atuadores, restrições de entrada e, em particular, a saturação são onipresentes em problemas reais de controle. Essas restrições são fonte de comportamentos não-lineares e de degradação do desempenho. Em muitos casos, apenas a estabilidade local (ou regional) do sistema em malha fechada pode ser garantida na presença de restrições e não-linearidades de entrada, mesmo para plantas lineares. Este trabalho lida com sistemas lineares amostrados aperiodicamente em que a entrada de controle, sujeita a restrições (por exemplo, saturação), é calculada com base em uma realimentação de estados do sistema. Dois problemas principais são abordados. O primeiro consiste na análise de estabilidade da origem de tais sistemas com a determinação de estimativas da região de atração da origem (RAO). O segundo, por sua vez, corresponde ao projeto de controle, onde uma lei de controle de realimentação de estados é calculada para otimizar o tamanho de uma estimativa da RAO do sistema em malha fechada resultante. Os métodos propostos são baseados no uso de programação semidefinida ou linear e, portanto, podem ser facilmente aplicados na prática. Um dos métodos propostos considera uma realimentação de estados linear sujeita a saturação e funções de Lyapunov quadráticas, resultando em estimativas elipsoidais da RAO do sistema. Dois outros métodos lidam com a análise de estabilidade do sistema amostrado sujeito a saturação fornecendo estimativas poliedrais da RAO. Devido à sua flexibilidade, a adoção de poliedros em vez de elipsóides permite uma redução de conservadorismo, mas é muito exigente em termos de complexidade computacional. Motivada por esse fato, esta tese também propõe um método de projeto de controle baseado em uma estratégia alternativa, onde a complexidade dos poliedros é fixada a priori. Essa ideia resulta em um problema de otimização com restrições bilineares, onde uma lei de controle linear por partes estabilizadora de complexidade relativamente baixa é encontrada para o sistema amostrado. Os métodos mencionados acima consideram uma abordagem não-estocástica, onde limites inferior e superior são impostos para o intervalo de amostragem do sistema, o qual é desconhecido e variante no tempo. Como contribuição adicional, esta tese também considera uma abordagem estocástica. Um método de projeto de controle é proposto para a estabilização global no sentido quadrático médio do sistema amostrado, onde a lei de realimentação de estados linear é sujeita a não-linearidades que satisfazem a uma condição de setor e os intervalos de amostragem correspondem a variáveis aleatórias com a distribuição de Erlang. A possibilidade de perda de pacotes de dados também é explicitamente levada em consideração através da distribuição de Bernoulli. Além disso, o método proposto, que se baseia na teoria de processos de Markov determinísticos por partes, resulta em condições de estabilização não-conservadoras no caso linear sem restrições de entrada.Motivated by the growing use of embedded controllers in different applications, where a communication protocol is responsible for the transmission of data between computer algorithms, actuators and sensors, the analysis and control design for sampled-data control systems have been addressed in many works. In this context, aperiodic sampling can be seen as a modeling abstraction employed to represent, in a theoretical framework, the effect of imperfections on the communication channel such as sampling jitters, fluctuations and, in some cases, packet dropouts. Moreover, due to physical limitations of actuators, input constraints and, in particular, input saturation are ubiquitous in real control problems. These constraints are source of nonlinear behaviors and performance degradation. In many cases, only local (or regional) stability of the closed-loop system can be ensured in the presence of actuators constraints and nonlinearities, even for linear plants. This work deals with linear aperiodic sampled-data systems where the control input, subject to constraints (e.g. saturation), is computed based on a feedback of the system state. It focuses on two main problems. The first one regards the stability analysis of the origin of such systems, with the determination of estimates of the region of attraction of the origin (RAO). The second one, in turn, corresponds to the control design, where a state-feedback control law is computed in order to enlarge an estimate of the RAO of the resulting closed-loop system. The proposed methods are based on the use of semidefinite or linear programming and can therefore be easily applied in practice. One of the proposed methods considers a linear saturating feedback of the system state and quadratic Lyapunov functions, leading to ellipsoidal estimates of the RAO of the system. Two other methods deal with the stability analysis of the sampled-data system subject to input saturation providing polyhedral estimates of the RAO. Because of their flexibility, adopting polyhedrons instead of ellipsoids allows a reduction of conservatism, but is very demanding in terms of computational complexity. Motivated by this fact, this thesis also proposes a control design method based on an alternative strategy, where the complexity of the polytopes is fixed a priori. This idea results in an optimization problem with bilinear constraints, where a stabilizing piecewise linear control law of relatively low complexity is found for the sampled-data system. The aforementioned methods consider a non-stochastic framework, where lower and upper bounds are imposed for the unknown, time-varying sampling interval of the system. As an additional contribution, this thesis also considers a stochastic setting. A control design method is proposed for the global stabilization in the mean square sense of the sampled-data system, where the linear feedback control law is subject to sector bounded nonlinearities and the sampling intervals are assumed to be random variables with the Erlang distribution. The possibility of packet dropouts is also explicitly taken into account through the Bernoulli distribution. Moreover, the proposed approach, which is based onthe framework of Piecewise Deterministic Markov Processes, leads to non-conservative stabilization conditions in the unconstrained linear case.Motivé par l’utilisation croissante de contrôleurs embarqués dans différentes applications, où un protocole de communication est responsable par la transmission de données entre les algorithmes numériques, les actionneurs et les capteurs, l’analyse et la conception de contrôle pour les systèmes de contrôle échantillonnées ont été abordées dans de nombreux travaux. Dans ce contexte, l’échantillonnage apériodique peut être considéré comme une abstraction mathématique employée pour représenter, dans un cadre théorique, l’effet des imperfections sur le canal de communication telles que la gigue d’échantillonnage, les fluctuations et, dans certains cas, les pertes de paquets. De plus, en raison des limitations physiques des actionneurs, les contraintes d’entrée et, en particulier, la saturation des entrées sont omniprésentes dans les problèmes de contrôle réels. Ces contraintes sont une source de comportements non-linéaires et de dégradation de la performance. Dans de nombreux cas, seule la stabilité locale (ou régionale) du système en boucle fermée peut être assurée en présence de contraintes et de non-linéarités des actionneurs, même pour les systèmes linéaires. Ce travail traite des systèmes linéaires échantillonnées apériodiquement où l’entrée de commande, soumise à des contraintes (par exemple la saturation), est calculée sur la base d’un retour d’état du système. Il se concentre sur deux problèmes principaux. Le premier consiste en l’analyse de stabilité de l’origine de tels systèmes avec la détermination d’estimations de la région d’attraction de l’origine (RAO). Le deuxième, à son tour, correspond à la conception de la commande, où une loi de commande à retour d’état est calculée afin d’agrandir une estimation de la RAO du système en boucle fermée résultant. Les méthodes proposées sont basées sur la programmation semi-définie ou linéaire et peuvent donc être facilement appliquées dans la pratique. L’une des méthodes proposées considère un retour d’état linéaire soumis à la saturation et des fonctions de Lyapunov quadratiques, conduisant à des estimations ellipsoïdales de la RAO du système. Deux autres méthodes traitent de l’analyse de stabilité du système échantillonné soumis à la saturation des entrées fournissant des estimations polyédriques de la RAO. En raison de leur flexibilité, l’adoption de polyèdres au lieu d’ellipsoïdes permet une réduction du conservatisme mais est très exigeante en termes de complexité de calcul. Motivée par ce fait, cette thèse propose également une méthode de conception de contrôle basée sur une stratégie alternative, où la complexité des polyèdres est fixée a priori. Cette idée se traduit par un problème d’optimisation avec des contraintes bilinéaires, où une loi de commande linéaire par morceaux stabilisante de complexité relativement faible est trouvée pour le système échantillonné. Les méthodes mentionnées ci-dessus considèrent un cadre non stochastique, où des limites inférieure et supérieure sont imposées pour l’intervalle d’échantillonnage inconnu et variable dans le temps du système. Comme contribution supplémentaire, cette thèseconsidère également un cadre stochastique. Une méthode de conception de contrôle est proposée pour la stabilisation globale dans le sens quadratique moyen du système échantillonné, où la loi de contrôle linéaire de retour d’état est soumise à des non-linéarités délimitées par secteur et les intervalles d’échantillonnage sont supposés être des variables aléatoires avec la distribution d’Erlang. La possibilité de pertes de paquets est aussi explicitement prise en compte via la distribution de Bernoulli. De plus, l’approche proposée, qui est basée sur le cadre des processus de Markov déterministes par morceaux, conduit à des conditions de stabilisation non conservatrices dans le cas linéaire sans contraintes

Robust Model Predictive Control via Scenario Optimization

This paper discusses a novel probabilistic approach for the design of robust model predictive control (MPC) laws for discrete-time linear systems affected by parametric uncertainty and additive disturbances. The proposed technique is based on the iterated solution, at each step, of a finite-horizon optimal control problem (FHOCP) that takes into account a suitable number of randomly extracted scenarios of uncertainty and disturbances, followed by a specific command selection rule implemented in a receding horizon fashion. The scenario FHOCP is always convex, also when the uncertain parameters and disturbance belong to non-convex sets, and irrespective of how the model uncertainty influences the system's matrices. Moreover, the computational complexity of the proposed approach does not depend on the uncertainty/disturbance dimensions, and scales quadratically with the control horizon. The main result in this paper is related to the analysis of the closed loop system under receding-horizon implementation of the scenario FHOCP, and essentially states that the devised control law guarantees constraint satisfaction at each step with some a-priori assigned probability p, while the system's state reaches the target set either asymptotically, or in finite time with probability at least p. The proposed method may be a valid alternative when other existing techniques, either deterministic or stochastic, are not directly usable due to excessive conservatism or to numerical intractability caused by lack of convexity of the robust or chance-constrained optimization problem.Comment: This manuscript is a preprint of a paper accepted for publication in the IEEE Transactions on Automatic Control, with DOI: 10.1109/TAC.2012.2203054, and is subject to IEEE copyright. The copy of record will be available at http://ieeexplore.ieee.or

Learning-based Predictive Control for Nonlinear Systems with Unknown Dynamics Subject to Safety Constraints

Model predictive control (MPC) has been widely employed as an effective method for model-based constrained control. For systems with unknown dynamics, reinforcement learning (RL) and adaptive dynamic programming (ADP) have received notable attention to solve the adaptive optimal control problems. Recently, works on the use of RL in the framework of MPC have emerged, which can enhance the ability of MPC for data-driven control. However, the safety under state constraints and the closed-loop robustness are difficult to be verified due to approximation errors of RL with function approximation structures. Aiming at the above problem, we propose a data-driven robust MPC solution based on incremental RL, called data-driven robust learning-based predictive control (dr-LPC), for perturbed unknown nonlinear systems subject to safety constraints. A data-driven robust MPC (dr-MPC) is firstly formulated with a learned predictor. The incremental Dual Heuristic Programming (DHP) algorithm using an actor-critic architecture is then utilized to solve the online optimization problem of dr-MPC. In each prediction horizon, the actor and critic learn time-varying laws for approximating the optimal control policy and costate respectively, which is different from classical MPCs. The state and control constraints are enforced in the learning process via building a Hamilton-Jacobi-Bellman (HJB) equation and a regularized actor-critic learning structure using logarithmic barrier functions. The closed-loop robustness and safety of the dr-LPC are proven under function approximation errors. Simulation results on two control examples have been reported, which show that the dr-LPC can outperform the DHP and dr-MPC in terms of state regulation, and its average computational time is much smaller than that with the dr-MPC in both examples.Comment: The paper has been submitted at a IEEE Journal for possible publicatio

Green Scheduling of Control Systems

Electricity usage under peak load conditions can cause issues such as reduced power quality and power outages. For this reason, commercial electricity customers are often subject to demand-based pricing, which charges very high prices for peak electricity demand. Consequently, reducing peaks in electricity demand is desirable for both economic and reliability reasons. In this thesis, we investigate the peak demand reduction problem from the perspective of safe scheduling of control systems under resource constraint. To this end, we propose Green Scheduling as an approach to schedule multiple interacting control systems within a constrained peak demand envelope while ensuring that safety and operational conditions are facilitated. The peak demand envelope is formulated as a constraint on the number of binary control inputs that can be activated simultaneously. Using two different approaches, we establish a range of sufficient and necessary schedulability conditions for various classes of affine dynamical systems. The schedulability analysis methods are shown to be scalable for large-scale systems consisting of up to 1000 subsystems. We then develop several scheduling algorithms for the Green Scheduling problem. First, we develop a periodic scheduling synthesis method, which is simple and scalable in computation but does not take into account the influence of disturbances. We then improve the method to be robust to small disturbances while preserving the simplicity and scalability of periodic scheduling. However the improved algorithm usually result in fast switching of the control inputs. Therefore, event-triggered and self-triggered techniques are used to alleviate this issue. Next, using a feedback control approach based on attracting sets and robust control Lyapunov functions, we develop event-triggered and self-triggered scheduling algorithms that can handle large disturbances affecting the system. These algorithms can also exploit prediction of the disturbances to improve their performance. Finally, a scheduling method for discrete-time systems is developed based on backward reachability analysis. The effectiveness of the proposed approach is demonstrated by an application to scheduling of radiant heating and cooling systems in buildings. Green Scheduling is able to significantly reduce the peak electricity demand and the total electricity consumption of the radiant systems, while maintaining thermal comfort for occupants

Stochastic Event-Based Control and Estimation

Digital controllers are traditionally implemented using periodic sampling, computation, and actuation events. As more control systems are implemented to share limited network and CPU bandwidth with other tasks, it is becoming increasingly attractive to use some form of event-based control instead, where precious events are used only when needed. Forms of event-based control have been used in practice for a very long time, but mostly in an ad-hoc way. Though optimal solutions to most event-based control problems are unknown, it should still be viable to compare performance between suggested approaches in a reasonable manner. This thesis investigates an event-based variation on the stochastic linear-quadratic (LQ) control problem, with a fixed cost per control event. The sporadic constraint of an enforced minimum inter-event time is introduced, yielding a mixed continuous-/discrete-time formulation. The quantitative trade-off between event rate and control performance is compared between periodic and sporadic control. Example problems for first-order plants are investigated, for a single control loop and for multiple loops closed over a shared medium. Path constraints are introduced to model and analyze higher-order event-based control systems. This component-based approach to stochastic hybrid systems allows to express continuous- and discrete-time dynamics, state and switching constraints, control laws, and stochastic disturbances in the same model. Sum-of-squares techniques are then used to find bounds on control objectives using convex semidefinite programming. The thesis also considers state estimation for discrete time linear stochastic systems from measurements with convex set uncertainty. The Bayesian observer is considered given log-concave process disturbances and measurement likelihoods. Strong log-concavity is introduced, and it is shown that the observer preserves log-concavity, and propagates strong log-concavity like inverse covariance in a Kalman filter. A recursive state estimator is developed for systems with both stochastic and set-bounded process and measurement noise terms. A time-varying linear filter gain is optimized using convex semidefinite programming and ellipsoidal over-approximation, given a relative weight on the two kinds of error

Fault-tolerant Stochastic Distributed Systems

The present doctoral thesis discusses the design of fault-tolerant distributed systems, placing emphasis in addressing the case where the actions of the nodes or their interactions are stochastic. The main objective is to detect and identify faults to improve the resilience of distributed systems to crash-type faults, as well as detecting the presence of malicious nodes in pursuit of exploiting the network. The proposed analysis considers malicious agents and computational solutions to detect faults. Crash-type faults, where the affected component ceases to perform its task, are tackled in this thesis by introducing stochastic decisions in deterministic distributed algorithms. Prime importance is placed on providing guarantees and rates of convergence for the steady-state solution. The scenarios of a social network (state-dependent example) and consensus (time- dependent example) are addressed, proving convergence. The proposed algorithms are capable of dealing with packet drops, delays, medium access competition, and, in particular, nodes failing and/or losing network connectivity. The concept of Set-Valued Observers (SVOs) is used as a tool to detect faults in a worst-case scenario, i.e., when a malicious agent can select the most unfavorable sequence of communi- cations and inject a signal of arbitrary magnitude. For other types of faults, it is introduced the concept of Stochastic Set-Valued Observers (SSVOs) which produce a confidence set where the state is known to belong with at least a pre-specified probability. It is shown how, for an algorithm of consensus, it is possible to exploit the structure of the problem to reduce the computational complexity of the solution. The main result allows discarding interactions in the model that do not contribute to the produced estimates. The main drawback of using classical SVOs for fault detection is their computational burden. By resorting to a left-coprime factorization for Linear Parameter-Varying (LPV) systems, it is shown how to reduce the computational complexity. By appropriately selecting the factorization, it is possible to consider detectable systems (i.e., unobservable systems where the unobservable component is stable). Such a result plays a key role in the domain of Cyber-Physical Systems (CPSs). These techniques are complemented with Event- and Self-triggered sampling strategies that enable fewer sensor updates. Moreover, the same triggering mechanisms can be used to make decisions of when to run the SVO routine or resort to over-approximations that temporarily compromise accuracy to gain in performance but maintaining the convergence characteristics of the set-valued estimates. A less stringent requirement for network resources that is vital to guarantee the applicability of SVO-based fault detection in the domain of Networked Control Systems (NCSs)

Computational Techniques for Stochastic Reachability

As automated control systems grow in prevalence and complexity, there is an increasing demand for verification and controller synthesis methods to ensure these systems perform safely and to desired specifications. In addition, uncertain or stochastic behaviors are often exhibited (such as wind affecting the motion of an aircraft), making probabilistic verification desirable. Stochastic reachability analysis provides a formal means of generating the set of initial states that meets a given objective (such as safety or reachability) with a desired level of probability, known as the reachable (or safe) set, depending on the objective. However, the applicability of reachability analysis is limited in the scope and size of system it can address. First, generating stochastic reachable or viable sets is computationally intensive, and most existing methods rely on an optimal control formulation that requires solving a dynamic program, and which scales exponentially in the dimension of the state space. Second, almost no results exist for extending stochastic reachability analysis to systems with incomplete information, such that the controller does not have access to the full state of the system. This thesis addresses both of the above limitations, and introduces novel computational methods for generating stochastic reachable sets for both perfectly and partially observable systems. We initially consider a linear system with additive Gaussian noise, and introduce two methods for computing stochastic reachable sets that do not require dynamic programming. The first method uses a particle approximation to formulate a deterministic mixed integer linear program that produces an estimate to reachability probabilities. The second method uses a convex chance-constrained optimization problem to generate an under-approximation to the reachable set. Using these methods we are able to generate stochastic reachable sets for a four-dimensional spacecraft docking example in far less time than it would take had we used a dynamic program. We then focus on discrete time stochastic hybrid systems, which provide a flexible modeling framework for systems that exhibit mode-dependent behavior, and whose state space has both discrete and continuous components. We incorporate a stochastic observation process into the hybrid system model, and derive both theoretical and computational results for generating stochastic reachable sets subject to an observation process. The derivation of an information state allows us to recast the problem as one of perfect information, and we prove that solving a dynamic program over the information state is equivalent to solving the original problem. We then demonstrate that the dynamic program to solve the reachability problem for a partially observable stochastic hybrid system shares the same properties as for a partially observable Markov decision process (POMDP) with an additive cost function, and so we can exploit approximation strategies designed for POMDPs to solve the reachability problem. To do so, however, we first generate approximate representations of the information state and value function as either vectors or Gaussian mixtures, through a finite state approximation to the hybrid system or using a Gaussian mixture approximation to an indicator function defined over a convex region. For a system with linear dynamics and Gaussian measurement noise, we show that it exhibits special properties that do not require an approximation of the information state, which enables much more efficient computation of the reachable set. In all cases we provide convergence results and numerical examples