Event-triggered near optimal adaptive control of interconnected systems

Increased interest in complex interconnected systems like smart-grid, cyber manufacturing have attracted researchers to develop optimal adaptive control schemes to elicit a desired performance when the complex system dynamics are uncertain. In this dissertation, motivated by the fact that aperiodic event sampling saves network resources while ensuring system stability, a suite of novel event-sampled distributed near-optimal adaptive control schemes are introduced for uncertain linear and affine nonlinear interconnected systems in a forward-in-time and online manner. First, a novel stochastic hybrid Q-learning scheme is proposed to generate optimal adaptive control law and to accelerate the learning process in the presence of random delays and packet losses resulting from the communication network for an uncertain linear interconnected system. Subsequently, a novel online reinforcement learning (RL) approach is proposed to solve the Hamilton-Jacobi-Bellman (HJB) equation by using neural networks (NNs) for generating distributed optimal control of nonlinear interconnected systems using state and output feedback. To relax the state vector measurements, distributed observers are introduced. Next, using RL, an improved NN learning rule is derived to solve the HJB equation for uncertain nonlinear interconnected systems with event-triggered feedback. Distributed NN identifiers are introduced both for approximating the uncertain nonlinear dynamics and to serve as a model for online exploration. Next, the control policy and the event-sampling errors are considered as non-cooperative players and a min-max optimization problem is formulated for linear and affine nonlinear systems by using zero-sum game approach for simultaneous optimization of both the control policy and the event based sampling instants. The net result is the development of optimal adaptive event-triggered control of uncertain dynamic systems --Abstract, page iv

Event sampled optimal adaptive regulation of linear and a class of nonlinear systems

In networked control systems (NCS), wherein a communication network is used to close the feedback loop, the transmission of feedback signals and execution of the controller is currently carried out at periodic sampling instants. Thus, this scheme requires a significant computational power and network bandwidth. In contrast, the event-based aperiodic sampling and control, which is introduced recently, appears to relieve the computational burden and high network resource utilization. Therefore, in this dissertation, a suite of novel event sampled adaptive regulation schemes in both discrete and continuous time domain for uncertain linear and nonlinear systems are designed. Event sampled Q-learning and adaptive/neuro dynamic programming (ADP) schemes without value and policy iterations are utilized for the linear and nonlinear systems, respectively, in both the time domains. Neural networks (NN) are employed as approximators for nonlinear systems and, hence, the universal approximation property of NN in the event-sampled framework is introduced. The tuning of the parameters and the NN weights are carried out in an aperiodic manner at the event sampled instants leading to a further saving in computation when compared to traditional NN based control. The adaptive regulator when applied on a linear NCS with time-varying network delays and packet losses shows a 30% and 56% reduction in computation and network bandwidth usage, respectively. In case of nonlinear NCS with event sampled ADP based regulator, a reduction of 27% and 66% is observed when compared to periodic sampled schemes. The sampling and transmission instants are determined through adaptive event sampling conditions derived using Lyapunov technique by viewing the closed-loop event sampled linear and nonlinear systems as switched and/or impulsive dynamical systems. --Abstract, page iii

Optimal control of nonlinear partially-unknown systems with unsymmetrical input constraints and its applications to the optimal UAV circumnavigation problem

Aimed at solving the optimal control problem for nonlinear systems with unsymmetrical input constraints, we present an online adaptive approach for partially unknown control systems/dynamics. The designed algorithm converges online to the optimal control solution without the knowledge of the internal system dynamics. The optimality of the obtained control policy and the stability for the closed-loop dynamic optimality are proved theoretically. The proposed method greatly relaxes the assumption on the form of the internal dynamics and input constraints in previous works. Besides, the control design framework proposed in this paper offers a new approach to solve the optimal circumnavigation problem involving a moving target for a fixed-wing unmanned aerial vehicle (UAV). The control performance of our method is compared with that of the existing circumnavigation control law in a numerical simulation and the simulation results validate the effectiveness of our algorithm

Event-triggered synchronization of saturated lur’e type systems

This dissertation addresses the problem of master-slave synchronization of nonlinear discrete-time Lur’e systems subject to input saturation via event-triggered control (ETC) techniques. Synchronization, which is considered a remarkable property in the physics literature specially when chaotic systems are under investigation, is achieved through the stabilization of the error between the states of the master and the slave system. Regarding the Lur’e type nonlinearity, two different cases are studied along this work: generic slope-restricted state-dependent nonlinearity and piecewise-affine function. In the ETC paradigm, the control signal is updated aperiodically only after the occurrence of an event, which is generated according to a triggering criterion that depends on the evaluation of a triggering function. In the emulation-based design, a synchronization error feedback controller is given a priori and the task is to compute the event generator parameters ensuring performance and closed-loop stability. On the other hand, in the co-design approach the event generator and the control law are simultaneously designed. Theoretical results are obtained for three types of event-triggered mechanism (ETM), namely: static, dynamic and relaxed. In the last case, practical synchronization conditions are derived as form of ultimately boundedness stability. In order to tune the parameters of the event-based strategy, optimization problems are formulated aiming to reduce the number of control signal updates (number of events) with respect to a time-triggered implementation. Numerical simulations are presented to illustrate the application of the proposed methods.Esta dissertação aborda o problema de sincronização mestre-escravo de sistemas Lur’e não lineares de tempo discreto sujeitos à saturação de entrada via técnicas de controle baseado em eventos. A sincronização, que é considerada uma propriedade importante na literatura de Física especialmente quando sistemas caóticos são investigados, é alcançada através da estabilização do erro entre os estados do mestre e do sistema escravo. Em relação à não linearidade do tipo Lur’e, dois casos diferentes são estudados ao longo do trabalho: não linearidade genérica dependente do estado e restrita em inclinação e função afim por partes. No paradigma de controle baseado em eventos (ETC), o sinal de controle é atualizado aperiodicamente apenas após a ocorrência de um evento, que é gerado de acordo com um critério de disparo que depende da avaliação de uma função de disparo. No projeto baseado em emulação, um controlador por realimentação do erro de sincronização é dado a priori e a tarefa é calcular os parâmetros do gerador de eventos garantindo desempenho e estabilidade em malha fechada. Na abordagem de co-design, o gerador de eventos e a lei de controle são projetados simultaneamente. Resultados teóricos são obtidos para três tipos de mecanismo de geração de eventos (ETM), nomeadamente: estático, dinâmico e relaxado. Neste último caso, condições de sincronização prática são derivadas como uma forma de estabilidade ultimamente limitada. Para sintonizar os parâmetros da estratégia baseada em eventos, problemas de otimização são formulados visando reduzir o número de atualizações do sinal de controle (número de eventos) em relação a uma implementação time-triggered. Simulações numéricas são apresentadas para ilustrar a aplicação dos métodos propostos

Adaptive consensus based formation control of unmanned vehicles

Over the past decade, the control research community has given significant attention to formation control of multiple unmanned vehicles due to a variety of commercial and defense applications. Consensus-based formation control is considered to be more robust and reliable when compared to other formation control methods due to scalability and inherent properties that enable the formation to continue even if one of the vehicles experiences a failure. In contrast to existing methods on formation control where the dynamics of the vehicles are neglected, this dissertation in the form of four papers presents consensus-based formation control of unmanned vehicles-both ground and aerial, by incorporating the vehicle dynamics. First, neural networks (NN)-based optimal adaptive consensus-based formation control over finite horizon is presented for networked mobile robots or agents in the presence of uncertain robot/agent dynamics and communication. In the second paper, a hybrid automaton is proposed to control the nonholonomic mobile robots in two discrete modes: a regulation mode and a formation keeping mode in order to overcome well-known stabilization problem. The third paper presents the design of a distributed consensus-based event-triggered formation control of networked mobile robots using NN in the presence of uncertain robot dynamics to minimize communication. All these papers assume state availability. Finally, the fourth paper extends the consensus effort by introducing the development of a novel nonlinear output feedback NN-based controller for a group of quadrotor UAVs --Abstract, page iv

Stochastic optimal adaptive controller and communication protocol design for networked control systems

Networked Control System (NCS) is a recent topic of research wherein the feedback control loops are closed through a real-time communication network. Many design challenges surface in such systems due to network imperfections such as random delays, packet losses, quantization effects and so on. Since existing control techniques are unsuitable for such systems, in this dissertation, a suite of novel stochastic optimal adaptive design methodologies is undertaken for both linear and nonlinear NCS in presence of uncertain system dynamics and unknown network imperfections such as network-induced delays and packet losses. The design is introduced in five papers. In Paper 1, a stochastic optimal adaptive control design is developed for unknown linear NCS with uncertain system dynamics and unknown network imperfections. A value function is adjusted forward-in-time and online, and a novel update law is proposed for tuning value function estimator parameters. Additionally, by using estimated value function, optimal adaptive control law is derived based on adaptive dynamic programming technique. Subsequently, this design methodology is extended to solve stochastic optimal strategies of linear NCS zero-sum games in Paper 2. Since most systems are inherently nonlinear, a novel stochastic optimal adaptive control scheme is then developed in Paper 3 for nonlinear NCS with unknown network imperfections. On the other hand, in Paper 4, the network protocol behavior (e.g. TCP and UDP) are considered and optimal adaptive control design is revisited using output feedback for linear NCS. Finally, Paper 5 explores a co-design framework where both the controller and network scheduling protocol designs are addressed jointly so that proposed scheme can be implemented into next generation Cyber Physical Systems --Abstract, page iv