Optimal adaptive control of time-delay dynamical systems with known and uncertain dynamics

Delays are found in many industrial pneumatic and hydraulic systems, and as a result, the performance of the overall closed-loop system deteriorates unless they are explicitly accounted. It is also possible that the dynamics of such systems are uncertain. On the other hand, optimal control of time-delay systems in the presence of known and uncertain dynamics by using state and output feedback is of paramount importance. Therefore, in this research, a suite of novel optimal adaptive control (OAC) techniques are undertaken for linear and nonlinear continuous time-delay systems in the presence of uncertain system dynamics using state and/or output feedback. First, the optimal regulation of linear continuous-time systems with state and input delays by utilizing a quadratic cost function over infinite horizon is addressed using state and output feedback. Next, the optimal adaptive regulation is extended to uncertain linear continuous-time systems under a mild assumption that the bounds on system matrices are known. Subsequently, the event-triggered optimal adaptive regulation of partially unknown linear continuous time systems with state-delay is addressed by using integral reinforcement learning (IRL). It is demonstrated that the optimal control policy renders asymptotic stability of the closed-loop system provided the linear time-delayed system is controllable and observable. The proposed event-triggered approach relaxed the need for continuous availability of state vector and proven to be zeno-free. Finally, the OAC using IRL neural network based control of uncertain nonlinear time-delay systems with input and state delays is investigated. An identifier is proposed for nonlinear time-delay systems to approximate the system dynamics and relax the need for the control coefficient matrix in generating the control policy. Lyapunov analysis is utilized to design the optimal adaptive controller, derive parameter/weight tuning law and verify stability of the closed-loop system”--Abstract, page iv

Stochastic optimal adaptive controller and communication protocol design for networked control systems

Networked Control System (NCS) is a recent topic of research wherein the feedback control loops are closed through a real-time communication network. Many design challenges surface in such systems due to network imperfections such as random delays, packet losses, quantization effects and so on. Since existing control techniques are unsuitable for such systems, in this dissertation, a suite of novel stochastic optimal adaptive design methodologies is undertaken for both linear and nonlinear NCS in presence of uncertain system dynamics and unknown network imperfections such as network-induced delays and packet losses. The design is introduced in five papers. In Paper 1, a stochastic optimal adaptive control design is developed for unknown linear NCS with uncertain system dynamics and unknown network imperfections. A value function is adjusted forward-in-time and online, and a novel update law is proposed for tuning value function estimator parameters. Additionally, by using estimated value function, optimal adaptive control law is derived based on adaptive dynamic programming technique. Subsequently, this design methodology is extended to solve stochastic optimal strategies of linear NCS zero-sum games in Paper 2. Since most systems are inherently nonlinear, a novel stochastic optimal adaptive control scheme is then developed in Paper 3 for nonlinear NCS with unknown network imperfections. On the other hand, in Paper 4, the network protocol behavior (e.g. TCP and UDP) are considered and optimal adaptive control design is revisited using output feedback for linear NCS. Finally, Paper 5 explores a co-design framework where both the controller and network scheduling protocol designs are addressed jointly so that proposed scheme can be implemented into next generation Cyber Physical Systems --Abstract, page iv

Event sampled optimal adaptive regulation of linear and a class of nonlinear systems

In networked control systems (NCS), wherein a communication network is used to close the feedback loop, the transmission of feedback signals and execution of the controller is currently carried out at periodic sampling instants. Thus, this scheme requires a significant computational power and network bandwidth. In contrast, the event-based aperiodic sampling and control, which is introduced recently, appears to relieve the computational burden and high network resource utilization. Therefore, in this dissertation, a suite of novel event sampled adaptive regulation schemes in both discrete and continuous time domain for uncertain linear and nonlinear systems are designed. Event sampled Q-learning and adaptive/neuro dynamic programming (ADP) schemes without value and policy iterations are utilized for the linear and nonlinear systems, respectively, in both the time domains. Neural networks (NN) are employed as approximators for nonlinear systems and, hence, the universal approximation property of NN in the event-sampled framework is introduced. The tuning of the parameters and the NN weights are carried out in an aperiodic manner at the event sampled instants leading to a further saving in computation when compared to traditional NN based control. The adaptive regulator when applied on a linear NCS with time-varying network delays and packet losses shows a 30% and 56% reduction in computation and network bandwidth usage, respectively. In case of nonlinear NCS with event sampled ADP based regulator, a reduction of 27% and 66% is observed when compared to periodic sampled schemes. The sampling and transmission instants are determined through adaptive event sampling conditions derived using Lyapunov technique by viewing the closed-loop event sampled linear and nonlinear systems as switched and/or impulsive dynamical systems. --Abstract, page iii

Event-triggered near optimal adaptive control of interconnected systems

Increased interest in complex interconnected systems like smart-grid, cyber manufacturing have attracted researchers to develop optimal adaptive control schemes to elicit a desired performance when the complex system dynamics are uncertain. In this dissertation, motivated by the fact that aperiodic event sampling saves network resources while ensuring system stability, a suite of novel event-sampled distributed near-optimal adaptive control schemes are introduced for uncertain linear and affine nonlinear interconnected systems in a forward-in-time and online manner. First, a novel stochastic hybrid Q-learning scheme is proposed to generate optimal adaptive control law and to accelerate the learning process in the presence of random delays and packet losses resulting from the communication network for an uncertain linear interconnected system. Subsequently, a novel online reinforcement learning (RL) approach is proposed to solve the Hamilton-Jacobi-Bellman (HJB) equation by using neural networks (NNs) for generating distributed optimal control of nonlinear interconnected systems using state and output feedback. To relax the state vector measurements, distributed observers are introduced. Next, using RL, an improved NN learning rule is derived to solve the HJB equation for uncertain nonlinear interconnected systems with event-triggered feedback. Distributed NN identifiers are introduced both for approximating the uncertain nonlinear dynamics and to serve as a model for online exploration. Next, the control policy and the event-sampling errors are considered as non-cooperative players and a min-max optimization problem is formulated for linear and affine nonlinear systems by using zero-sum game approach for simultaneous optimization of both the control policy and the event based sampling instants. The net result is the development of optimal adaptive event-triggered control of uncertain dynamic systems --Abstract, page iv

A control theoretic approach for security of cyber-physical systems

In this dissertation, several novel defense methodologies for cyber-physical systems have been proposed. First, a special type of cyber-physical system, the RFID system, is considered for which a lightweight mutual authentication and ownership management protocol is proposed in order to protect the data confidentiality and integrity. Then considering the fact that the protection of the data confidentiality and integrity is insufficient to guarantee the security in cyber-physical systems, we turn to the development of a general framework for developing security schemes for cyber-physical systems wherein the cyber system states affect the physical system and vice versa. After that, we apply this general framework by selecting the traffic flow as the cyber system state and a novel attack detection scheme that is capable of capturing the abnormality in the traffic flow in those communication links due to a class of attacks has been proposed. On the other hand, an attack detection scheme that is capable of detecting both sensor and actuator attacks is proposed for the physical system in the presence of network induced delays and packet losses. Next, an attack detection scheme is proposed when the network parameters are unknown by using an optimal Q-learning approach. Finally, this attack detection and accommodation scheme has been further extended to the case where the network is modeled as a nonlinear system with unknown system dynamics --Abstract, page iv

Intelligent Design for Real Time Networked Multi-Agent Systems

Past decade has witnessed an unprecedented growth in reasearch for Unmanned Aerial Vehicles (UAVs) both in military and nonmilitary fronts. They have become ubiquitous in almost every military operations which includes domestic and overseas missions. With rapidly advancing technology, open source nature of the flight controllers, and significantly lesser costs than before, companies around the world are delving into UAV market as one of the upcoming lucrative investments. Companies like Amazon Inc., Dominos Pizza Inc. have had some successful test runs which again solidifies the research opportunities. Delivery services and recreational uses seems to have increased in the past 3-4 years which has let the Federal Aviation Administration to update their rules and regulations. Mapping, Surveying and search/rescue mission are some of the applications of UAVs that are most appealing. Making these applications airborne cuts the time and cost at considerable and affordable levels. Using UAVs for operations has advantages in both response time and need of manpower compared to piloted aricrafts. Obtaining prior information of a person/people in distress can become a deciding factor for a successful mission. It can help in making critical decision as which location or type of helicopter / vehicle to be used for extraction, equipment to bring and how many crew members that are needed. The idea here is to make this system of UAVs automated to coordinate with each other without human intervention (other than high level commands like takeoff and land). Researchers and Military experts have recognized the use of drones for search and rescue missions to be of utmost importance. Year 2016 saw a first of its kind UAV search and rescue symposium held in Nevada. The objective was to give a platform for UAV enthusiasts and researchers and share their experiences and concerns while using UAVs as first responders. The biggest drawback of using an aerial vehicle for inspection/search/rescue mission is its airborne time. The batteries used are big and heavy which increases the weight and decreases the flight time. One can go about solving this issue by using a swarm of UAVs which would inspect/search a given area in less amount of time. This has advantage in both response time and need for lesser man power.The main challenges for Multiple Drone Control (MDC) includes 1) Address the periodic sampling frequency issue of information of assets so as to maintain stability; 2) Optimize the communication channel while providing minimum Quality of Service (QoS); 3) Optimal control strategy which includes non-linearity in state space model; 4) Optimal control in presence of uncertainties; 5) Admitting new agents for dynamic agents in the Networked Multi-Agent System (MAS) Scenario.This dissertation aims at building a hardware and a software platform for communication of multiple UAVs upon which additional control algorithms can be implementated. It starts with building a DJI S1000 octacopter from the ground up. The components used are specified in the following sections. The idea here is to make a drone that can autonomously travel to specified location with safety features like geofencing and land on emergency situations. The user has to provide the necessary commands like GPS locations and takeoff/land commands via a Radio Controller (RC) remote. At any point of the flight, the UAV should be able to receive new commands from the ground control stations (GCS). After successful implementation, the UAV would not be restricted to the range of RC remote. It would be able to travel greater distances given the GPS signal remains operational in the field. This is possible at a global scale with limitation of only the batteries and flight time

Adaptive consensus based formation control of unmanned vehicles

Over the past decade, the control research community has given significant attention to formation control of multiple unmanned vehicles due to a variety of commercial and defense applications. Consensus-based formation control is considered to be more robust and reliable when compared to other formation control methods due to scalability and inherent properties that enable the formation to continue even if one of the vehicles experiences a failure. In contrast to existing methods on formation control where the dynamics of the vehicles are neglected, this dissertation in the form of four papers presents consensus-based formation control of unmanned vehicles-both ground and aerial, by incorporating the vehicle dynamics. First, neural networks (NN)-based optimal adaptive consensus-based formation control over finite horizon is presented for networked mobile robots or agents in the presence of uncertain robot/agent dynamics and communication. In the second paper, a hybrid automaton is proposed to control the nonholonomic mobile robots in two discrete modes: a regulation mode and a formation keeping mode in order to overcome well-known stabilization problem. The third paper presents the design of a distributed consensus-based event-triggered formation control of networked mobile robots using NN in the presence of uncertain robot dynamics to minimize communication. All these papers assume state availability. Finally, the fourth paper extends the consensus effort by introducing the development of a novel nonlinear output feedback NN-based controller for a group of quadrotor UAVs --Abstract, page iv