Stochastic optimal adaptive controller and communication protocol design for networked control systems

Networked Control System (NCS) is a recent topic of research wherein the feedback control loops are closed through a real-time communication network. Many design challenges surface in such systems due to network imperfections such as random delays, packet losses, quantization effects and so on. Since existing control techniques are unsuitable for such systems, in this dissertation, a suite of novel stochastic optimal adaptive design methodologies is undertaken for both linear and nonlinear NCS in presence of uncertain system dynamics and unknown network imperfections such as network-induced delays and packet losses. The design is introduced in five papers. In Paper 1, a stochastic optimal adaptive control design is developed for unknown linear NCS with uncertain system dynamics and unknown network imperfections. A value function is adjusted forward-in-time and online, and a novel update law is proposed for tuning value function estimator parameters. Additionally, by using estimated value function, optimal adaptive control law is derived based on adaptive dynamic programming technique. Subsequently, this design methodology is extended to solve stochastic optimal strategies of linear NCS zero-sum games in Paper 2. Since most systems are inherently nonlinear, a novel stochastic optimal adaptive control scheme is then developed in Paper 3 for nonlinear NCS with unknown network imperfections. On the other hand, in Paper 4, the network protocol behavior (e.g. TCP and UDP) are considered and optimal adaptive control design is revisited using output feedback for linear NCS. Finally, Paper 5 explores a co-design framework where both the controller and network scheduling protocol designs are addressed jointly so that proposed scheme can be implemented into next generation Cyber Physical Systems --Abstract, page iv

Analysis and Synthesis Methods for Nonlinear Network Systems

Over the past two decades the interactions between systems and their control components have undergone some significant changes. These interactions are no more localized, but usually take place over a network and even the control components may be remotely located, thus involving aspects of communication in control systems. Furthermore, the last decade has also seen a surge in intermingling ideas from control and communication and their application to biological systems, power systems giving rise to new research areas like Networked Control Systems (NCS), Cyber-Physical Systems (CPS), Gene Regulatory Networks (GRN) to name a few. This has led researchers to study control systems with practical constraints imposed on them. One such practical constraint identified as a major challenge, is the fragility of control systems and performance degradation, when the interconnection is not reliable. Design of controllers and estimators for such systems needs to take into account these constraints and mitigate them, to ensure sufficient robustness against unreliability of the interconnection. Considerable research has been done over the past decade in analyzing these new challenges and developing design tools to extract desired performance. Control over communication channels is one such widely researched area where the effect of unreliable interconnection on the stability performance of the system has been studied. The reliability of communication could manifest in various ways like sensor failure at output measurement, control actuator failure, interconnection links failures in the form of packet erasure channel, fading channel, quantization etc. Significant research progress has been made, in areas of control and estimation over unreliable communication links, consensus over unreliable network interconnections, etc., albeit the work has dealt with linear time invariant (LTI) systems theory. This has led to fruitful results for special cases of packet-drop communication channel modeled as a Bernoulli erasure channel. In the case of linear systems these results have demonstrated a connection between the performance characteristics of the interconnection and the expansion or destabilizing characteristics of the linear system, in obtaining desired performance of the closed loop system. Most of the current research for control over communication channels have focused on LTI plant dynamics. Furthermore the results involving nonlinear plant dynamics have reverted to local linearization techniques. It is well-known that for nonlinear systems, results based on local linearization at an equilibrium point will be local in nature and does not account for the global dynamics of the nonlinear system. For the proposed applications of network control systems to electric power grid and biological networks it is essential to develop results for the analysis of nonlinear systems over networks. In this work, we are primarily interested in the interaction of nonlinear systems and controllers over unreliable interconnections modelled as a stochastic multiplicative uncertainty. We provide analysis and synthesis methods for the control and observation of uncertain nonlinear network controlled systems. Our analysis methods indicate, fundamental limitations arise in the stabilization and observation of nonlinear systems over uncertain channels. Our main result provides the limitation for observation of nonlinear system over erasure channel expressed in terms of the probability of erasure and positive Lyapunov exponents of the open loop nonlinear plant. The positive Lyapunov exponents are measure of dynamical complexity and comparing our results with existing results for LTI systems, we show that Lyapunov exponents emerge as a natural generalization of eigenvalues from linear to nonlinear systems. Entropy is another measure of dynamical complexity. Using results from ergodic theory of dynamical systems we also relate the limitation for stabilization and observation with the entropy corresponding to the invariant measure capturing the global dynamics of the nonlinear systems. Existing Bode-like fundamental limitation results for nonlinear systems relate limitation for stabilization with the entropy corresponding to the invariant measure at the equilibrium point. Our results are the first to connect the limitation for stabilization with the entropy corresponding to invariant measure other than the one associated with equilibrium point. Our synthesis methods for the design of robust controller and observer against uncertain channels revolves around special class of nonlinear systems -Lure systems. These systems are essentially linear systems with sector-bounded nonlinearity in the feedback loop. For this special class of nonlinear systems, we delve into the theoretical tools of absolute stability to obtain some synthesis methods which provide design criteria for nonlinear systems over unreliable interconnections. Stability of Lur\u27e systems is a special case of the stability of interconnected passive systems. Thus we can characterize the unreliability of the interconnection, that guarantees the desired performance for Lur\u27e systems, in terms of the passivity of the linear system. Passivity theory is a rich theory with wide spread applications to nonlinear controller design and observation, which extends ideas of system stability to input-output systems using the ideas of dissipativity. Our synthesis methods developed for Lure systems with input and output stochastic channel uncertainties provide natural extension of the powerful passivity based synthesis tools developed for deterministic Lure systems. In particular, our results help understand the trade-off between passivity and stochastic uncertainty in feedback control systems