Optimal adaptive control of time-delay dynamical systems with known and uncertain dynamics

Delays are found in many industrial pneumatic and hydraulic systems, and as a result, the performance of the overall closed-loop system deteriorates unless they are explicitly accounted. It is also possible that the dynamics of such systems are uncertain. On the other hand, optimal control of time-delay systems in the presence of known and uncertain dynamics by using state and output feedback is of paramount importance. Therefore, in this research, a suite of novel optimal adaptive control (OAC) techniques are undertaken for linear and nonlinear continuous time-delay systems in the presence of uncertain system dynamics using state and/or output feedback. First, the optimal regulation of linear continuous-time systems with state and input delays by utilizing a quadratic cost function over infinite horizon is addressed using state and output feedback. Next, the optimal adaptive regulation is extended to uncertain linear continuous-time systems under a mild assumption that the bounds on system matrices are known. Subsequently, the event-triggered optimal adaptive regulation of partially unknown linear continuous time systems with state-delay is addressed by using integral reinforcement learning (IRL). It is demonstrated that the optimal control policy renders asymptotic stability of the closed-loop system provided the linear time-delayed system is controllable and observable. The proposed event-triggered approach relaxed the need for continuous availability of state vector and proven to be zeno-free. Finally, the OAC using IRL neural network based control of uncertain nonlinear time-delay systems with input and state delays is investigated. An identifier is proposed for nonlinear time-delay systems to approximate the system dynamics and relax the need for the control coefficient matrix in generating the control policy. Lyapunov analysis is utilized to design the optimal adaptive controller, derive parameter/weight tuning law and verify stability of the closed-loop system”--Abstract, page iv

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Analysis, filtering, and control for Takagi-Sugeno fuzzy models in networked systems

Copyright © 2015 Sunjie Zhang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.The fuzzy logic theory has been proven to be effective in dealing with various nonlinear systems and has a great success in industry applications. Among different kinds of models for fuzzy systems, the so-called Takagi-Sugeno (T-S) fuzzy model has been quite popular due to its convenient and simple dynamic structure as well as its capability of approximating any smooth nonlinear function to any specified accuracy within any compact set. In terms of such a model, the performance analysis and the design of controllers and filters play important roles in the research of fuzzy systems. In this paper, we aim to survey some recent advances on the T-S fuzzy control and filtering problems with various network-induced phenomena. The network-induced phenomena under consideration mainly include communication delays, packet dropouts, signal quantization, and randomly occurring uncertainties (ROUs). With such network-induced phenomena, the developments on T-S fuzzy control and filtering issues are reviewed in detail. In addition, some latest results on this topic are highlighted. In the end, conclusions are drawn and some possible future research directions are pointed out.This work was supported in part by the National Natural Science Foundation of China under Grants 61134009, 61329301, 11301118 and 61174136, the Natural Science Foundation of Jiangsu Province of China under Grant BK20130017, the Fundamental Research Funds for the Central Universities of China under Grant CUSF-DH-D-2013061, the Royal Society of the U.K., and the Alexander von Humboldt Foundation of Germany

Active Classification for POMDPs: a Kalman-like State Estimator

The problem of state tracking with active observation control is considered for a system modeled by a discrete-time, finite-state Markov chain observed through conditionally Gaussian measurement vectors. The measurement model statistics are shaped by the underlying state and an exogenous control input, which influence the observations' quality. Exploiting an innovations approach, an approximate minimum mean-squared error (MMSE) filter is derived to estimate the Markov chain system state. To optimize the control strategy, the associated mean-squared error is used as an optimization criterion in a partially observable Markov decision process formulation. A stochastic dynamic programming algorithm is proposed to solve for the optimal solution. To enhance the quality of system state estimates, approximate MMSE smoothing estimators are also derived. Finally, the performance of the proposed framework is illustrated on the problem of physical activity detection in wireless body sensing networks. The power of the proposed framework lies within its ability to accommodate a broad spectrum of active classification applications including sensor management for object classification and tracking, estimation of sparse signals and radar scheduling.Comment: 38 pages, 6 figure

A method for the reconstruction of unknown non-monotonic growth functions in the chemostat

We propose an adaptive control law that allows one to identify unstable steady states of the open-loop system in the single-species chemostat model without the knowledge of the growth function. We then show how one can use this control law to trace out (reconstruct) the whole graph of the growth function. The process of tracing out the graph can be performed either continuously or step-wise. We present and compare both approaches. Even in the case of two species in competition, which is not directly accessible with our approach due to lack of controllability, feedback control improves identifiability of the non-dominant growth rate.Comment: expansion of ideas from proceedings paper (17 pages, 8 figures), proceedings paper is version v

Learning and Management for Internet-of-Things: Accounting for Adaptivity and Scalability

Internet-of-Things (IoT) envisions an intelligent infrastructure of networked smart devices offering task-specific monitoring and control services. The unique features of IoT include extreme heterogeneity, massive number of devices, and unpredictable dynamics partially due to human interaction. These call for foundational innovations in network design and management. Ideally, it should allow efficient adaptation to changing environments, and low-cost implementation scalable to massive number of devices, subject to stringent latency constraints. To this end, the overarching goal of this paper is to outline a unified framework for online learning and management policies in IoT through joint advances in communication, networking, learning, and optimization. From the network architecture vantage point, the unified framework leverages a promising fog architecture that enables smart devices to have proximity access to cloud functionalities at the network edge, along the cloud-to-things continuum. From the algorithmic perspective, key innovations target online approaches adaptive to different degrees of nonstationarity in IoT dynamics, and their scalable model-free implementation under limited feedback that motivates blind or bandit approaches. The proposed framework aspires to offer a stepping stone that leads to systematic designs and analysis of task-specific learning and management schemes for IoT, along with a host of new research directions to build on.Comment: Submitted on June 15 to Proceeding of IEEE Special Issue on Adaptive and Scalable Communication Network