A survey on gain-scheduled control and filtering for parameter-varying systems

Copyright © 2014 Guoliang Wei et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.This paper presents an overview of the recent developments in the gain-scheduled control and filtering problems for the parameter-varying systems. First of all, we recall several important algorithms suitable for gain-scheduling method including gain-scheduled proportional-integral derivative (PID) control, H 2, H ∞ and mixed H 2 / H ∞ gain-scheduling methods as well as fuzzy gain-scheduling techniques. Secondly, various important parameter-varying system models are reviewed, for which gain-scheduled control and filtering issues are usually dealt with. In particular, in view of the randomly occurring phenomena with time-varying probability distributions, some results of our recent work based on the probability-dependent gain-scheduling methods are reviewed. Furthermore, some latest progress in this area is discussed. Finally, conclusions are drawn and several potential future research directions are outlined.The National Natural Science Foundation of China under Grants 61074016, 61374039, 61304010, and 61329301; the Natural Science Foundation of Jiangsu Province of China under Grant BK20130766; the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning; the Program for New Century Excellent Talents in University under Grant NCET-11-1051, the Leverhulme Trust of the U.K., the Alexander von Humboldt Foundation of Germany

Actor-Critic Reinforcement Learning for Control with Stability Guarantee

Reinforcement Learning (RL) and its integration with deep learning have achieved impressive performance in various robotic control tasks, ranging from motion planning and navigation to end-to-end visual manipulation. However, stability is not guaranteed in model-free RL by solely using data. From a control-theoretic perspective, stability is the most important property for any control system, since it is closely related to safety, robustness, and reliability of robotic systems. In this paper, we propose an actor-critic RL framework for control which can guarantee closed-loop stability by employing the classic Lyapunov's method in control theory. First of all, a data-based stability theorem is proposed for stochastic nonlinear systems modeled by Markov decision process. Then we show that the stability condition could be exploited as the critic in the actor-critic RL to learn a controller/policy. At last, the effectiveness of our approach is evaluated on several well-known 3-dimensional robot control tasks and a synthetic biology gene network tracking task in three different popular physics simulation platforms. As an empirical evaluation on the advantage of stability, we show that the learned policies can enable the systems to recover to the equilibrium or way-points when interfered by uncertainties such as system parametric variations and external disturbances to a certain extent.Comment: IEEE RA-L + IROS 202

Recent advances on recursive filtering and sliding mode design for networked nonlinear stochastic systems: A survey

Copyright © 2013 Jun Hu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Some recent advances on the recursive filtering and sliding mode design problems for nonlinear stochastic systems with network-induced phenomena are surveyed. The network-induced phenomena under consideration mainly include missing measurements, fading measurements, signal quantization, probabilistic sensor delays, sensor saturations, randomly occurring nonlinearities, and randomly occurring uncertainties. With respect to these network-induced phenomena, the developments on filtering and sliding mode design problems are systematically reviewed. In particular, concerning the network-induced phenomena, some recent results on the recursive filtering for time-varying nonlinear stochastic systems and sliding mode design for time-invariant nonlinear stochastic systems are given, respectively. Finally, conclusions are proposed and some potential future research works are pointed out.This work was supported in part by the National Natural Science Foundation of China under Grant nos. 61134009, 61329301, 61333012, 61374127 and 11301118, the Engineering and Physical Sciences Research Council (EPSRC) of the UK under Grant no. GR/S27658/01, the Royal Society of the UK, and the Alexander von Humboldt Foundation of Germany

Linearly Solvable Stochastic Control Lyapunov Functions

This paper presents a new method for synthesizing stochastic control Lyapunov functions for a class of nonlinear stochastic control systems. The technique relies on a transformation of the classical nonlinear Hamilton-Jacobi-Bellman partial differential equation to a linear partial differential equation for a class of problems with a particular constraint on the stochastic forcing. This linear partial differential equation can then be relaxed to a linear differential inclusion, allowing for relaxed solutions to be generated using sum of squares programming. The resulting relaxed solutions are in fact viscosity super/subsolutions, and by the maximum principle are pointwise upper and lower bounds to the underlying value function, even for coarse polynomial approximations. Furthermore, the pointwise upper bound is shown to be a stochastic control Lyapunov function, yielding a method for generating nonlinear controllers with pointwise bounded distance from the optimal cost when using the optimal controller. These approximate solutions may be computed with non-increasing error via a hierarchy of semidefinite optimization problems. Finally, this paper develops a-priori bounds on trajectory suboptimality when using these approximate value functions, as well as demonstrates that these methods, and bounds, can be applied to a more general class of nonlinear systems not obeying the constraint on stochastic forcing. Simulated examples illustrate the methodology.Comment: Published in SIAM Journal of Control and Optimizatio

Observer-based H∞ control for systems with repeated scalar nonlinearities and multiple packet losses

This paper is concerned with the H∞ control problem for a class of systems with repeated scalar nonlinearities and multiple missing measurements. The nonlinear system is described by a discrete-time state equation involving a repeated scalar nonlinearity, which typically appears in recurrent neural networks. The measurement missing phenomenon is assumed to occur, simultaneously, in the communication channels from the sensor to the controller and from the controller to the actuator, where the missing probability for each sensor/actuator is governed by an individual random variable satisfying a certain probabilistic distribution in the interval [0 1]. Attention is focused on the analysis and design of an observer-based feedback controller such that the closed-loop control system is stochastically stable and preserves a guaranteed H∞ performance. Sufficient conditions are obtained for the existence of admissible controllers. It is shown that the controller design problem under consideration is solvable if certain linear matrix inequalities (LMIs) are feasible. Three examples are provided to illustrate the effectiveness of the developed theoretical result

Robust H∞ fuzzy output-feedback control with multiple probabilistic delays and multiple missing measurements

Copyright [2010] IEEE. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of Brunel University's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to [email protected]. By choosing to view this document, you agree to all provisions of the copyright laws protecting it.In this paper, the robust H∞-control problem is investigated for a class of uncertain discrete-time fuzzy systems with both multiple probabilistic delays and multiple missing measurements. A sequence of random variables, all of which are mutually independent but obey the Bernoulli distribution, is introduced to account for the probabilistic communication delays. The measurement-missing phenomenon occurs in a random way. The missing probability for each sensor satisfies a certain probabilistic distribution in the interval. Here, the attention is focused on the analysis and design of H∞ fuzzy output-feedback controllers such that the closed-loop Takagi-Sugeno (T-S) fuzzy-control system is exponentially stable in the mean square. The disturbance-rejection attenuation is constrained to a given level by means of the H∞-performance index. Intensive analysis is carried out to obtain sufficient conditions for the existence of admissible output feedback controllers, which ensures the exponential stability as well as the prescribed H∞ performance. The cone-complementarity-linearization procedure is employed to cast the controller-design problem into a sequential minimization one that is solved by the semi-definite program method. Simulation results are utilized to demonstrate the effectiveness of the proposed design technique in this paper.This work was supported in part by the Engineering and Physical Sciences Research Council, U.K., under Grant GR/S27658/01, in part by the Royal Society, U.K., in part by the National Natural Science Foundation of China under Grant 60825303, in part by the National 973 Project of China under Grant 2009CB320600, in part by the Heilongjiang Outstanding Youth Science Fund of China under Grant JC200809, in part by the Youth Science Fund of Heilongjiang Province of China under Grant QC2009C63, and in part by the Alexander von Humboldt Foundation of Germany

Nonlinear analysis of dynamical complex networks

Copyright © 2013 Zidong Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Complex networks are composed of a large number of highly interconnected dynamical units and therefore exhibit very complicated dynamics. Examples of such complex networks include the Internet, that is, a network of routers or domains, the World Wide Web (WWW), that is, a network of websites, the brain, that is, a network of neurons, and an organization, that is, a network of people. Since the introduction of the small-world network principle, a great deal of research has been focused on the dependence of the asymptotic behavior of interconnected oscillatory agents on the structural properties of complex networks. It has been found out that the general structure of the interaction network may play a crucial role in the emergence of synchronization phenomena in various fields such as physics, technology, and the life sciences

Recent advances on filtering and control for nonlinear stochastic complex systems with incomplete information: A survey

This Article is provided by the Brunel Open Access Publishing Fund - Copyright @ 2012 Hindawi PublishingSome recent advances on the filtering and control problems for nonlinear stochastic complex systems with incomplete information are surveyed. The incomplete information under consideration mainly includes missing measurements, randomly varying sensor delays, signal quantization, sensor saturations, and signal sampling. With such incomplete information, the developments on various filtering and control issues are reviewed in great detail. In particular, the addressed nonlinear stochastic complex systems are so comprehensive that they include conventional nonlinear stochastic systems, different kinds of complex networks, and a large class of sensor networks. The corresponding filtering and control technologies for such nonlinear stochastic complex systems are then discussed. Subsequently, some latest results on the filtering and control problems for the complex systems with incomplete information are given. Finally, conclusions are drawn and several possible future research directions are pointed out.This work was supported in part by the National Natural Science Foundation of China under Grant nos. 61134009, 61104125, 61028008, 61174136, 60974030, and 61074129, the Qing Lan Project of Jiangsu Province of China, the Project sponsored by SRF for ROCS of SEM of China, the Engineering and Physical Sciences Research Council EPSRC of the UK under Grant GR/S27658/01, the Royal Society of the UK, and the Alexander von Humboldt Foundation of Germany

Optimal Stabilization using Lyapunov Measures

Numerical solutions for the optimal feedback stabilization of discrete time dynamical systems is the focus of this paper. Set-theoretic notion of almost everywhere stability introduced by the Lyapunov measure, weaker than conventional Lyapunov function-based stabilization methods, is used for optimal stabilization. The linear Perron-Frobenius transfer operator is used to pose the optimal stabilization problem as an infinite dimensional linear program. Set-oriented numerical methods are used to obtain the finite dimensional approximation of the linear program. We provide conditions for the existence of stabilizing feedback controls and show the optimal stabilizing feedback control can be obtained as a solution of a finite dimensional linear program. The approach is demonstrated on stabilization of period two orbit in a controlled standard map