Identification and Optimal Control of Large-Scale Systems Using Selective Decentralization

In this paper, we explore the capability of selective decentralization in improving the control performance for unknown large-scale systems using model-based approaches. In selective decentralization, we explore all of the possible communication policies among subsystems and show that with the appropriate switching among the resulting multiple identification models (with corresponding communication policies), such selective decentralization significantly outperforms a centralized identification model when the system is weakly interconnected, and performs at least equivalent to the centralized model when the system is strongly interconnected. To derive the sub-optimal control, our control design include two phases. First, we apply system identification to train the approximation model for the unknown system. Second, we find the suboptimal solution of the Halminton-Jacobi-Bellman (HJB) equation to derive the suboptimal control. In linear systems, the HJB equation transforms to the well-solved Riccati equation with closed-form solution. In nonlinear systems, we discretize the approximation model in order to acquire the control unit by using dynamic programming methods for the resulting Markov Decision Process (MDP). We compare the performance among the selective decentralization, the complete decentralization and the centralization in our two-phase control design. Our results show that selective decentralization outperforms the complete decentralization and the centralization approaches when the systems are completely decoupled or strongly interconnected

Nondeterministic hybrid dynamical systems

This thesis is concerned with the analysis, control and identification of hybrid dynamical systems. The main focus is on a particular class of hybrid systems consisting of linear subsystems. The discrete dynamic, i.e., the change between subsystems, is unknown or nondeterministic and cannot be influenced, i.e. controlled, directly. However changes in the discrete dynamic can be detected immediately, such that the current dynamic (subsystem) is known. In order to motivate the study of hybrid systems and show the merits of hybrid control theory, an example is given. It is shown that real world systems like Anti Locking Brakes (ABS) are naturally modelled by such a class of linear hybrids systems. It is shown that purely continuous feedback is not suitable since it cannot achieve maximum braking performance. A hybrid control strategy, which overcomes this problem, is presented. For this class of linear hybrid system with unknown discrete dynamic, a framework for robust control is established. The analysis methodology developed gives a robustness radius such that the stability under parameter variations can be analysed. The controller synthesis procedure is illustrated in a practical example where the control for an active suspension of a car is designed. Optimal control for this class of hybrid system is introduced. It is shows how a control law is obtained which minimises a quadratic performance index. The synthesis procedure is stated in terms of a convex optimisation problem using linear matrix inequalities (LMI). The solution of the LMI not only returns the controller but also the performance bound. Since the proposed controller structures require knowledge of the continuous state, an observer design is proposed. It is shown that the estimation error converges quadratically while minimising the covariance of the estimation error. This is similar to the Kalman filter for discrete or continuous time systems. Further, we show that the synthesis of the observer can be cast into an LMI, which conveniently solves the synthesis problem

Two-phase Selective Decentralization to Improve Reinforcement Learning Systems with MDP

In this paper, we explore the capability of selective decentralization in improving the reinforcement learning performance for unknown systems using model-based approaches. In selective decentralization, we automatically select the best communication policies among agents. Our learning design, which is built on the control system principles, includes two phases. First, we apply system identification to train an approximated model for the unknown systems. Second, we find the suboptimal solution of the Hamilton–Jacobi–Bellman (HJB) equation to derive the suboptimal control. For linear systems, the HJB equation transforms to the well-known Riccati equation with closed-form solution. In nonlinear system, we discretize the approximation model as a Markov Decision Process (MDP) in order to determine the control using dynamic programming algorithms. Since the theoretical foundation of using MDP to control the nonlinear system has not been thoroughly developed, we prove that the control law learned by the discrete-MDP approach is guarantee to stabilize the system, which is the learning goal, given several sufficient conditions. These learning and control techniques could be applied in centralized, completely decentralized and selectively decentralized manner. Our results show that selective decentralization outperforms the complete decentralization and the centralization approaches when the systems are completely decoupled or strongly interconnected

Reverse Engineering Biological Control Systems for Applications in Process Control.

The main emphasis of this dissertation is the development of nonlinear control strategies based on biological control systems. Commonly utilized biological control schemes have been studied in order to reverse engineer the important concepts for applications in process control. This approach has led to the development of a nonlinear habituating control strategy and nonlinear model reference adaptive control schemes. Habituating control is a controller design strategy for nonlinear systems with more manipulated inputs than controlled outputs. Nonlinear control laws that provide input-output linearization while simultaneously minimizing the cost of affecting control are derived. Local stability analysis shows the controller can provide a simple solution to singularity and non-minimum phase problems. A direct adaptive control strategy for a class of single-input, single-output non-linear systems is presented. The major advantage is that a detailed dynamic non-linear model is not required for controller design. Unknown controller functions in the associated input-output linearizing control law are approximated using locally supported radial basis functions. Lyapunov stability analysis is used to derive parameter update laws which ensure the state vector remains bounded and the plant output asymptotically tracks the output of a linear reference model. A nonlinear model reference adaptive control strategy in which a linear model (or multiple linear models) is embedded within the nonlinear controller is presented. The nonlinear control law is constructed by embedding linear controller gains derived from models obtained using standard linear system identification techniques within the associated input-output linearizing control law. Higher-order controller functions are approximated with radial basis functions. Lyapunov stability analysis is used to derive stable parameter update laws. The major disadvantage of the previous techniques is computational expense. Two modifications have been developed. First, the effective dimension is reduced by applying nonlinear principal component analysis to the state variable data obtained from open-loop tests. This allows basis functions to be placed in a lower dimensional space than the original state space. Second, the total number of basis functions is fixed a priori and an algorithm which adds/prunes basis function centers to surround the current operating point on-line is utilized

Kernel-based system identification from noisy and incomplete input-output data

In this contribution, we propose a kernel-based method for the identification of linear systems from noisy and incomplete input-output datasets. We model the impulse response of the system as a Gaussian process whose covariance matrix is given by the recently introduced stable spline kernel. We adopt an empirical Bayes approach to estimate the posterior distribution of the impulse response given the data. The noiseless and missing data samples, together with the kernel hyperparameters, are estimated maximizing the joint marginal likelihood of the input and output measurements. To compute the marginal-likelihood maximizer, we build a solution scheme based on the Expectation-Maximization method. Simulations on a benchmark dataset show the effectiveness of the method.Comment: 16 pages, submitted to IEEE Conference on Decision and Control 201