Almost Sure Stabilization for Adaptive Controls of Regime-switching LQ Systems with A Hidden Markov Chain

This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a hidden Markov chain. Different from previous work on stabilization of adaptive controlled systems with a hidden Markov chain, where average criteria were considered, this work focuses on the almost sure stabilization or sample path stabilization of the underlying processes. Under simple conditions, it is shown that as long as the feedback controls have linear growth in the continuous component, the resulting process is regular. Moreover, by appropriate choice of the Lyapunov functions, it is shown that the adaptive system is stabilizable almost surely. As a by-product, it is also established that the controlled process is positive recurrent

On the continuous dependence with respect to sampling of the linear quadratic regulator problem for distributed parameter systems

The convergence of solutions to the discrete or sampled time linear quadratic regulator problem and associated Riccati equation for infinite dimensional systems to the solutions to the corresponding continuous time problem and equation, as the length of the sampling interval (the sampling rate) tends toward zero (infinity) is established. Both the finite and infinite time horizon problems are studied. In the finite time horizon case, strong continuity of the operators which define the control system and performance index together with a stability and consistency condition on the sampling scheme are required. For the infinite time horizon problem, in addition, the sampled systems must be stabilizable and detectable, uniformly with respect to the sampling rate. Classes of systems for which this condition can be verified are discussed. Results of numerical studies involving the control of a heat/diffusion equation, a hereditary of delay system, and a flexible beam are presented and discussed

p

The problem of pth mean exponential stability and stabilizability of a class of stochastic nonlinear and bilinear hybrid systems with unstable and stable subsystems is considered. Sufficient conditions for the pth mean exponential stability and stabilizability under a feedback control and stabilizing switching rules are derived. A method for the construction of stabilizing switching rules based on the Lyapunov technique and the knowledge of regions of decreasing the Lyapunov functions for subsystems is given. Two cases, including single Lyapunov function and a a single Lyapunov-like function, are discussed. Obtained results are illustrated by examples

Stability of singular jump-linear systems with a large state space : a two-time-scale approach

This paper considers singular systems that involve both continuous dynamics and discrete events with the coefficients being modulated by a continuous-time Markov chain. The underlying systems have two distinct characteristics. First, the systems are singular, that is, characterized by a singular coefficient matrix. Second, the Markov chain of the modulating force has a large state space. We focus on stability of such hybrid singular systems. To carry out the analysis, we use a two-time-scale formulation, which is based on the rationale that, in a large-scale system, not all components or subsystems change at the same speed. To highlight the different rates of variation, we introduce a small parameter ε>0. Under suitable conditions, the system has a limit. We then use a perturbed Lyapunov function argument to show that if the limit system is stable then so is the original system in a suitable sense for ε small enough. This result presents a perspective on reduction of complexity from a stability point of view

Approximation methods for hybrid diffusion systems with state-dependent switching processes : numerical algorithms and existence and uniqueness of solutions

By focusing on hybrid diffusions in which continuous dynamics and discrete events coexist, this work is concerned with approximation of solutions for hybrid stochastic differential equations with a state-dependent switching process. Iterative algorithms are developed. The continuous-state dependent switching process presents added difficulties in analyzing the numerical procedures. Weak convergence of the algorithms is established by a martingale problem formulation first. This weak convergence result is then used as a bridge to obtain strong convergence. In this process, the existence and uniqueness of the solution of the switching diffusions with continuous-state-dependent switching are obtained. Different from the existing results of solutions of stochastic differential equations in which the Picard iterations are utilized, Euler's numerical schemes are considered here. Moreover, decreasing stepsize algorithms together with their weak convergence are given. Numerical experiments are also provided for demonstration

LQ-optimal Sample-data Control under Stochastic Delays: Gridding Approach for Stabilizability and Detectability

We solve a linear quadratic optimal control problem for sampled-data systems with stochastic delays. The delays are stochastically determined by the last few delays. The proposed optimal controller can be efficiently computed by iteratively solving a Riccati difference equation, provided that a discrete-time Markov jump system equivalent to the sampled-data system is stochastic stabilizable and detectable. Sufficient conditions for these notions are provided in the form of linear matrix inequalities, from which stabilizing controllers and state observers can be constructed.Comment: 28 pages, 3 figure