Perturbation of Linear Quadratic Systems with Jump Parameters and Hybrid Controls

We consider the problem of the perturbation of a class of linear-quadratic differential games with piecewise deterministic dynamics, where the changes from one structure (for the dynamics) to another are governed by a finite-stat- e Markov process. Player 1 controls the continuous dynamics, whereas Player 2 controls the rate of transition for the finite-state Markov process; both have access to the states of both processes. Player 1 wishes to minimize a given quadratic performance index, while player 2 wishes to maximize or minimize the same quantity. The problem above leads to the analysis of some linearly coupled set of quadratic equations (Riccati Equation). We obtain a Taylor expansion in the perturbation for the solution of the equation for a fixed stationary policy of the player 2. This allows us to solve the game or team problem as a function of the perturbation

Almost Sure Stabilization for Adaptive Controls of Regime-switching LQ Systems with A Hidden Markov Chain

This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a hidden Markov chain. Different from previous work on stabilization of adaptive controlled systems with a hidden Markov chain, where average criteria were considered, this work focuses on the almost sure stabilization or sample path stabilization of the underlying processes. Under simple conditions, it is shown that as long as the feedback controls have linear growth in the continuous component, the resulting process is regular. Moreover, by appropriate choice of the Lyapunov functions, it is shown that the adaptive system is stabilizable almost surely. As a by-product, it is also established that the controlled process is positive recurrent

Control of singularly perturbed hybrid stochastic systems

In this paper, we study a class of optimal stochastic control problems involving two different time scales. The fast mode of the system is represented by deterministic state equations whereas the slow mode of the system corresponds to a jump disturbance process. Under a fundamental “ergodicity” property for a class of “infinitesimal control systems” associated with the fast mode, we show that there exists a limit problem which provides a good approximation to the optimal control of the perturbed system. Both the finite- and infinite-discounted horizon cases are considered. We show how an approximate optimal control law can be constructed from the solution of the limit control problem. In the particular case where the infinitesimal control systems possess the so-called turnpike property, i.e., are characterized by the existence of global attractors, the limit control problem can be given an interpretation related to a decomposition approach

The linear quadratic regulator problem for a class of controlled systems modeled by singularly perturbed Ito differential equations

This paper discusses an infinite-horizon linear quadratic (LQ) optimal control problem involving state- and control-dependent noise in singularly perturbed stochastic systems. First, an asymptotic structure along with a stabilizing solution for the stochastic algebraic Riccati equation (ARE) are newly established. It is shown that the dominant part of this solution can be obtained by solving a parameter-independent system of coupled Riccati-type equations. Moreover, sufficient conditions for the existence of the stabilizing solution to the problem are given. A new sequential numerical algorithm for solving the reduced-order AREs is also described. Based on the asymptotic behavior of the ARE, a class of O(√ε) approximate controller that stabilizes the system is obtained. Unlike the existing results in singularly perturbed deterministic systems, it is noteworthy that the resulting controller achieves an O(ε) approximation to the optimal cost of the original LQ optimal control problem. As a result, the proposed control methodology can be applied to practical applications even if the value of the small parameter ε is not precisely known. © 2012 Society for Industrial and Applied Mathematics.Vasile Dragan, Hiroaki Mukaidani and Peng Sh

Estimation and control of non-linear and hybrid systems with applications to air-to-air guidance

Issued as Progress report, and Final report, Project no. E-21-67