An Excursion-Theoretic Approach to Stability of Discrete-Time Stochastic Hybrid Systems

We address stability of a class of Markovian discrete-time stochastic hybrid systems. This class of systems is characterized by the state-space of the system being partitioned into a safe or target set and its exterior, and the dynamics of the system being different in each domain. We give conditions for $L_1$-boundedness of Lyapunov functions based on certain negative drift conditions outside the target set, together with some more minor assumptions. We then apply our results to a wide class of randomly switched systems (or iterated function systems), for which we give conditions for global asymptotic stability almost surely and in $L_1$. The systems need not be time-homogeneous, and our results apply to certain systems for which functional-analytic or martingale-based estimates are difficult or impossible to get.Comment: Revised. 17 pages. To appear in Applied Mathematics & Optimizatio

On the ergodic decomposition for a class of Markov chains

In this paper we present sufficient conditions for the Doeblin decomposition, and necessary and sufficient conditions for an ergodic decomposition for a Markov chain satisfying a T'-condition, which is a condition adapted from the paper (Statist. and Probab. Lett. 50 (2000) 13). Under no separability assumption on the [sigma]-field, it is shown that the T'-condition is sufficient for the condition that there are no uncountable disjoint absorbing sets and, under some hypothesis, it is also necessary. For the case in which the [sigma]-field is countable generated and separated, this condition is equivalent to the existence of a T continuous component for the Markov chain. Furthermore, under the assumption that the space is a compact separable metric space, it is shown that the Foster-Lyapunov criterion is necessary and sufficient for the existence of an invariant probability measure for the Markov chain, and that every probability measure for the Markov chain is, in this case, non-singular.Markov chain Invariant probability measures Countable ergodic decomposition

Uncoupled Riccati Iterations For The Linear Quadratic Control Problem Of Discrete-time Markov Jump Linear Systems

This paper deals with recursive methods for solving coupled Riccati equations arising in the linear quadratic control for Markovian jump linear systems. Two algorithms, based on solving uncoupled Riccati equations at each iteration, are presented. The standard method for this problem relies on finite stage approximations with receding horizon, whereas the methods presented here are based on sequences of stopping times to define the terminal time of the approximating control problems. The methods can be ordered in terms of rate of convergence. Comparisons with other methods in the current literature are also presented.431217271733Abou-Kandil, H., Freiling, G., Jank, G., On the solution of discrete-time Markovian jump linear quadratic control problems (1995) Automatica, 31 (5), pp. 765-768Ait-Rami, M., Ghaoui, L.E., LMI optimization for nonstandard Riccati equations arising in stochastic control (1996) IEEE Trans. Automat. Contr., 41, pp. 1666-1671. , NovCosta, O.L.V., Do Val, J.B.R., Geromel, J.C., A convex programming approach to H2-control of discrete-time Markovian jump linear systems (1997) Int. J. Contr., 66, pp. 557-579Costa, O.L.V., Fragoso, M.D., Stability results for discrete-time linear systems with Markovian jumping parameters (1993) J. Math. Analysis and Appl., 179, pp. 154-178Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems (1995) IEEE Trans. Automat. Contr., 40, pp. 2076-2088Davis, M.H.A., Vinter, R.B., (1985) Stochastic Modeling and Control, , London, U.K.: Chapman and HallEl Ghaoui, L., Nikoukhah, R., Delebecque, F., LMITOOL: A Front-End for LMI Optimization - User's Guide, , ftp.ensa.fr/pub/elghaoui/lmitoolFragoso, M.D., Ribeiro Do Val, J.B., Pinto Jr., D.L., Jump linear H∞-control: The discrete-time case (1995) Contr. Th. and Adv. Tech., 10, pp. 1459-1474Gajic, Z., Borno, I., Lyapunov iterations for optimal control of jump linear systems at steady state (1995) IEEE Trans. Automat. Contr., 40 (11), pp. 1971-11075Geromel, J.C., Peres, P.L.D., Souza, S.R., H2-guaranteed cost control for uncertain discrete-time linear systems (1993) Int. J. Contr., 57, pp. 853-864Ji, Y., Chizeck, H.J., Controllability, observability and discrete-time Markovian jump linear quadratic control (1988) Int. J. Contr., 48, pp. 481-498Ji, Y., Chizeck, H.J., Feng, X., Loparo, K.A., Stability and control of discrete-time jump linear systems (1991) Control Th. and Adv. Tech., 7, pp. 247-270Laub, A., Algebraic aspects of generalized eigenvalue problems for solving Riccati equations (1986) Computational and Combinatorial Methods in Systems Theory, pp. 213-227. , C. Byrnes and A. Lindquist, Eds. Amsterdam, The Netherlands: ElsevierMariton, M., (1990) Jump Linear Systems in Automatic Control, , New York: Marcel DekkerVandenberghe, L., Boyd, S., Software for Semidefinite Programming - User's Guide, , isl.stanford.edu/pub/boyd/semidef_progWimmer, H.K., Monotonicity and maximality of solutions of discrete-time algebraic Riccati equations (1992) J. Math. Syst., Estimation and Contr., 2 (2), pp. 219-23

Continuous-time State-feedback H 2-control Of Markovian Jump Linear Systems Via Convex Analysis

Continuous-time H 2-control problem for the class of linear systems with Markovian jumps (MJLS) using convex analysis is considered in this paper. The definition of the H 2-norm for continuous-time MJLS is presented and related to the appropriate observability and controllability Gramians. A convex programming formulation for the H 2-control problem of MJLS is developed. That enables us to tackle the optimization problem of MJLS under the assumption that the transition rate matrix Π = [π ij] for the Markov chain may not be exactly known, but belongs to an appropriate convex set. An equivalence between the convex formulation when Π is exactly known and the usual dynamic programming approach of quadratic optimal control of MJLS is established. It is shown that there exists a solution for the convex programming problem if and only if there exists the mean-square stabilizing solution for a set of coupled algebraic Riccati equations. These results are compared with other related works in the current literature. © 1999 Elsevier Science Ltd. All rights reserved.352259268Abou-Khandil, H., Freiling, G., Jank, G., Solution and asymptotic behavior of coupled Riccati equations in jump linear systems (1994) IEEE Trans. on Automat. Control, 39, pp. 1631-1636Blair Jr., W.P., Sworder, D.D., Continuous-time regulation of a class of econometric models (1975) IEEE Trans. Systems Man Cybernet, 5, pp. 341-346Boyd, S., El Ghaoui, L., Feron, E., Balakrishnan, V., (1994) Linear Matrix Inequalities and Control Theory, , Philadelphia: SIAMEl Ghaoui, E., Nikoukhah, R., Delebecque, F., (1995) LMITOOL: A Front-end for LMI Optimization -User's Guide, , ftp.ensa.fr/pub/elghaoui/lmitoolCosta, O.L.V., Do Val, J.B.R., Geromel, J.C., A convex programming approach to H 2-control of discrete-time Markovian jump linear systems (1997) Int. J. Control, 66, pp. 557-579Gajic, Z., Borno, I., Lyapunov iterations for optimal control of jump linear systems at steady state (1995) IEEE Trans. Automat. Control, 40, pp. 1971-1975Feng, X., Loparo, K.A., Ji, Y., Chizeck, H.J., Stochastic stability properties of jump linear systems (1992) IEEE Trans. Automat. Control, 37, pp. 38-53Ji, Y., Chizeck, H.J., Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control (1990) IEEE Trans. Automat. Control, 35, pp. 777-788Mariton, M., Almost sure and moments stability of jump linear systems (1988) Systems and Control Lett., 11, pp. 393-397Mariton, M., (1990) Jump Linear Systems in Automatic Control, , New York: Marcel DekkerRami, M.A., El Ghaoui, E., Robust state-feedback stabilization of jump linear systems via LMIs (1996) Int. J. Robust Nonlinear Control, 6, pp. 1015-1022Rami, M.A., El Ghaoui, E., LMI optimization for nonstandard Riccati equations arising in stochastic control (1996) IEEE Trans. Automat. Control, 41, pp. 1666-1671Vandenberghe, L., Boyd, S., (1994) Software for Semidefinite Programming -User's Guide, , isl.stanford.edu/pub/boyd/semidef_pro

A linear programming formulation for constrained discounted continuous control for piecewise deterministic Markov processes

This paper deals with the constrained discounted control of piecewise deterministic Markov process (PDMPs) in general Borel spaces. The control variable acts on the jump rate and transition measure, and the goal is to minimize the total expected discounted cost, composed of positive running and boundary costs, while satisfying some constraints also in this form. The basic idea is, by using the special features of the PDMPs, to re-write the problem via an embedded discrete-time Markov chain associated to the PDMP and re-formulate the problem as an infinite dimensional linear programming (LP) problem, via the occupation measures associated to the discrete-time process. It is important to stress however that our new discrete-time problem is not in the same framework of a general constrained discrete-time Markov Decision Process and, due to that, some conditions are required to get the equivalence between the continuous-time problem and the LP formulation. We provide in the sequel sufficient conditions for the solvability of the associated LP problem, based on a generalization of Theorem 4.1 in [8]. In Appendix A we present the proof of this generalization which, we believe, is of interest on its own. The paper is concluded with some examples to illustrate the obtained results

State-feedback control of positive switching systems with Markovian jumps

none4This chapter deals with positive linear systems in continuous-time affected by a switching signal representing a disturbance driven by a Markov chain. A state-feedback control law has to be designed in order to ensure mean stability and input-output L∞-induced or L1-induced mean performance. The chapter is divided into two parts. In the first,the control action is based on the knowledge of both the state of the system and the sample path of the Markovian process (mode-dependent control). In the second,instead,only the state-variable is known (mode-independent control). In the mode-dependent case,as well as in the single-input mode-independent case,necessary and sufficient conditions for the existence of feasible feedback gains are provided based on linear programming tools,also yielding a full parametrization of feasible solutions. In the multi-input mode-independent case,sufficient conditions are worked out in terms of convex programming. Some numerical examples illustrate the theory.Colaneri, Patrizio; Bolzern, Paolo; Geromel, José C.; Deaecto, Grace S.Colaneri, Patrizio; Bolzern, PAOLO GIUSEPPE EMILIO; Geromel, José C.; Deaecto, Grace S