Singularly perturbed piecewise deterministic games

In this paper we consider a class of hybrid stochastic games with the piecewise open-loop information structure. These games are indexed over a parameter $\varepsilon$ which represents the time scale ratio between the stochastic (jump process) and the deterministic (differential state equation) parts of the dynamical system. We study the limit behavior of Nash equilibrium solutions to the hybrid stochastic games when the time scale ratio tends to 0. We also establish that an approximate equilibrium can be obtained for the hybrid stochastic games using a Nash equilibrium solution of a reduced order sequential discrete state stochastic game and a family of local deterministic infinite horizon open-loop differential games defined in the stretched out time scale. A numerical illustration of this approximation scheme is also developed

Control of singularly perturbed hybrid stochastic systems

In this paper, we study a class of optimal stochastic control problems involving two different time scales. The fast mode of the system is represented by deterministic state equations whereas the slow mode of the system corresponds to a jump disturbance process. Under a fundamental “ergodicity” property for a class of “infinitesimal control systems” associated with the fast mode, we show that there exists a limit problem which provides a good approximation to the optimal control of the perturbed system. Both the finite- and infinite-discounted horizon cases are considered. We show how an approximate optimal control law can be constructed from the solution of the limit control problem. In the particular case where the infinitesimal control systems possess the so-called turnpike property, i.e., are characterized by the existence of global attractors, the limit control problem can be given an interpretation related to a decomposition approach

On average control generating families for singularly perturbed optimal control problems with long run average optimality criteria

The paper aims at the development of tools for analysis and construction of near optimal solutions of singularly perturbed (SP) optimal controls problems with long run average optimality criteria. The idea that we exploit is to first asymptotically approximate a given problem of optimal control of the SP system by a certain averaged optimal control problem, then reformulate this averaged problem as an infinite-dimensional (ID) linear programming (LP) problem, and then approximate the latter by semi-infinite LP problems. We show that the optimal solution of these semi-infinite LP problems and their duals (that can be found with the help of a modification of an available LP software) allow one to construct near optimal controls of the SP system. We demonstrate the construction with a numerical example.Comment: 36 pages, 4 figures. arXiv admin note: substantial text overlap with arXiv:1309.373

Control of singularly perturbed hybrid stochastic systems

In this paper we study a class of optimal stochastic control problems involving two different time scales. The fast mode of the system is represented by deterministic state equations whereas the slow mode of the system corresponds to a jump disturbance process. Under a fundamental ”ergodicity” property for a class of ”infinitesimal control systems” associated with the fast mode, we show that there exists a limit problem which provides a good approximation to the optimal control of the perturbed system. Both the finite and infinite discounted horizon cases are considered. We show how an approximate optimal control law can be constructed from the solution of the limit control problem. In the particular case where the infinitesimal control systems possess the so-called turnpike property, i.e. are characterized by the existence of global attractors, the limit control problem can be given an interpretation related to a decomposition approach

Averaging and linear programming in some singularly perturbed problems of optimal control

The paper aims at the development of an apparatus for analysis and construction of near optimal solutions of singularly perturbed (SP) optimal controls problems (that is, problems of optimal control of SP systems) considered on the infinite time horizon. We mostly focus on problems with time discounting criteria but a possibility of the extension of results to periodic optimization problems is discussed as well. Our consideration is based on earlier results on averaging of SP control systems and on linear programming formulations of optimal control problems. The idea that we exploit is to first asymptotically approximate a given problem of optimal control of the SP system by a certain averaged optimal control problem, then reformulate this averaged problem as an infinite-dimensional (ID) linear programming (LP) problem, and then approximate the latter by semi-infinite LP problems. We show that the optimal solution of these semi-infinite LP problems and their duals (that can be found with the help of a modification of an available LP software) allow one to construct near optimal controls of the SP system. We demonstrate the construction with two numerical examples.Comment: 53 pages, 10 figure

Asymptotic Control for a Class of Piecewise Deterministic Markov Processes Associated to Temperate Viruses

We aim at characterizing the asymptotic behavior of value functions in the control of piece-wise deterministic Markov processes (PDMP) of switch type under nonexpansive assumptions. For a particular class of processes inspired by temperate viruses, we show that uniform limits of discounted problems as the discount decreases to zero and time-averaged problems as the time horizon increases to infinity exist and coincide. The arguments allow the limit value to depend on initial configuration of the system and do not require dissipative properties on the dynamics. The approach strongly relies on viscosity techniques, linear programming arguments and coupling via random measures associated to PDMP. As an intermediate step in our approach, we present the approximation of discounted value functions when using piecewise constant (in time) open-loop policies.Comment: In this revised version, statements of the main results are gathered in Section 3. Proofs of the main results (Theorem 4 and Theorem 7) make the object of separate sections (Section 5, resp. Section 6). The biological example makes the object of Section 4. Notations are gathered in Subsection 2.1. This is the final version to be published in SICO

Optimal control of multiscale systems using reduced-order models

We study optimal control of diffusions with slow and fast variables and address a question raised by practitioners: is it possible to first eliminate the fast variables before solving the optimal control problem and then use the optimal control computed from the reduced-order model to control the original, high-dimensional system? The strategy "first reduce, then optimize"--rather than "first optimize, then reduce"--is motivated by the fact that solving optimal control problems for high-dimensional multiscale systems is numerically challenging and often computationally prohibitive. We state sufficient and necessary conditions, under which the "first reduce, then control" strategy can be employed and discuss when it should be avoided. We further give numerical examples that illustrate the "first reduce, then optmize" approach and discuss possible pitfalls

Singularly Perturbed Markov Chains with Two Small Parameters: A Matched Asymptotic Expansion

AbstractThis work is concerned with asymptotic properties of solutions to forward equations for singularly perturbed Markov chains with two small parameters. It is motivated by the model of a cost-minimizing firm involving production planning and capacity expansion and a two-level hierarchical decomposition. Our effort focuses on obtaining asymptotic expansions of the solutions to the forward equation. Different from previous work on singularly perturbed Markov chains, the inner expansion terms are constructed by solving certain partial differential equations. The methods of undetermined coefficients are used. The error bound is obtained

Final Scientific Report: Control Strategies for Complex Systems for Use in Aerospace Avionics

Coordinated Science Laboratory was formerly known as Control Systems LaboratoryAir Force Office of Scientific Research (AFOSR), U.S. Air Force / AF-AFOSR 73-257