Different Approaches on Stochastic Reachability as an Optimal Stopping Problem

Reachability analysis is the core of model checking of time systems. For stochastic hybrid systems, this safety verification method is very little supported mainly because of complexity and difficulty of the associated mathematical problems. In this paper, we develop two main directions of studying stochastic reachability as an optimal stopping problem. The first approach studies the hypotheses for the dynamic programming corresponding with the optimal stopping problem for stochastic hybrid systems. In the second approach, we investigate the reachability problem considering approximations of stochastic hybrid systems. The main difficulty arises when we have to prove the convergence of the value functions of the approximating processes to the value function of the initial process. An original proof is provided

Dynamic Management of Portfolios with Transaction Costs under Tychastic Uncertainty.

We use in this chapter the viability/capturability approach for studying the problem of dynamic valuation and management of a portfolio with transaction costs in the framework of tychastic control systems (or dynamical games against nature) instead of stochastic control systems. Indeed, the very definition of the guaranteed valuation set can be formulated directly in terms of guaranteed viable-capture basin of a dynamical game. Hence, we shall “compute” the guaranteed viable-capture basin and find a formula for the valuation function involving an underlying criterion, use the tangential properties of such basins for proving that the valuation function is a solution to Hamilton-Jacobi-Isaacs partial differential equations. We then derive a dynamical feedback providing an adjustment law regulating the evolution of the portfolios obeying viability constraints until it achieves the given objective in finite time. We shall show that the Pujal—Saint-Pierre viability/capturability algorithm applied to this specific case provides both the valuation function and the associated portfolios.dynamic games; dynamic valuation; tychastic control systems; management of portfolio;

Controlled diffusion processes

This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.Comment: Published at http://dx.doi.org/10.1214/154957805100000131 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org

Cloud-Based Centralized/Decentralized Multi-Agent Optimization with Communication Delays

We present and analyze a computational hybrid architecture for performing multi-agent optimization. The optimization problems under consideration have convex objective and constraint functions with mild smoothness conditions imposed on them. For such problems, we provide a primal-dual algorithm implemented in the hybrid architecture, which consists of a decentralized network of agents into which centralized information is occasionally injected, and we establish its convergence properties. To accomplish this, a central cloud computer aggregates global information, carries out computations of the dual variables based on this information, and then distributes the updated dual variables to the agents. The agents update their (primal) state variables and also communicate among themselves with each agent sharing and receiving state information with some number of its neighbors. Throughout, communications with the cloud are not assumed to be synchronous or instantaneous, and communication delays are explicitly accounted for in the modeling and analysis of the system. Experimental results are presented to support the theoretical developments made.Comment: 8 pages, 4 figure

On Singular Control Problems with State Constraints and Regime-Switching: A Viscosity Solution Approach

This paper investigates a singular stochastic control problem for a multi-dimensional regime-switching diffusion process confined in an unbounded domain. The objective is to maximize the total expected discounted rewards from exerting the singular control. Such a formulation stems from application areas such as optimal harvesting multiple species and optimal dividends payments schemes in random environments. With the aid of weak dynamic programming principle and an exponential transformation, we characterize the value function to be the unique constrained viscosity solution of a certain system of coupled nonlinear quasi-variational inequalities. Several examples are analyzed in details to demonstrate the main results

A unified framework for hybrid control : b background, model, and theory

Caption title. "April 1994: Revised June 1994."Includes bibliographical references (p. 24-25).Supported by the Army Research Office and the Center for Intelligent Control Systems. DAAL03-92-G-0164 DAAL03-92-G-0115Michael S. Branicky, Vivek S. Borkar, Sanjoy K. Mitter