Backward stochastic differential equations associated to jump Markov processes and applications

In this paper we study backward stochastic differential equations (BSDEs) driven by the compensated random measure associated to a given pure jump Markov process X on a general state space K. We apply these results to prove well-posedness of a class of nonlinear parabolic differential equations on K, that generalize the Kolmogorov equation of X. Finally we formulate and solve optimal control problems for Markov jump processes, relating the value function and the optimal control law to an appropriate BSDE that also allows to construct probabilistically the unique solution to the Hamilton-Jacobi-Bellman equation and to identify it with the value function

Backward stochastic differential equations and optimal control of marked point processes

We study a class of backward stochastic differential equations (BSDEs) driven by a random measure or, equivalently, by a marked point process. Under appropriate assumptions we prove well-posedness and continuous dependence of the solution on the data. We next address optimal control problems for point processes of general non-markovian type and show that BSDEs can be used to prove existence of an optimal control and to represent the value function. Finally we introduce a Hamilton-Jacobi-Bellman equation, also stochastic and of backward type, for this class of control problems: when the state space is finite or countable we show that it admits a unique solution which identifies the (random) value function and can be represented by means of the BSDEs introduced above

Backward stochastic differential equation driven by a marked point process: An elementary approach with an application to optimal control

We address a class of backward stochastic differential equations on a bounded interval, where the driving noise is a marked, or multivariate, point process. Assuming that the jump times are totally inaccessible and a technical condition holds (see Assumption (A) below), we prove existence and uniqueness results under Lipschitz conditions on the coefficients. Some counter-examples show that our assumptions are indeed needed. We use a novel approach that allows reduction to a (finite or infinite) system of deterministic differential equations, thus avoiding the use of martingale representation theorems and allowing potential use of standard numerical methods. Finally, we apply the main results to solve an optimal control problem for a marked point process, formulated in a classical way.Comment: Published at http://dx.doi.org/10.1214/15-AAP1132 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Halkın Ümitleri boşa çıkmıştır

Taha Toros Arşivi, Dosya No: 256-Osman Bölükbaş

BSDE Representation and Randomized Dynamic Programming Principle for Stochastic Control Problems of Infinite-Dimensional Jump-Diffusions

We consider a general class of stochastic optimal control problems, where the state process lives in a real separable Hilbert space and is driven by a cylindrical Brownian motion and a Poisson random measure; no special structure is imposed on the coefficients, which are also allowed to be path-dependent; in addition, the diffusion coefficient can be degenerate. For such a class of stochastic control problems, we prove, by means of purely probabilistic techniques based on the so-called randomization method, that the value of the control problem admits a probabilistic representation formula (known as non-linear Feynman-Kac formula) in terms of a suitable backward stochastic differential equation. This probabilistic representation considerably extends current results in the literature on the infinite-dimensional case, and it is also relevant in finite dimension. Such a representation allows to show, in the non-path-dependent (or Markovian) case, that the value function satisfies the so-called randomized dynamic programming principle. As a consequence, we are able to prove that the value function is a viscosity solution of the corresponding Hamilton-Jacobi-Bellman equation, which turns out to be a second-order fully non-linear integro-differential equation in Hilbert space

Optimal control of semi-Markov processes with a backward stochastic differential equations approach

In the present work, we employ backward stochastic differential equations (BSDEs) to study the optimal control problem of semi-Markov processes on a finite horizon, with general state and action spaces. More precisely, we prove that the value function and the optimal control law can be represented by means of the solution of a class of BSDEs driven by a semi-Markov process or, equivalently, by the associated random measure. We also introduce a suitable Hamilton\u2013Jacobi\u2013Bellman (HJB) equation. With respect to the pure jump Markov framework, the HJB equation in the semi-Markov case is characterized by an additional differential term 02a. Taking into account the particular structure of semi-Markov processes, we rewrite the HJB equation in a suitable integral form which involves a directional derivative operator D related to 02a. Then, using a formula of Ito^ type tailor-made for semi-Markov processes and the operator D, we are able to prove that a BSDE of the above-mentioned type provides the unique classical solution to the HJB equation, which identifies the value function of our control problem

Feedback optimal control for stochastic Volterra equations with completely monotone kernels.

In this paper we are concerned with a class of stochastic Volterra integro-dierential problems with completely monotone kernels, where we assume that the noise enters the system when we introduce a control. We start by reformulating the state equation into a semilinear evolution equation which can be treated by semigroup methods. The application to optimal control provide other interesting result and require a precise descriprion of the properties of the generated semigroup. The rst main result of the paper is the proof of existence and uniqueness of a mild solution for the corresponding Hamilton-Jacobi-Bellman (HJB) equation. The main technical point consists in the dierentiability of the BSDE associated with the reformulated equation with respect to its initial datum x

Potential of remote sensing and open street data for flood mapping in poorly gauged areas: a case study in Gonaives, Haiti

The Hispaniola Island, in the Caribbean tropical zone, is prone to extreme flood events. Floods are caused by tropical springs and hurricanes and may lead to human losses, economical damages, and spreading of waterborne diseases. Flood studies based upon hydrological and hydraulic modelling are hampered by almost complete lack of hydro-meteorological data. Thenceforth, and given the cost and complexity in the organization of field measurement campaigns, the need for exploitation of remote sensing data, and open source data bases. We present here a feasibility study to explore the potential of (i) high-resolution of digital elevation models (DEMs) from remote imagery and (ii) remotely sensed precipitation data, to feed hydrological flow routing and hydraulic flood modelling, applied to the case study of river La Quinte closed to Gonaives (585 km2), Haiti. We studied one recent flood episode, namely hurricane Ike in 2008, when flood maps from remote sensing were available for validation. The atmospheric input given by hourly rainfall was taken from downscaled Tropical Rainfall Measuring Mission (TRMM) daily estimates, and subsequently fed to a semi-distributed DEM-based hydrological model, providing an hourly flood hydrograph. Then, flood modelling using Hydrologic Engineering Center River Analysis System (HEC-RAS 1D, one-dimensional model for unsteady open channel flow) was carried out under different scenarios of available digital elevation models. The DEMs were generated using optical remote sensing satellite WorldView-1 and Shuttle Radar Topography Mission (SRTM), combined with information from an open source database (OpenStreetMap). Observed flood extent and land use have been extracted using Système Pour l’Observation de la Terre-4 (SPOT-4) imagery. The hydraulic model was tuned for floodplain friction against the observed flooded area. We compared different scenarios of flood simulation and the predictive power given by model tuning. Our study provides acceptable results in depicting flooded areas, especially considering the tremendous lack of ground data, and shows the potential of hydrological modelling approach fed by remote sensing information in Haiti, and in similarly data-scarce areas. Our approach may be useful to provide depiction of flooded areas for the purpose of (i) flood design for urban planning under a frequency-driven approach and (ii) forecasting of flooded areas for warning procedures, pending availability of weather forecast with proper lead time