Second order parabolic Hamilton–Jacobi–Bellman equations in Hilbert spaces and stochastic control: Lμ2 approach

AbstractWe study a Hamilton–Jacobi–Bellman equation related to the optimal control of a stochastic semilinear equation on a Hilbert space X. We show the existence and uniqueness of solutions to the HJB equation and prove the existence and uniqueness of feedback controls for the associated control problem via dynamic programming. The main novelty is that we look for solutions in the space L2(X,μ), where μ is an invariant measure for an associated uncontrolled process. This allows us to treat controlled systems with degenerate diffusion term that are not covered by the existing literature. In particular, we prove the existence and uniqueness of solutions and obtain the optimal feedbacks for controlled stochastic delay equations and for the first order stochastic PDE’s arising in economic and financial models

Path-dependent equations and viscosity solutions in infinite dimension

Path-dependent PDEs (PPDEs) are natural objects to study when one deals with non Markovian models. Recently, after the introduction of the so-called pathwise (or functional or Dupire) calculus (see [15]), in the case of finite-dimensional underlying space various papers have been devoted to studying the well-posedness of such kind of equations, both from the point of view of regular solutions (see e.g. [15, 9]) and viscosity solutions (see e.g. [16]). In this paper, motivated by the study of models driven by path-dependent stochastic PDEs, we give a first well-posedness result for viscosity solutions of PPDEs when the underlying space is a separable Hilbert space. We also observe that, in contrast with the finite-dimensional case, our well-posedness result, even in the Markovian case, applies to equations which cannot be treated, up to now, with the known theory of viscosity solutions.Comment: To appear in the Annals of Probabilit

Entropic and displacement interpolation: a computational approach using the Hilbert metric

Monge-Kantorovich optimal mass transport (OMT) provides a blueprint for geometries in the space of positive densities -- it quantifies the cost of transporting a mass distribution into another. In particular, it provides natural options for interpolation of distributions (displacement interpolation) and for modeling flows. As such it has been the cornerstone of recent developments in physics, probability theory, image processing, time-series analysis, and several other fields. In spite of extensive work and theoretical developments, the computation of OMT for large scale problems has remained a challenging task. An alternative framework for interpolating distributions, rooted in statistical mechanics and large deviations, is that of Schroedinger bridges (entropic interpolation). This may be seen as a stochastic regularization of OMT and can be cast as the stochastic control problem of steering the probability density of the state-vector of a dynamical system between two marginals. In this approach, however, the actual computation of flows had hardly received any attention. In recent work on Schroedinger bridges for Markov chains and quantum evolutions, we noted that the solution can be efficiently obtained from the fixed-point of a map which is contractive in the Hilbert metric. Thus, the purpose of this paper is to show that a similar approach can be taken in the context of diffusion processes which i) leads to a new proof of a classical result on Schroedinger bridges and ii) provides an efficient computational scheme for both, Schroedinger bridges and OMT. We illustrate this new computational approach by obtaining interpolation of densities in representative examples such as interpolation of images.Comment: 20 pages, 7 figure

Stochastic Minimum Principle for Partially Observed Systems Subject to Continuous and Jump Diffusion Processes and Driven by Relaxed Controls

In this paper we consider non convex control problems of stochastic differential equations driven by relaxed controls. We present existence of optimal controls and then develop necessary conditions of optimality. We cover both continuous diffusion and Jump processes.Comment: Pages 23, Submitted to SIAM Journal on Control and Optimizatio

Differentiability of backward stochastic differential equations in Hilbert spaces with monotone generators

The aim of the present paper is to study the regularity properties of the solution of a backward stochastic differential equation with a monotone generator in infinite dimension. We show some applications to the nonlinear Kolmogorov equation and to stochastic optimal control

Linear PDEs and eigenvalue problems corresponding to ergodic stochastic optimization problems on compact manifolds

We consider long term average or `ergodic' optimal control poblems with a special structure: Control is exerted in all directions and the control costs are proportional to the square of the norm of the control field with respect to the metric induced by the noise. The long term stochastic dynamics on the manifold will be completely characterized by the long term density $\rho$ and the long term current density $J$. As such, control problems may be reformulated as variational problems over $\rho$ and $J$. We discuss several optimization problems: the problem in which both $\rho$ and $J$ are varied freely, the problem in which $\rho$ is fixed and the one in which $J$ is fixed. These problems lead to different kinds of operator problems: linear PDEs in the first two cases and a nonlinear PDE in the latter case. These results are obtained through through variational principle using infinite dimensional Lagrange multipliers. In the case where the initial dynamics are reversible we obtain the result that the optimally controlled diffusion is also symmetrizable. The particular case of constraining the dynamics to be reversible of the optimally controlled process leads to a linear eigenvalue problem for the square root of the density process

Mild solutions of semilinear elliptic equations in Hilbert spaces

This paper extends the theory of regular solutions ($C^1$ in a suitable sense) for a class of semilinear elliptic equations in Hilbert spaces. The notion of regularity is based on the concept of $G$-derivative, which is introduced and discussed. A result of existence and uniqueness of solutions is stated and proved under the assumption that the transition semigroup associated to the linear part of the equation has a smoothing property, that is, it maps continuous functions into $G$-differentiable ones. The validity of this smoothing assumption is fully discussed for the case of the Ornstein-Uhlenbeck transition semigroup and for the case of invertible diffusion coefficient covering cases not previously addressed by the literature. It is shown that the results apply to Hamilton-Jacobi-Bellman (HJB) equations associated to infinite horizon optimal stochastic control problems in infinite dimension and that, in particular, they cover examples of optimal boundary control of the heat equation that were not treatable with the approaches developed in the literature up to now