28 research outputs found
Robustness of Stochastic Optimal Control to Approximate Diffusion Models under Several Cost Evaluation Criteria
In control theory, typically a nominal model is assumed based on which an
optimal control is designed and then applied to an actual (true) system. This
gives rise to the problem of performance loss due to the mismatch between the
true model and the assumed model. A robustness problem in this context is to
show that the error due to the mismatch between a true model and an assumed
model decreases to zero as the assumed model approaches the true model. We
study this problem when the state dynamics of the system are governed by
controlled diffusion processes. In particular, we will discuss continuity and
robustness properties of finite horizon and infinite-horizon
-discounted/ergodic optimal control problems for a general class of
non-degenerate controlled diffusion processes, as well as for optimal control
up to an exit time. Under a general set of assumptions and a convergence
criterion on the models, we first establish that the optimal value of the
approximate model converges to the optimal value of the true model. We then
establish that the error due to mismatch that occurs by application of a
control policy, designed for an incorrectly estimated model, to a true model
decreases to zero as the incorrect model approaches the true model. We will see
that, compared to related results in the discrete-time setup, the
continuous-time theory will let us utilize the strong regularity properties of
solutions to optimality (HJB) equations, via the theory of uniformly elliptic
PDEs, to arrive at strong continuity and robustness properties.Comment: 33 page
Average cost optimal control under weak ergodicity hypotheses: Relative value iterations
We study Markov decision processes with Polish state and action spaces. The
action space is state dependent and is not necessarily compact. We first
establish the existence of an optimal ergodic occupation measure using only a
near-monotone hypothesis on the running cost. Then we study the well-posedness
of Bellman equation, or what is commonly known as the average cost optimality
equation, under the additional hypothesis of the existence of a small set. We
deviate from the usual approach which is based on the vanishing discount method
and instead map the problem to an equivalent one for a controlled split chain.
We employ a stochastic representation of the Poisson equation to derive the
Bellman equation. Next, under suitable assumptions, we establish convergence
results for the 'relative value iteration' algorithm which computes the
solution of the Bellman equation recursively. In addition, we present some
results concerning the stability and asymptotic optimality of the associated
rolling horizon policies.Comment: 32 page
Yet again on iteration improvement for averaged expected cost control for 1D ergodic diffusions
The paper is a full version of the short presentation in \cite{amv17}.
Ergodic control for one-dimensional controlled diffusion is tackled; both drift
and diffusion coefficients may depend on a strategy which is assumed markovian.
Ergodic HJB equation is established and existence and uniqueness of its
solution is proved, as well as the convergence of the reward improvement
algorithm.Comment: 28 pages, 30 reference
International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book
The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions.
This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more
New Directions for Contact Integrators
Contact integrators are a family of geometric numerical schemes which
guarantee the conservation of the contact structure. In this work we review the
construction of both the variational and Hamiltonian versions of these methods.
We illustrate some of the advantages of geometric integration in the
dissipative setting by focusing on models inspired by recent studies in
celestial mechanics and cosmology.Comment: To appear as Chapter 24 in GSI 2021, Springer LNCS 1282