5 research outputs found
Singular perturbations in stochastic optimal control with unbounded data
We study singular perturbations of a class of two-scale stochastic control
systems with unbounded data. The assumptions are designed to cover some
relaxation problems for deep neural networks. We construct effective
Hamiltonian and initial data and prove the convergence of the value function to
the solution of a limit (effective) Cauchy problem for a parabolic equation of
HJB type. We use methods of probability, viscosity solutions and
homogenization.Comment: 26 page
On average control generating families for singularly perturbed optimal control problems with long run average optimality criteria
The paper aims at the development of tools for analysis and construction of
near optimal solutions of singularly perturbed (SP) optimal controls problems
with long run average optimality criteria. The idea that we exploit is to first
asymptotically approximate a given problem of optimal control of the SP system
by a certain averaged optimal control problem, then reformulate this averaged
problem as an infinite-dimensional (ID) linear programming (LP) problem, and
then approximate the latter by semi-infinite LP problems. We show that the
optimal solution of these semi-infinite LP problems and their duals (that can
be found with the help of a modification of an available LP software) allow one
to construct near optimal controls of the SP system. We demonstrate the
construction with a numerical example.Comment: 36 pages, 4 figures. arXiv admin note: substantial text overlap with
arXiv:1309.373
Deep Relaxation of Controlled Stochastic Gradient Descent via Singular Perturbations
We consider a singularly perturbed system of stochastic differential
equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate
the Entropic Gradient Descent in the optimization of deep neural networks, via
homogenisation. We embed it in a much larger class of two-scale stochastic
control problems and rely on convergence results for Hamilton-Jacobi-Bellman
equations with unbounded data proved recently by ourselves (ESAIM Control
Optim. Calc. Var. 2023). We show that the limit of the value functions is
itself the value function of an effective control problem with extended
controls, and that the trajectories of the perturbed system converge in a
suitable sense to the trajectories of the limiting effective control system.
These rigorous results improve the understanding of the convergence of the
algorithms used by Chaudhari et al., as well as of their possible extensions
where some tuning parameters are modelled as dynamic controls
Averaging and linear programming in some singularly perturbed problems of optimal control
The paper aims at the development of an apparatus for analysis and
construction of near optimal solutions of singularly perturbed (SP) optimal
controls problems (that is, problems of optimal control of SP systems)
considered on the infinite time horizon.
We mostly focus on problems with time discounting criteria but a possibility
of the extension of results to periodic optimization problems is discussed as
well. Our consideration is based on earlier results on averaging of SP control
systems and on linear programming formulations of optimal control problems. The
idea that we exploit is to first asymptotically approximate a given problem of
optimal control of the SP system by a certain averaged optimal control problem,
then reformulate this averaged problem as an infinite-dimensional (ID) linear
programming (LP) problem, and then approximate the latter by semi-infinite LP
problems. We show that the optimal solution of these semi-infinite LP problems
and their duals (that can be found with the help of a modification of an
available LP software) allow one to construct near optimal controls of the SP
system. We demonstrate the construction with two numerical examples.Comment: 53 pages, 10 figure