2 research outputs found
Learning Deep Stochastic Optimal Control Policies using Forward-Backward SDEs
In this paper we propose a new methodology for decision-making under
uncertainty using recent advancements in the areas of nonlinear stochastic
optimal control theory, applied mathematics, and machine learning. Grounded on
the fundamental relation between certain nonlinear partial differential
equations and forward-backward stochastic differential equations, we develop a
control framework that is scalable and applicable to general classes of
stochastic systems and decision-making problem formulations in robotics and
autonomy. The proposed deep neural network architectures for stochastic control
consist of recurrent and fully connected layers. The performance and
scalability of the aforementioned algorithm are investigated in three
non-linear systems in simulation with and without control constraints. We
conclude with a discussion on future directions and their implications to
robotics
Deep Forward-Backward SDEs for Min-max Control
This paper presents a novel approach to numerically solve stochastic
differential games for nonlinear systems. The proposed approach relies on the
nonlinear Feynman-Kac theorem that establishes a connection between parabolic
deterministic partial differential equations and forward-backward stochastic
differential equations. Using this theorem the Hamilton-Jacobi-Isaacs partial
differential equation associated with differential games is represented by a
system of forward-backward stochastic differential equations. Numerical
solution of the aforementioned system of stochastic differential equations is
performed using importance sampling and a Long-Short Term Memory recurrent
neural network, which is trained in an offline fashion. The resulting algorithm
is tested on two example systems in simulation and compared against the
standard risk neutral stochastic optimal control formulations