3 research outputs found
Price decomposition in large-scale stochastic optimal control
We are interested in optimally driving a dynamical system that can be
influenced by exogenous noises. This is generally called a Stochastic Optimal
Control (SOC) problem and the Dynamic Programming (DP) principle is the natural
way of solving it. Unfortunately, DP faces the so-called curse of
dimensionality: the complexity of solving DP equations grows exponentially with
the dimension of the information variable that is sufficient to take optimal
decisions (the state variable). For a large class of SOC problems, which
includes important practical problems, we propose an original way of obtaining
strategies to drive the system. The algorithm we introduce is based on
Lagrangian relaxation, of which the application to decomposition is well-known
in the deterministic framework. However, its application to such closed-loop
problems is not straightforward and an additional statistical approximation
concerning the dual process is needed. We give a convergence proof, that
derives directly from classical results concerning duality in optimization, and
enlghten the error made by our approximation. Numerical results are also
provided, on a large-scale SOC problem. This idea extends the original DADP
algorithm that was presented by Barty, Carpentier and Girardeau (2010)
Decomposition of large-scale stochastic optimal control problems
In this paper, we present an Uzawa-based heuristic that is adapted to some
type of stochastic optimal control problems. More precisely, we consider
dynamical systems that can be divided into small-scale independent subsystems,
though linked through a static almost sure coupling constraint at each time
step. This type of problem is common in production/portfolio management where
subsystems are, for instance, power units, and one has to supply a stochastic
power demand at each time step. We outline the framework of our approach and
present promising numerical results on a simplified power management problem
A Stochastic Gradient Type Algorithm for Closed Loop Problems
We focus on solving closed-loop stochastic problems, and propose a perturbed gradient algorithm to achieve this goal. The main hurdle in such problems is the fact that the control variables are infinite dimensional, and have hence to be represented in a finite way in order to numerically solve the problem. In the same way, the gradient of the criterion is itself an infinite dimensional object. Our algorithm replaces this exact (and unknown) gradient by a perturbed one, which consists in the product of the true gradient evaluated at a random point and a kernel function which extends this gradient to the neighbourhood of the random point. Proceeding this way, we explore the whole space iteration after iteration through random points. Since each kernel function is perfectly known by a finite (and small) number of parameters, say N, the control at iteration k is perfectly known as an infinite dimensional object by at most N x k parameters.The main strength of this method is that it avoids any discretization of the underlying space, provided that we can draw as many points as needed in this space. Hence, we can take into account in a new way the possible measurability constraints of the problem.Moreover, the randomization of this algorithm implies that the most probable parts of the space are the most explored ones, what is a priori an interesting feature.In this paper, we first show a convergence result of this algorithm in the general case, and then give a few numerical examples showing the interest of this method for solving practical stochastic optimization problems