134,640 research outputs found
A Mean Field Approach for Optimization in Particle Systems and Applications
This paper investigates the limit behavior of Markov decision processes made of independent particles evolving in a common environment, when the number of particles goes to infinity. In the finite horizon case or with a discounted cost and an infinite horizon, we show that when the number of particles becomes large, the optimal cost of the system converges to the optimal cost of a deterministic system. Convergence also holds for optimal policies. We further provide insights on the speed of convergence by proving several central limits theorems for the cost and the state of the Markov decision process with explicit formulas for the limit. Then, our framework is applied to a brokering problem in grid computing. Several simulations with growing numbers of processors are reported. They compare the performance of the optimal policy of the limit system used in the finite case with classical policies by measuring its asymptotic gain
A Mean Field Approach for Optimization in Particles Systems and Applications
This paper investigates the limit behavior of Markov Decision Processes
(MDPs) made of independent particles evolving in a common environment, when the
number of particles goes to infinity. In the finite horizon case or with a
discounted cost and an infinite horizon, we show that when the number of
particles becomes large, the optimal cost of the system converges almost surely
to the optimal cost of a discrete deterministic system (the ``optimal mean
field''). Convergence also holds for optimal policies. We further provide
insights on the speed of convergence by proving several central limits theorems
for the cost and the state of the Markov decision process with explicit
formulas for the variance of the limit Gaussian laws. Then, our framework is
applied to a brokering problem in grid computing. The optimal policy for the
limit deterministic system is computed explicitly. Several simulations with
growing numbers of processors are reported. They compare the performance of the
optimal policy of the limit system used in the finite case with classical
policies (such as Join the Shortest Queue) by measuring its asymptotic gain as
well as the threshold above which it starts outperforming classical policies
Mean-field optimal control and optimality conditions in the space of probability measures
We derive a framework to compute optimal controls for problems with states in
the space of probability measures. Since many optimal control problems
constrained by a system of ordinary differential equations (ODE) modelling
interacting particles converge to optimal control problems constrained by a
partial differential equation (PDE) in the mean-field limit, it is interesting
to have a calculus directly on the mesoscopic level of probability measures
which allows us to derive the corresponding first-order optimality system. In
addition to this new calculus, we provide relations for the resulting system to
the first-order optimality system derived on the particle level, and the
first-order optimality system based on -calculus under additional
regularity assumptions. We further justify the use of the -adjoint in
numerical simulations by establishing a link between the adjoint in the space
of probability measures and the adjoint corresponding to -calculus.
Moreover, we prove a convergence rate for the convergence of the optimal
controls corresponding to the particle formulation to the optimal controls of
the mean-field problem as the number of particles tends to infinity
Instantaneous control of interacting particle systems in the mean-field limit
Controlling large particle systems in collective dynamics by a few agents is
a subject of high practical importance, e.g., in evacuation dynamics. In this
paper we study an instantaneous control approach to steer an interacting
particle system into a certain spatial region by repulsive forces from a few
external agents, which might be interpreted as shepherd dogs leading sheep to
their home. We introduce an appropriate mathematical model and the
corresponding optimization problem. In particular, we are interested in the
interaction of numerous particles, which can be approximated by a mean-field
equation. Due to the high-dimensional phase space this will require a tailored
optimization strategy. The arising control problems are solved using adjoint
information to compute the descent directions. Numerical results on the
microscopic and the macroscopic level indicate the convergence of optimal
controls and optimal states in the mean-field limit,i.e., for an increasing
number of particles.Comment: arXiv admin note: substantial text overlap with arXiv:1610.0132
- …