17 research outputs found
A diffusion model of scheduling control in queueing systems with many servers
This paper studies a diffusion model that arises as the limit of a queueing
system scheduling problem in the asymptotic heavy traffic regime of Halfin and
Whitt. The queueing system consists of several customer classes and many
servers working in parallel, grouped in several stations. Servers in different
stations offer service to customers of each class at possibly different rates.
The control corresponds to selecting what customer class each server serves at
each time. The diffusion control problem does not seem to have explicit
solutions and therefore a characterization of optimal solutions via the
Hamilton-Jacobi-Bellman equation is addressed. Our main result is the existence
and uniqueness of solutions of the equation. Since the model is set on an
unbounded domain and the cost per unit time is unbounded, the analysis requires
estimates on the state process that are subexponential in the time variable. In
establishing these estimates, a key role is played by an integral formula that
relates queue length and idle time processes, which may be of independent
interest.Comment: Published at http://dx.doi.org/10.1214/105051604000000963 in the
Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute
of Mathematical Statistics (http://www.imstat.org
Dynamic fluid-based scheduling in a multi-class abandonment queue
International audienceWe investigate how to share a common resource among multiple classes of customers in the presence of abandonments. We consider two different models: (1) customers can abandon both while waiting in the queue and while being served, (2) only customers that are in the queue can abandon. Given the complexity of the stochastic optimization problem we propose a fluid model as a deterministic approximation. For the overload case we directly obtain that the c˜µ/θ rule is optimal. For the underload case we use Pontryagin’s Maximum Principle to obtain the optimal solution for two classes of customers; there exists a switching curve that splits the two-dimensional state-space into two regions such that when the number of customers in both classes is sufficiently small the optimal policy follows the c˜µ-rule and when the number of customers is sufficiently large the optimal policy follows the c˜µ/θ-rule. The same structure is observed in the optimal policy of the stochastic model for an arbitrary number of classes. Based on this we develop a heuristic and by numerical experiments we evaluate its performance and compare it to several index policies. We observe that the suboptimality gap of our solution is small
Controlled diffusion processes
This article gives an overview of the developments in controlled diffusion
processes, emphasizing key results regarding existence of optimal controls and
their characterization via dynamic programming for a variety of cost criteria
and structural assumptions. Stochastic maximum principle and control under
partial observations (equivalently, control of nonlinear filters) are also
discussed. Several other related topics are briefly sketched.Comment: Published at http://dx.doi.org/10.1214/154957805100000131 in the
Probability Surveys (http://www.i-journals.org/ps/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Temporal Concatenation for Markov Decision Processes
We propose and analyze a Temporal Concatenation (TC) heuristic for solving
large-scale finite-horizon Markov decision processes (MDP). The Temporal
Concatenation divides a finite-horizon MDP into smaller sub-problems along the
time horizon, and generates an overall solution by simply concatenating the
optimal solutions from these sub-problems. As a "black box" architecture,
Temporal Concatenation works with a wide range of existing MDP algorithms with
the potential of substantial speed-up at the expense of minor performance
degradation. Our main results characterize the regret of Temporal
Concatenation, defined as the gap between the expected rewards from Temporal
Concatenation's solution and that from the optimal solution. We provide upper
bounds that show, when the underlying MDP satisfies a bounded-diameter
criterion, the regret of Temporal Concatenation is bounded by a constant
independent of the length of the horizon. Conversely, we provide matching lower
bounds that demonstrate that, for any finite diameter, there exist MDP
instances for which the regret upper bound is tight. We further contextualize
the theoretical results in an illustrative example of dynamic energy management
with storage, and provide simulation results to assess Temporal Concatenation's
average-case regret within a family of MDPs related to graph traversal.Comment: Added references and updated the theoretical result in Section