1,018 research outputs found
Average optimality for continuous-time Markov decision processes under weak continuity conditions
This article considers the average optimality for a continuous-time Markov
decision process with Borel state and action spaces and an arbitrarily
unbounded nonnegative cost rate. The existence of a deterministic stationary
optimal policy is proved under a different and general set of conditions as
compared to the previous literature; the controlled process can be explosive,
the transition rates can be arbitrarily unbounded and are weakly continuous,
the multifunction defining the admissible action spaces can be neither
compact-valued nor upper semi-continuous, and the cost rate is not necessarily
inf-compact
Dynamic Service Rate Control for a Single Server Queue with Markov Modulated Arrivals
We consider the problem of service rate control of a single server queueing
system with a finite-state Markov-modulated Poisson arrival process. We show
that the optimal service rate is non-decreasing in the number of customers in
the system; higher congestion rates warrant higher service rates. On the
contrary, however, we show that the optimal service rate is not necessarily
monotone in the current arrival rate. If the modulating process satisfies a
stochastic monotonicity property the monotonicity is recovered. We examine
several heuristics and show where heuristics are reasonable substitutes for the
optimal control. None of the heuristics perform well in all the regimes.
Secondly, we discuss when the Markov-modulated Poisson process with service
rate control can act as a heuristic itself to approximate the control of a
system with a periodic non-homogeneous Poisson arrival process. Not only is the
current model of interest in the control of Internet or mobile networks with
bursty traffic, but it is also useful in providing a tractable alternative for
the control of service centers with non-stationary arrival rates.Comment: 32 Pages, 7 Figure
Existence of Markov equilibrium control in discrete time
For time-inconsistent stochastic controls in discrete time and finite
horizon, an open problem in Bj\"ork and Murgoci (Finance Stoch, 2014) is the
existence of an equilibrium control. A nonrandomized Borel measurable Markov
equilibrium policy exists if the objective is inf-compact in every time step.
We provide a sufficient condition for the inf-compactness and thus existence,
with costs that are lower semicontinuous (l.s.c.) and bounded from below and
transition kernels that are continuous in controls under given states. The
control spaces need not to be compact
- …