8,425 research outputs found
Dynamic Server Allocation over Time Varying Channels with Switchover Delay
We consider a dynamic server allocation problem over parallel queues with
randomly varying connectivity and server switchover delay between the queues.
At each time slot the server decides either to stay with the current queue or
switch to another queue based on the current connectivity and the queue length
information. Switchover delay occurs in many telecommunications applications
and is a new modeling component of this problem that has not been previously
addressed. We show that the simultaneous presence of randomly varying
connectivity and switchover delay changes the system stability region and the
structure of optimal policies. In the first part of the paper, we consider a
system of two parallel queues, and develop a novel approach to explicitly
characterize the stability region of the system using state-action frequencies
which are stationary solutions to a Markov Decision Process (MDP) formulation.
We then develop a frame-based dynamic control (FBDC) policy, based on the
state-action frequencies, and show that it is throughput-optimal asymptotically
in the frame length. The FBDC policy is applicable to a broad class of network
control systems and provides a new framework for developing throughput-optimal
network control policies using state-action frequencies. Furthermore, we
develop simple Myopic policies that provably achieve more than 90% of the
stability region. In the second part of the paper, we extend our results to
systems with an arbitrary but finite number of queues.Comment: 38 Pages, 18 figures. arXiv admin note: substantial text overlap with
arXiv:1008.234
Scalable Multiagent Coordination with Distributed Online Open Loop Planning
We propose distributed online open loop planning (DOOLP), a general framework
for online multiagent coordination and decision making under uncertainty. DOOLP
is based on online heuristic search in the space defined by a generative model
of the domain dynamics, which is exploited by agents to simulate and evaluate
the consequences of their potential choices.
We also propose distributed online Thompson sampling (DOTS) as an effective
instantiation of the DOOLP framework. DOTS models sequences of agent choices by
concatenating a number of multiarmed bandits for each agent and uses Thompson
sampling for dealing with action value uncertainty. The Bayesian approach
underlying Thompson sampling allows to effectively model and estimate
uncertainty about (a) own action values and (b) other agents' behavior. This
approach yields a principled and statistically sound solution to the
exploration-exploitation dilemma when exploring large search spaces with
limited resources.
We implemented DOTS in a smart factory case study with positive empirical
results. We observed effective, robust and scalable planning and coordination
capabilities even when only searching a fraction of the potential search space
- …