1,020 research outputs found
Influence-Optimistic Local Values for Multiagent Planning
Over the last decade, methods for multiagent planning under uncertainty have increased in scalability. However, many methods assume value factorization or are not able to provide quality guarantees. We propose a novel family of influence-optimistic upper bounds on the optimal value for problems with lOOs of agents that do not exhibit value fec-torization
Influence-Optimistic Local Values for Multiagent Planning --- Extended Version
Recent years have seen the development of methods for multiagent planning
under uncertainty that scale to tens or even hundreds of agents. However, most
of these methods either make restrictive assumptions on the problem domain, or
provide approximate solutions without any guarantees on quality. Methods in the
former category typically build on heuristic search using upper bounds on the
value function. Unfortunately, no techniques exist to compute such upper bounds
for problems with non-factored value functions. To allow for meaningful
benchmarking through measurable quality guarantees on a very general class of
problems, this paper introduces a family of influence-optimistic upper bounds
for factored decentralized partially observable Markov decision processes
(Dec-POMDPs) that do not have factored value functions. Intuitively, we derive
bounds on very large multiagent planning problems by subdividing them in
sub-problems, and at each of these sub-problems making optimistic assumptions
with respect to the influence that will be exerted by the rest of the system.
We numerically compare the different upper bounds and demonstrate how we can
achieve a non-trivial guarantee that a heuristic solution for problems with
hundreds of agents is close to optimal. Furthermore, we provide evidence that
the upper bounds may improve the effectiveness of heuristic influence search,
and discuss further potential applications to multiagent planning.Comment: Long version of IJCAI 2015 paper (and extended abstract at AAMAS
2015
Scalable Multiagent Coordination with Distributed Online Open Loop Planning
We propose distributed online open loop planning (DOOLP), a general framework
for online multiagent coordination and decision making under uncertainty. DOOLP
is based on online heuristic search in the space defined by a generative model
of the domain dynamics, which is exploited by agents to simulate and evaluate
the consequences of their potential choices.
We also propose distributed online Thompson sampling (DOTS) as an effective
instantiation of the DOOLP framework. DOTS models sequences of agent choices by
concatenating a number of multiarmed bandits for each agent and uses Thompson
sampling for dealing with action value uncertainty. The Bayesian approach
underlying Thompson sampling allows to effectively model and estimate
uncertainty about (a) own action values and (b) other agents' behavior. This
approach yields a principled and statistically sound solution to the
exploration-exploitation dilemma when exploring large search spaces with
limited resources.
We implemented DOTS in a smart factory case study with positive empirical
results. We observed effective, robust and scalable planning and coordination
capabilities even when only searching a fraction of the potential search space
Q-CP: Learning Action Values for Cooperative Planning
Research on multi-robot systems has demonstrated promising results in manifold applications and domains. Still, efficiently learning an effective robot behaviors is very difficult, due to unstructured scenarios, high uncertainties, and large state dimensionality (e.g. hyper-redundant and groups of robot). To alleviate this problem, we present Q-CP a cooperative model-based reinforcement learning algorithm, which exploits action values to both (1) guide the exploration of the state space and (2) generate effective policies. Specifically, we exploit Q-learning to attack the curse-of-dimensionality in the iterations of a Monte-Carlo Tree Search. We implement and evaluate Q-CP on different stochastic cooperative (general-sum) games: (1) a simple cooperative navigation problem among 3 robots, (2) a cooperation scenario between a pair of KUKA YouBots performing hand-overs, and (3) a coordination task between two mobile robots entering a door. The obtained results show the effectiveness of Q-CP in the chosen applications, where action values drive the exploration and reduce the computational demand of the planning process while achieving good performance
- …