6,062 research outputs found
Cooperative Games with Overlapping Coalitions
In the usual models of cooperative game theory, the outcome of a coalition
formation process is either the grand coalition or a coalition structure that
consists of disjoint coalitions. However, in many domains where coalitions are
associated with tasks, an agent may be involved in executing more than one
task, and thus may distribute his resources among several coalitions. To tackle
such scenarios, we introduce a model for cooperative games with overlapping
coalitions--or overlapping coalition formation (OCF) games. We then explore the
issue of stability in this setting. In particular, we introduce a notion of the
core, which generalizes the corresponding notion in the traditional
(non-overlapping) scenario. Then, under some quite general conditions, we
characterize the elements of the core, and show that any element of the core
maximizes the social welfare. We also introduce a concept of balancedness for
overlapping coalitional games, and use it to characterize coalition structures
that can be extended to elements of the core. Finally, we generalize the notion
of convexity to our setting, and show that under some natural assumptions
convex games have a non-empty core. Moreover, we introduce two alternative
notions of stability in OCF that allow a wider range of deviations, and explore
the relationships among the corresponding definitions of the core, as well as
the classic (non-overlapping) core and the Aubin core. We illustrate the
general properties of the three cores, and also study them from a computational
perspective, thus obtaining additional insights into their fundamental
structure
Scaling reinforcement learning to the unconstrained multi-agent domain
Reinforcement learning is a machine learning technique designed to mimic the
way animals learn by receiving rewards and punishment. It is designed to train
intelligent agents when very little is known about the agentās environment, and consequently
the agentās designer is unable to hand-craft an appropriate policy. Using
reinforcement learning, the agentās designer can merely give reward to the agent when
it does something right, and the algorithm will craft an appropriate policy automatically.
In many situations it is desirable to use this technique to train systems of agents
(for example, to train robots to play RoboCup soccer in a coordinated fashion). Unfortunately,
several significant computational issues occur when using this technique
to train systems of agents. This dissertation introduces a suite of techniques that
overcome many of these difficulties in various common situations.
First, we show how multi-agent reinforcement learning can be made more tractable
by forming coalitions out of the agents, and training each coalition separately. Coalitions
are formed by using information-theoretic techniques, and we find that by using
a coalition-based approach, the computational complexity of reinforcement-learning
can be made linear in the total system agent count. Next we look at ways to integrate
domain knowledge into the reinforcement learning process, and how this can signifi-cantly improve the policy quality in multi-agent situations. Specifically, we find that
integrating domain knowledge into a reinforcement learning process can overcome training data deficiencies and allow the learner to converge to acceptable solutions
when lack of training data would have prevented such convergence without domain
knowledge. We then show how to train policies over continuous action spaces, which
can reduce problem complexity for domains that require continuous action spaces
(analog controllers) by eliminating the need to finely discretize the action space. Finally,
we look at ways to perform reinforcement learning on modern GPUs and show
how by doing this we can tackle significantly larger problems. We find that by offloading
some of the RL computation to the GPU, we can achieve almost a 4.5 speedup
factor in the total training process
- ā¦