3,380 research outputs found
Learn what matters: cross-domain imitation learning with task-relevant embeddings
We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. Such cross-domain imitation learning is required to, for example, train an artificial agent from demonstrations of a human expert. We propose a scalable framework that enables cross-domain imitation learning without access to additional demonstrations or further domain knowledge. We jointly train the learner agent's policy and learn a mapping between the learner and expert domains with adversarial training. We effect this by using a mutual information criterion to find an embedding of the expert's state space that contains task-relevant information and is invariant to domain specifics. This step significantly simplifies estimating the mapping between the learner and expert domains and hence facilitates end-to-end learning. We demonstrate successful transfer of policies between considerably different domains, without extra supervision such as additional demonstrations, and in situations where other methods fail
Inverse Factorized Q-Learning for Cooperative Multi-agent Imitation Learning
This paper concerns imitation learning (IL) (i.e, the problem of learning to
mimic expert behaviors from demonstrations) in cooperative multi-agent systems.
The learning problem under consideration poses several challenges,
characterized by high-dimensional state and action spaces and intricate
inter-agent dependencies. In a single-agent setting, IL has proven to be done
efficiently through an inverse soft-Q learning process given expert
demonstrations. However, extending this framework to a multi-agent context
introduces the need to simultaneously learn both local value functions to
capture local observations and individual actions, and a joint value function
for exploiting centralized learning. In this work, we introduce a novel
multi-agent IL algorithm designed to address these challenges. Our approach
enables the centralized learning by leveraging mixing networks to aggregate
decentralized Q functions. A main advantage of this approach is that the
weights of the mixing networks can be trained using information derived from
global states. We further establish conditions for the mixing networks under
which the multi-agent objective function exhibits convexity within the Q
function space. We present extensive experiments conducted on some challenging
competitive and cooperative multi-agent game environments, including an
advanced version of the Star-Craft multi-agent challenge (i.e., SMACv2), which
demonstrates the effectiveness of our proposed algorithm compared to existing
state-of-the-art multi-agent IL algorithms
- …