400 research outputs found
Provable Representation Learning for Imitation Learning via Bi-level Optimization
A common strategy in modern learning systems is to learn a representation
that is useful for many tasks, a.k.a. representation learning. We study this
strategy in the imitation learning setting for Markov decision processes (MDPs)
where multiple experts' trajectories are available. We formulate representation
learning as a bi-level optimization problem where the "outer" optimization
tries to learn the joint representation and the "inner" optimization encodes
the imitation learning setup and tries to learn task-specific parameters. We
instantiate this framework for the imitation learning settings of behavior
cloning and observation-alone. Theoretically, we show using our framework that
representation learning can provide sample complexity benefits for imitation
learning in both settings. We also provide proof-of-concept experiments to
verify our theory.Comment: 26 page
- …