Diffusion models have demonstrated highly-expressive generative capabilities
in vision and NLP. Recent studies in reinforcement learning (RL) have shown
that diffusion models are also powerful in modeling complex policies or
trajectories in offline datasets. However, these works have been limited to
single-task settings where a generalist agent capable of addressing multi-task
predicaments is absent. In this paper, we aim to investigate the effectiveness
of a single diffusion model in modeling large-scale multi-task offline data,
which can be challenging due to diverse and multimodal data distribution.
Specifically, we propose Multi-Task Diffusion Model (\textsc{MTDiff}), a
diffusion-based method that incorporates Transformer backbones and prompt
learning for generative planning and data synthesis in multi-task offline
settings. \textsc{MTDiff} leverages vast amounts of knowledge available in
multi-task data and performs implicit knowledge sharing among tasks. For
generative planning, we find \textsc{MTDiff} outperforms state-of-the-art
algorithms across 50 tasks on Meta-World and 8 maps on Maze2D. For data
synthesis, \textsc{MTDiff} generates high-quality data for testing tasks given
a single demonstration as a prompt, which enhances the low-quality datasets for
even unseen tasks.Comment: 21 page