7,010 research outputs found
LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulation
To assist with everyday human activities, robots must solve complex
long-horizon tasks and generalize to new settings. Recent deep reinforcement
learning (RL) methods show promise in fully autonomous learning, but they
struggle to reach long-term goals in large environments. On the other hand,
Task and Motion Planning (TAMP) approaches excel at solving and generalizing
across long-horizon tasks, thanks to their powerful state and action
abstractions. But they assume predefined skill sets, which limits their
real-world applications. In this work, we combine the benefits of these two
paradigms and propose an integrated task planning and skill learning framework
named LEAGUE (Learning and Abstraction with Guidance). LEAGUE leverages the
symbolic interface of a task planner to guide RL-based skill learning and
creates abstract state space to enable skill reuse. More importantly, LEAGUE
learns manipulation skills in-situ of the task planning system, continuously
growing its capability and the set of tasks that it can solve. We evaluate
LEAGUE on four challenging simulated task domains and show that LEAGUE
outperforms baselines by large margins. We also show that the learned skills
can be reused to accelerate learning in new tasks domains and transfer to a
physical robot platform.Comment: Accepted to RA-L 202
Task and Motion Planning with Large Language Models for Object Rearrangement
Multi-object rearrangement is a crucial skill for service robots, and
commonsense reasoning is frequently needed in this process. However, achieving
commonsense arrangements requires knowledge about objects, which is hard to
transfer to robots. Large language models (LLMs) are one potential source of
this knowledge, but they do not naively capture information about plausible
physical arrangements of the world. We propose LLM-GROP, which uses prompting
to extract commonsense knowledge about semantically valid object configurations
from an LLM and instantiates them with a task and motion planner in order to
generalize to varying scene geometry. LLM-GROP allows us to go from
natural-language commands to human-aligned object rearrangement in varied
environments. Based on human evaluations, our approach achieves the highest
rating while outperforming competitive baselines in terms of success rate while
maintaining comparable cumulative action costs. Finally, we demonstrate a
practical implementation of LLM-GROP on a mobile manipulator in real-world
scenarios. Supplementary materials are available at:
https://sites.google.com/view/llm-gro
- …