4 research outputs found
Words into Action: Learning Diverse Humanoid Robot Behaviors using Language Guided Iterative Motion Refinement
Humanoid robots are well suited for human habitats due to their morphological
similarity, but developing controllers for them is a challenging task that
involves multiple sub-problems, such as control, planning and perception. In
this paper, we introduce a method to simplify controller design by enabling
users to train and fine-tune robot control policies using natural language
commands. We first learn a neural network policy that generates behaviors given
a natural language command, such as "walk forward", by combining Large Language
Models (LLMs), motion retargeting, and motion imitation. Based on the
synthesized motion, we iteratively fine-tune by updating the text prompt and
querying LLMs to find the best checkpoint associated with the closest motion in
history. We validate our approach using a simulated Digit humanoid robot and
demonstrate learning of diverse motions, such as walking, hopping, and kicking,
without the burden of complex reward engineering. In addition, we show that our
iterative refinement enables us to learn 3x times faster than a naive
formulation that learns from scratch
A Holistic Approach to Human-Supervised Humanoid Robot Operations in Extreme Environments
Nuclear energy will play a critical role in meeting clean energy targets worldwide. However, nuclear environments are dangerous for humans to operate in due to the presence of highly radioactive materials. Robots can help address this issue by allowing remote access to nuclear and other highly hazardous facilities under human supervision to perform inspection and maintenance tasks during normal operations, help with clean-up missions, and aid in decommissioning. This paper presents our research to help realize humanoid robots in supervisory roles in nuclear environments. Our research focuses on National Aeronautics and Space Administration (NASA’s) humanoid robot, Valkyrie, in the areas of constrained manipulation and motion planning, increasing stability using support contact, dynamic non-prehensile manipulation, locomotion on deformable terrains, and human-in-the-loop control interfaces