915 research outputs found
Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration
We propose a technique for multi-task learning from demonstration that trains
the controller of a low-cost robotic arm to accomplish several complex picking
and placing tasks, as well as non-prehensile manipulation. The controller is a
recurrent neural network using raw images as input and generating robot arm
trajectories, with the parameters shared across the tasks. The controller also
combines VAE-GAN-based reconstruction with autoregressive multimodal action
prediction. Our results demonstrate that it is possible to learn complex
manipulation tasks, such as picking up a towel, wiping an object, and
depositing the towel to its previous position, entirely from raw images with
direct behavior cloning. We show that weight sharing and reconstruction-based
regularization substantially improve generalization and robustness, and
training on multiple tasks simultaneously increases the success rate on all
tasks
Deep Visual Foresight for Planning Robot Motion
A key challenge in scaling up robot learning to many skills and environments
is removing the need for human supervision, so that robots can collect their
own data and improve their own performance without being limited by the cost of
requesting human feedback. Model-based reinforcement learning holds the promise
of enabling an agent to learn to predict the effects of its actions, which
could provide flexible predictive models for a wide range of tasks and
environments, without detailed human supervision. We develop a method for
combining deep action-conditioned video prediction models with model-predictive
control that uses entirely unlabeled training data. Our approach does not
require a calibrated camera, an instrumented training set-up, nor precise
sensing and actuation. Our results show that our method enables a real robot to
perform nonprehensile manipulation -- pushing objects -- and can handle novel
objects not seen during training.Comment: ICRA 2017. Supplementary video:
https://sites.google.com/site/robotforesight
Sensorless Physical Human-robot Interaction Using Deep-Learning
Physical human-robot interaction has been an area of interest for decades.
Collaborative tasks, such as joint compliance, demand high-quality joint torque
sensing. While external torque sensors are reliable, they come with the
drawbacks of being expensive and vulnerable to impacts. To address these
issues, studies have been conducted to estimate external torques using only
internal signals, such as joint states and current measurements. However,
insufficient attention has been given to friction hysteresis approximation,
which is crucial for tasks involving extensive dynamic to static state
transitions. In this paper, we propose a deep-learning-based method that
leverages a novel long-term memory scheme to achieve dynamics identification,
accurately approximating the static hysteresis. We also introduce modifications
to the well-known Residual Learning architecture, retaining high accuracy while
reducing inference time. The robustness of the proposed method is illustrated
through a joint compliance and task compliance experiment.Comment: 7 pages, ICRA 2024 Submissio
Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions
Comprehension of spoken natural language is an essential component for robots
to communicate with human effectively. However, handling unconstrained spoken
instructions is challenging due to (1) complex structures including a wide
variety of expressions used in spoken language and (2) inherent ambiguity in
interpretation of human instructions. In this paper, we propose the first
comprehensive system that can handle unconstrained spoken language and is able
to effectively resolve ambiguity in spoken instructions. Specifically, we
integrate deep-learning-based object detection together with natural language
processing technologies to handle unconstrained spoken instructions, and
propose a method for robots to resolve instruction ambiguity through dialogue.
Through our experiments on both a simulated environment as well as a physical
industrial robot arm, we demonstrate the ability of our system to understand
natural instructions from human operators effectively, and how higher success
rates of the object picking task can be achieved through an interactive
clarification process.Comment: 9 pages. International Conference on Robotics and Automation (ICRA)
2018. Accompanying videos are available at the following links:
https://youtu.be/_Uyv1XIUqhk (the system submitted to ICRA-2018) and
http://youtu.be/DGJazkyw0Ws (with improvements after ICRA-2018 submission
Language Understanding for Text-based Games Using Deep Reinforcement Learning
In this paper, we consider the task of learning control policies for
text-based games. In these games, all interactions in the virtual world are
through text and the underlying state is not observed. The resulting language
barrier makes such environments challenging for automatic game players. We
employ a deep reinforcement learning framework to jointly learn state
representations and action policies using game rewards as feedback. This
framework enables us to map text descriptions into vector representations that
capture the semantics of the game states. We evaluate our approach on two game
worlds, comparing against baselines using bag-of-words and bag-of-bigrams for
state representations. Our algorithm outperforms the baselines on both worlds
demonstrating the importance of learning expressive representations.Comment: 11 pages, Appearing at EMNLP, 201
- …