36 research outputs found
Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies
Various approaches have been proposed to learn visuo-motor policies for
real-world robotic applications. One solution is first learning in simulation
then transferring to the real world. In the transfer, most existing approaches
need real-world images with labels. However, the labelling process is often
expensive or even impractical in many robotic applications. In this paper, we
propose an adversarial discriminative sim-to-real transfer approach to reduce
the cost of labelling real data. The effectiveness of the approach is
demonstrated with modular networks in a table-top object reaching task where a
7 DoF arm is controlled in velocity mode to reach a blue cuboid in clutter
through visual observations. The adversarial transfer approach reduced the
labelled real data requirement by 50%. Policies can be transferred to real
environments with only 93 labelled and 186 unlabelled real images. The
transferred visuo-motor policies are robust to novel (not seen in training)
objects in clutter and even a moving target, achieving a 97.8% success rate and
1.8 cm control accuracy.Comment: Under review for the International Journal of Robotics Researc
Transferring visuomotor learning from simulation to the real world for robotics manipulation tasks
Hand-eye coordination is a requirement for many manipulation tasks including grasping and reaching. However, accurate hand-eye coordination has shown to be especially difficult to achieve in complex robots like the iCub humanoid. In this work, we solve the hand-eye coordination task using a visuomotor deep neural network predictor that estimates the arm's joint configuration given a stereo image pair of the arm and the underlying head configuration. As there are various unavoidable sources of sensing error on the physical robot, we train the predictor on images obtained from simulation. The images from simulation were modified to look realistic using an image-to-image translation approach. In various experiments, we first show that the visuomotor predictor provides accurate joint estimates of the iCub's hand in simulation. We then show that the predictor can be used to obtain the systematic error of the robot's joint measurements on the physical iCub robot. We demonstrate that a calibrator can be designed to automatically compensate this error. Finally, we validate that this enables accurate reaching of objects while circumventing manual fine-calibration of the robot
Sim2Real Grasp Pose Estimation for Adaptive Robotic Applications
Adaptive robotics plays an essential role in achieving truly co-creative
cyber physical systems. In robotic manipulation tasks, one of the biggest
challenges is to estimate the pose of given workpieces. Even though the recent
deep-learning-based models show promising results, they require an immense
dataset for training. In this paper, we propose two vision-based, multiobject
grasp-pose estimation models, the MOGPE Real-Time (RT) and the MOGPE
High-Precision (HP). Furthermore, a sim2real method based on domain
randomization to diminish the reality gap and overcome the data shortage. We
yielded an 80% and a 96.67% success rate in a real-world robotic pick-and-place
experiment, with the MOGPE RT and the MOGPE HP model respectively. Our
framework provides an industrial tool for fast data generation and model
training and requires minimal domain-specific data.Comment: Submitted to the 22nd World Congress of the International Federation
of Automatic Control (IFAC 2023
Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models
We present a data-efficient framework for solving sequential decision-making
problems which exploits the combination of reinforcement learning (RL) and
latent variable generative models. The framework, called GenRL, trains deep
policies by introducing an action latent variable such that the feed-forward
policy search can be divided into two parts: (i) training a sub-policy that
outputs a distribution over the action latent variable given a state of the
system, and (ii) unsupervised training of a generative model that outputs a
sequence of motor actions conditioned on the latent action variable. GenRL
enables safe exploration and alleviates the data-inefficiency problem as it
exploits prior knowledge about valid sequences of motor actions. Moreover, we
provide a set of measures for evaluation of generative models such that we are
able to predict the performance of the RL policy training prior to the actual
training on a physical robot. We experimentally determine the characteristics
of generative models that have most influence on the performance of the final
policy training on two robotics tasks: shooting a hockey puck and throwing a
basketball. Furthermore, we empirically demonstrate that GenRL is the only
method which can safely and efficiently solve the robotics tasks compared to
two state-of-the-art RL methods.Comment: arXiv admin note: substantial text overlap with arXiv:2007.1313