36 research outputs found
Unsupervised state representation learning with robotic priors: a robustness benchmark
Our understanding of the world depends highly on our capacity to produce
intuitive and simplified representations which can be easily used to solve
problems. We reproduce this simplification process using a neural network to
build a low dimensional state representation of the world from images acquired
by a robot. As in Jonschkowski et al. 2015, we learn in an unsupervised way
using prior knowledge about the world as loss functions called robotic priors
and extend this approach to high dimension richer images to learn a 3D
representation of the hand position of a robot from RGB images. We propose a
quantitative evaluation of the learned representation using nearest neighbors
in the state space that allows to assess its quality and show both the
potential and limitations of robotic priors in realistic environments. We
augment image size, add distractors and domain randomization, all crucial
components to achieve transfer learning to real robots. Finally, we also
contribute a new prior to improve the robustness of the representation. The
applications of such low dimensional state representation range from easing
reinforcement learning (RL) and knowledge transfer across tasks, to
facilitating learning from raw data with more efficient and compact high level
representations. The results show that the robotic prior approach is able to
extract high level representation as the 3D position of an arm and organize it
into a compact and coherent space of states in a challenging dataset.Comment: ICRA 2018 submissio
Deep Predictive Policy Training using Reinforcement Learning
Skilled robot task learning is best implemented by predictive action policies
due to the inherent latency of sensorimotor processes. However, training such
predictive policies is challenging as it involves finding a trajectory of motor
activations for the full duration of the action. We propose a data-efficient
deep predictive policy training (DPPT) framework with a deep neural network
policy architecture which maps an image observation to a sequence of motor
activations. The architecture consists of three sub-networks referred to as the
perception, policy and behavior super-layers. The perception and behavior
super-layers force an abstraction of visual and motor data trained with
synthetic and simulated training samples, respectively. The policy super-layer
is a small sub-network with fewer parameters that maps data in-between the
abstracted manifolds. It is trained for each task using methods for policy
search reinforcement learning. We demonstrate the suitability of the proposed
architecture and learning framework by training predictive policies for skilled
object grasping and ball throwing on a PR2 robot. The effectiveness of the
method is illustrated by the fact that these tasks are trained using only about
180 real robot attempts with qualitative terminal rewards.Comment: This work is submitted to IEEE/RSJ International Conference on
Intelligent Robots and Systems 2017 (IROS2017
Deep unsupervised state representation learning with robotic priors: a robustness analysis
International audienceOur understanding of the world depends highly on our capacity to produce intuitive and simplified representations which can be easily used to solve problems. We reproduce this simplification process using a neural network to build a low dimensional state representation of the world from images acquired by a robot. As in Jonschkowski et al. 2015, we learn in an unsupervised way using prior knowledge about the world as loss functions called robotic priors and extend this approach to high dimension richer images to learn a 3D representation of the hand position of a robot from RGB images. We propose a quantitative evaluation metric of the learned representation that uses nearest neighbors in the state space and allows to assess its quality and show both the potential and limitations of robotic priors in realistic environments. We augment image size, add distractors and domain randomization, all crucial components to achieve transfer learning to real robots. Finally, we also contribute a new prior to improve the robustness of the representation. The applications of such low dimensional state representation range from easing reinforcement learning (RL) and knowledge transfer across tasks, to facilitating learning from raw data with more efficient and compact high level representations. The results show that the robotic prior approach is able to extract high level representation as the 3D position of an arm and organize it into a compact and coherent space of states in a challenging dataset