529 research outputs found
Deep Predictive Policy Training using Reinforcement Learning
Skilled robot task learning is best implemented by predictive action policies
due to the inherent latency of sensorimotor processes. However, training such
predictive policies is challenging as it involves finding a trajectory of motor
activations for the full duration of the action. We propose a data-efficient
deep predictive policy training (DPPT) framework with a deep neural network
policy architecture which maps an image observation to a sequence of motor
activations. The architecture consists of three sub-networks referred to as the
perception, policy and behavior super-layers. The perception and behavior
super-layers force an abstraction of visual and motor data trained with
synthetic and simulated training samples, respectively. The policy super-layer
is a small sub-network with fewer parameters that maps data in-between the
abstracted manifolds. It is trained for each task using methods for policy
search reinforcement learning. We demonstrate the suitability of the proposed
architecture and learning framework by training predictive policies for skilled
object grasping and ball throwing on a PR2 robot. The effectiveness of the
method is illustrated by the fact that these tasks are trained using only about
180 real robot attempts with qualitative terminal rewards.Comment: This work is submitted to IEEE/RSJ International Conference on
Intelligent Robots and Systems 2017 (IROS2017
Neural Dynamic Movement Primitives -- a survey
One of the most important challenges in robotics is producing accurate
trajectories and controlling their dynamic parameters so that the robots can
perform different tasks. The ability to provide such motion control is closely
related to how such movements are encoded. Advances on deep learning have had a
strong repercussion in the development of novel approaches for Dynamic Movement
Primitives. In this work, we survey scientific literature related to Neural
Dynamic Movement Primitives, to complement existing surveys on Dynamic Movement
Primitives
Robot Learning from Human Demonstration: Interpretation, Adaptation, and Interaction
Robot Learning from Demonstration (LfD) is a research area that focuses on how robots can learn new skills by observing how people perform various activities. As humans, we have a remarkable ability to imitate other human’s behaviors and adapt to new situations. Endowing robots with these critical capabilities is a significant but very challenging problem considering the complexity and variation of human activities in highly dynamic environments.
This research focuses on how robots can learn new skills by interpreting human activities, adapting the learned skills to new situations, and naturally interacting with humans. This dissertation begins with a discussion of challenges in each of these three problems. A new unified representation approach is introduced to enable robots to simultaneously interpret the high-level semantic meanings and generalize the low-level trajectories of a broad range of human activities. An adaptive framework based on feature space decomposition is then presented for robots to not only reproduce skills, but also autonomously and efficiently adjust the learned skills to new environments that are significantly different from demonstrations. To achieve natural Human Robot Interaction (HRI), this dissertation presents a Recurrent Neural Network based deep perceptual control approach, which is capable of integrating multi-modal perception sequences with actions for robots to interact with humans in long-term tasks.
Overall, by combining the above approaches, an autonomous system is created for robots to acquire important skills that can be applied to human-centered applications. Finally, this dissertation concludes with a discussion of future directions that could accelerate the upcoming technological revolution of robot learning from human demonstration
Deep Segmented DMP Networks for Learning Discontinuous Motions
Discontinuous motion which is a motion composed of multiple continuous
motions with sudden change in direction or velocity in between, can be seen in
state-aware robotic tasks. Such robotic tasks are often coordinated with sensor
information such as image. In recent years, Dynamic Movement Primitives (DMP)
which is a method for generating motor behaviors suitable for robotics has
garnered several deep learning based improvements to allow associations between
sensor information and DMP parameters. While the implementation of deep
learning framework does improve upon DMP's inability to directly associate to
an input, we found that it has difficulty learning DMP parameters for complex
motion which requires large number of basis functions to reconstruct. In this
paper we propose a novel deep learning network architecture called Deep
Segmented DMP Network (DSDNet) which generates variable-length segmented motion
by utilizing the combination of multiple DMP parameters predicting network
architecture, double-stage decoder network, and number of segments predictor.
The proposed method is evaluated on both artificial data (object cutting &
pick-and-place) and real data (object cutting) where our proposed method could
achieve high generalization capability, task-achievement, and data-efficiency
compared to previous method on generating discontinuous long-horizon motions.Comment: 7 pages, Accepted by the 2023 International Conference on Automation
Science and Engineering (CASE 2023
Imitation and Mirror Systems in Robots through Deep Modality Blending Networks
Learning to interact with the environment not only empowers the agent with
manipulation capability but also generates information to facilitate building
of action understanding and imitation capabilities. This seems to be a strategy
adopted by biological systems, in particular primates, as evidenced by the
existence of mirror neurons that seem to be involved in multi-modal action
understanding. How to benefit from the interaction experience of the robots to
enable understanding actions and goals of other agents is still a challenging
question. In this study, we propose a novel method, deep modality blending
networks (DMBN), that creates a common latent space from multi-modal experience
of a robot by blending multi-modal signals with a stochastic weighting
mechanism. We show for the first time that deep learning, when combined with a
novel modality blending scheme, can facilitate action recognition and produce
structures to sustain anatomical and effect-based imitation capabilities. Our
proposed system, can be conditioned on any desired sensory/motor value at any
time-step, and can generate a complete multi-modal trajectory consistent with
the desired conditioning in parallel avoiding accumulation of prediction
errors. We further showed that given desired images from different
perspectives, i.e. images generated by the observation of other robots placed
on different sides of the table, our system could generate image and joint
angle sequences that correspond to either anatomical or effect based imitation
behavior. Overall, the proposed DMBN architecture not only serves as a
computational model for sustaining mirror neuron-like capabilities, but also
stands as a powerful machine learning architecture for high-dimensional
multi-modal temporal data with robust retrieval capabilities operating with
partial information in one or multiple modalities
ProDMPs: A Unified Perspective on Dynamic and Probabilistic Movement Primitives
Movement Primitives (MPs) are a well-known concept to represent and generate
modular trajectories. MPs can be broadly categorized into two types: (a)
dynamics-based approaches that generate smooth trajectories from any initial
state, e. g., Dynamic Movement Primitives (DMPs), and (b) probabilistic
approaches that capture higher-order statistics of the motion, e. g.,
Probabilistic Movement Primitives (ProMPs). To date, however, there is no
method that unifies both, i. e. that can generate smooth trajectories from an
arbitrary initial state while capturing higher-order statistics. In this paper,
we introduce a unified perspective of both approaches by solving the ODE
underlying the DMPs. We convert expensive online numerical integration of DMPs
into basis functions that can be computed offline. These basis functions can be
used to represent trajectories or trajectory distributions similar to ProMPs
while maintaining all the properties of dynamical systems. Since we inherit the
properties of both methodologies, we call our proposed model Probabilistic
Dynamic Movement Primitives (ProDMPs). Additionally, we embed ProDMPs in deep
neural network architecture and propose a new cost function for efficient
end-to-end learning of higher-order trajectory statistics. To this end, we
leverage Bayesian Aggregation for non-linear iterative conditioning on sensory
inputs. Our proposed model achieves smooth trajectory generation,
goal-attractor convergence, correlation analysis, non-linear conditioning, and
online re-planing in one framework.Comment: 12 pages, 13 figure
- …