9 research outputs found
Learning Riemannian Stable Dynamical Systems via Diffeomorphisms
Dexterous and autonomous robots should be capable of executing elaborated
dynamical motions skillfully. Learning techniques may be leveraged to build
models of such dynamic skills. To accomplish this, the learning model needs to
encode a stable vector field that resembles the desired motion dynamics. This
is challenging as the robot state does not evolve on a Euclidean space, and
therefore the stability guarantees and vector field encoding need to account
for the geometry arising from, for example, the orientation representation. To
tackle this problem, we propose learning Riemannian stable dynamical systems
(RSDS) from demonstrations, allowing us to account for different geometric
constraints resulting from the dynamical system state representation. Our
approach provides Lyapunov-stability guarantees on Riemannian manifolds that
are enforced on the desired motion dynamics via diffeomorphisms built on neural
manifold ODEs. We show that our Riemannian approach makes it possible to learn
stable dynamical systems displaying complicated vector fields on both
illustrative examples and real-world manipulation tasks, where Euclidean
approximations fail.Comment: To appear at CoRL 202
Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning
Telerobotic systems must adapt to new environmental conditions and deal with
high uncertainty caused by long-time delays. As one of the best alternatives to
human-level intelligence, Reinforcement Learning (RL) may offer a solution to
cope with these issues. This paper proposes to integrate RL with the Model
Mediated Teleoperation (MMT) concept. The teleoperator interacts with a
simulated virtual environment, which provides instant feedback. Whereas
feedback from the real environment is delayed, feedback from the model is
instantaneous, leading to high transparency. The MMT is realized in combination
with an intelligent system with two layers. The first layer utilizes Dynamic
Movement Primitives (DMP) which accounts for certain changes in the avatar
environment. And, the second layer addresses the problems caused by uncertainty
in the model using RL methods. Augmented reality was also provided to fuse the
avatar device and virtual environment models for the teleoperator. Implemented
on DLR's Exodex Adam hand-arm haptic exoskeleton, the results show RL methods
are able to find different solutions when changes are applied to the object
position after the demonstration. The results also show DMPs to be effective at
adapting to new conditions where there is no uncertainty involved
Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning
elerobotic systems must adapt to new environmental conditions and deal with high uncertainty caused by long-time delays. As one of the best alternatives to human-level intelligence, Reinforcement Learning (RL) may offer a solution to cope with these issues. This paper proposes to integrate RL with the Model Mediated Teleoperation (MMT) concept. The teleoperator interacts with a simulated virtual environment, which provides instant feedback. Whereas feedback from the real environment is delayed, feedback from the model is instantaneous, leading to high transparency. The MMT is realized in combination with an intelligent system with two layers. The first layer utilizes Dynamic Movement Primitives (DMP) which accounts for certain changes in the avatar environment. And, the second layer addresses the problems caused by uncertainty in the model using RL methods. Augmented reality was also provided to fuse the avatar device and virtual environment models for the teleoperator. Implemented on DLR's Exodex Adam hand-arm haptic exoskeleton, the results show RL methods are able to find different solutions when changes are applied to the object position after the demonstration. The results also show DMPs to be effective at adapting to new conditions where there is no uncertainty involved
Exodex Adam—A Reconfigurable Dexterous Haptic User Interface for the Whole Hand
Applications for dexterous robot teleoperation and immersive virtual reality are growing. Haptic user input devices need to allow the user to intuitively command and seamlessly “feel” the environment they work in, whether virtual or a remote site through an avatar. We introduce the DLR Exodex Adam, a reconfigurable, dexterous, whole-hand haptic input device. The device comprises multiple modular, three degrees of freedom (3-DOF) robotic fingers, whose placement on the device can be adjusted to optimize manipulability for different user hand sizes. Additionally, the device is mounted on a 7-DOF robot arm to increase the user’s workspace. Exodex Adam uses a front-facing interface, with robotic fingers coupled to two of the user’s fingertips, the thumb, and two points on the palm. Including the palm, as opposed to only the fingertips as is common in existing devices, enables accurate tracking of the whole hand without additional sensors such as a data glove or motion capture. By providing “whole-hand” interaction with omnidirectional force-feedback at the attachment points, we enable the user to experience the environment with the complete hand instead of only the fingertips, thus realizing deeper immersion. Interaction using Exodex Adam can range from palpation of objects and surfaces to manipulation using both power and precision grasps, all while receiving haptic feedback. This article details the concept and design of the Exodex Adam, as well as use cases where it is deployed with different command modalities. These include mixed-media interaction in a virtual environment, gesture-based telemanipulation, and robotic hand–arm teleoperation using adaptive model-mediated teleoperation. Finally, we share the insights gained during our development process and use case deployments
Adaptive Model Mediated Control Using Reinforcement Learning
Due to similarities in learning techniques, Reinforcement Learning (RL) is the closest alternative to human-level intelligence. Teleoperation systems using RL can adapt to new environmental conditions and deal with high uncertainty due to long-time delays. In this thesis, we propose a method that takes advantage of RL capabilities to extend the human reach in dangerous remote environments. The proposed method utilizes the Model Mediated Teleoperation (MMT) concept in which the teleoperator interacts with a simulated setup that resembles the real environment. The simulation can provide instant haptic feedback where the data from the real environment are delayed. The proposed approach enables haptic feedback teleoperation of high-DOF dexterous robots under long time delays in a time-varying environment with high uncertainty.
In existence of time delay, when the data is received by the remote system the environment may change drastically, therefore, the attempt for task execution will fail. To prevent failure, an intelligence system is realized in two layers, the first layer utilizes the Dynamic Movement Primitives (DMP) which accounts for certain changes in the environment. DMPs can adjust the shape of a trajectory based on given criteria, for example, a new target position or avoiding a new obstacle. But in an uncertain environment, DMPs fail, therefore, the second layer of intelligence makes use of different reinforcement learning methods based on expectation-maximization, stochastic optimal control and policy gradient to guarantee the successful completion of the task.
Furthermore, To ensure the safety of the system, and speed up the learning process, each learning session for RL happens in multiple simulations of the remote system and environment, simultaneously.
The proposed approach was realized on DLR's haptic hand-arm user interface/exoskeleton, Exodex Adam. It has been used for the first time in this work as the master device to teleoperate a high-DOF dexterous robot. This slave device is an anthropomorphic hand-arm system combining a five-finger hand (FFH) attached to a custom configured DLR lightweight robot (LWR 4+) more closely fitting to the kinematics of the human arm. An augmented reality visualization implemented on the Microsoft Hololens fuses the slave device and virtual environment models to provide environment immersion for the teleoperator.
A preliminary user-study was carried out to help evaluate the human-robot interaction capabilities and performance of the system. Meanwhile, the RL approaches are evaluated separately in two different levels of difficulty; with and without uncertainty in perceived object position.
The results from the unweighted NASA Task load Index (NASA TLX) and System Usability Score (SUS) questionnaires show a low workload (27) and above-average perceived usability (71). The learning results show all RL methods can find a solution for all challenges in a limited time. Meanwhile, the method based on stochastic optimal control has a better performance. The results also show DMPs to be effective at adapting to new conditions where there is no uncertainty involved
Learning Riemannian Manifolds for Geodesic Motion Skills
For robots to work alongside humans and perform in unstructured environments,
they must learn new motion skills and adapt them to unseen situations on the
fly. This demands learning models that capture relevant motion patterns, while
offering enough flexibility to adapt the encoded skills to new requirements,
such as dynamic obstacle avoidance. We introduce a Riemannian manifold
perspective on this problem, and propose to learn a Riemannian manifold from
human demonstrations on which geodesics are natural motion skills. We realize
this with a variational autoencoder (VAE) over the space of position and
orientations of the robot end-effector. Geodesic motion skills let a robot plan
movements from and to arbitrary points on the data manifold. They also provide
a straightforward method to avoid obstacles by redefining the ambient metric in
an online fashion. Moreover, geodesics naturally exploit the manifold resulting
from multiple--mode tasks to design motions that were not explicitly
demonstrated previously. We test our learning framework using a 7-DoF robotic
manipulator, where the robot satisfactorily learns and reproduces realistic
skills featuring elaborated motion patterns, avoids previously unseen
obstacles, and generates novel movements in multiple-mode settings
Reactive Motion Generation on Learned Riemannian Manifolds
In recent decades, advancements in motion learning have enabled robots to
acquire new skills and adapt to unseen conditions in both structured and
unstructured environments. In practice, motion learning methods capture
relevant patterns and adjust them to new conditions such as dynamic obstacle
avoidance or variable targets. In this paper, we investigate the robot motion
learning paradigm from a Riemannian manifold perspective. We argue that
Riemannian manifolds may be learned via human demonstrations in which geodesics
are natural motion skills. The geodesics are generated using a learned
Riemannian metric produced by our novel variational autoencoder (VAE), which is
especially intended to recover full-pose end-effector states and joint space
configurations. In addition, we propose a technique for facilitating on-the-fly
end-effector/multiple-limb obstacle avoidance by reshaping the learned manifold
using an obstacle-aware ambient metric. The motion generated using these
geodesics may naturally result in multiple-solution tasks that have not been
explicitly demonstrated previously. We extensively tested our approach in task
space and joint space scenarios using a 7-DoF robotic manipulator. We
demonstrate that our method is capable of learning and generating motion skills
based on complicated motion patterns demonstrated by a human operator.
Additionally, we assess several obstacle avoidance strategies and generate
trajectories in multiple-mode settings