Verifiable Learned Behaviors via Motion Primitive Composition: Applications to Scooping of Granular Media

A robotic behavior model that can reliably generate behaviors from natural language inputs in real time would substantially expedite the adoption of industrial robots due to enhanced system flexibility. To facilitate these efforts, we construct a framework in which learned behaviors, created by a natural language abstractor, are verifiable by construction. Leveraging recent advancements in motion primitives and probabilistic verification, we construct a natural-language behavior abstractor that generates behaviors by synthesizing a directed graph over the provided motion primitives. If these component motion primitives are constructed according to the criteria we specify, the resulting behaviors are probabilistically verifiable. We demonstrate this verifiable behavior generation capacity in both simulation on an exploration task and on hardware with a robot scooping granular media

Challenges and Solutions for Autonomous Robotic Mobile Manipulation for Outdoor Sample Collection

In refinery, petrochemical, and chemical plants, process technicians collect uncontaminated samples to be analyzed in the quality control laboratory all time and all weather. This traditionally manual operation not only exposes the process technicians to hazardous chemicals, but also imposes an economical burden on the management. The recent development in mobile manipulation provides an opportunity to fully automate the operation of sample collection. This paper reviewed the various challenges in sample collection in terms of navigation of the mobile platform and manipulation of the robotic arm from four aspects, namely mobile robot positioning/attitude using global navigation satellite system (GNSS), vision-based navigation and visual servoing, robotic manipulation, mobile robot path planning and control. This paper further proposed solutions to these challenges and pointed the main direction of development in mobile manipulation

Data-Driven Grasp Synthesis - A Survey

We review the work on data-driven grasp synthesis and the methodologies for sampling and ranking candidate grasps. We divide the approaches into three groups based on whether they synthesize grasps for known, familiar or unknown objects. This structure allows us to identify common object representations and perceptual processes that facilitate the employed data-driven grasp synthesis technique. In the case of known objects, we concentrate on the approaches that are based on object recognition and pose estimation. In the case of familiar objects, the techniques use some form of a similarity matching to a set of previously encountered objects. Finally for the approaches dealing with unknown objects, the core part is the extraction of specific features that are indicative of good grasps. Our survey provides an overview of the different methodologies and discusses open problems in the area of robot grasping. We also draw a parallel to the classical approaches that rely on analytic formulations.Comment: 20 pages, 30 Figures, submitted to IEEE Transactions on Robotic

Sample and feedback efficient hierarchical reinforcement learning from human preferences

While reinforcement learning has led to promising results in robotics, defining an informative reward function can sometimes prove to be challenging. Prior work considered including the human in the loop to jointly learn the reward function and the optimal policy. Generating samples from a physical robot and requesting human feedback are both taxing efforts for which efficiency is critical. In contrast to prior work, in this paper we propose to learn reward functions from both the robot and the human perspectives in order to improve on both efficiency metrics. On one side, learning a reward function from the human perspective increases feedback efficiency by assuming that humans rank trajectories according to an outcome space of reduced dimensionaltiy. On the other side, learning a reward function from the robot perspective circumvents the need for learning a dynamics model while retaining the sample efficiency of model-based approaches. We provide an algorithm that incorporates bi-perspective reward learning into a general hierarchical reinforcement learning framework and demonstrate the merits of our approach on a toy task and a simulated robot grasping task

Low-Dimensional Learning for Complex Robots

© 2015 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.DOI: 10.1109/TASE.2014.2349915This paper presents an algorithm for learning the switching policy and the boundaries conditions between primitive controllers that maximize the translational movements of a complex locomoting system. The algorithm learns an optimal action for each boundary condition instead of one for each discretized state-action pair of the system, as is typically done in machine learning. The system is model as a hybrid system because it contains both discrete and continuous dynamics. With this hybridification of the system and with this abstraction of learning boundary-action pairs, the “curse of dimensionality” is mitigated. The effectiveness of this learning algorithm is demonstrated on both a simulated system and on a physical robotic system. In both cases, the algorithm is able to learn the hybrid control strategy that maximizes the forward translational movement of the system without the need for human involvement

Data-Driven Grasp Synthesis—A Survey

We review the work on data-driven grasp synthesis and the methodologies for sampling and ranking candidate grasps. We divide the approaches into three groups based on whether they synthesize grasps for known, familiar, or unknown objects. This structure allows us to identify common object representations and perceptual processes that facilitate the employed data-driven grasp synthesis technique. In the case of known objects, we concentrate on the approaches that are based on object recognition and pose estimation. In the case of familiar objects, the techniques use some form of a similarity matching to a set of previously encountered objects. Finally, for the approaches dealing with unknown objects, the core part is the extraction of specific features that are indicative of good grasps. Our survey provides an overview of the different methodologies and discusses open problems in the area of robot grasping. We also draw a parallel to the classical approaches that rely on analytic formulations

Equilibri del robot AIBO utilitzant DMPs

El treball presentat s'emmarca en una iniciativa global que te com a objectiu la recuperació del robot AIBO de Sony. Aquest treball s'ha fet per demostrar el bon funcionament de l'arquitectura proposada. L'arquitectura que es proposa es tal que permet la comunicació de l'AIBO amb ROS (Robot Operating System), a través d'un client URBI (Universal Robot Body Interface). En aquest treball, s'exposa com s'es capaç d'implementar l'algorisme de DMP (Dyamic Movement Primitive) a través d'un entorn estat de l'art com es ROS. El robot AIBO es controlat en tot moment, tot i processar-se l'algorisme de DMPs fora del robot, amb un temps de resposta adequat per la tasca de reaccionar davant els moviments no desitjats de la plataforma sobre la que es troba. Aquests moviments són interpretats gràcies a un sensor triaxial d'accelerometria (MPU6050) i un giroscopi de tres eixos (GY-521), col locats sobre el robot. Finalment, es plantegen futurs treballs per millorar la tasca utilitzant l'algorisme PI2 (Path Integral Policy Improvement), una plataforma automatitzada, visi o i la millora d'un model creat a l'inici del treball

Learning motion primitive goals for robust manipulation

Abstract — Applying model-free reinforcement learning to manipulation remains challenging for several reasons. First, manipulation involves physical contact, which causes discontinuous cost functions. Second, in manipulation, the end-point of the movement must be chosen carefully, as it represents a grasp which must be adapted to the pose and shape of the object. Finally, there is uncertainty in the object pose, and even the most carefully planned movement may fail if the object is not at the expected position. To address these challenges we 1) present a simplified, computationally more efficient version of our model-free reinforcement learning algorithm PI 2; 2) extend PI 2 so that it simultaneously learns shape parameters and goal parameters of motion primitives; 3) use shape and goal learning to acquire motion primitives that are robust to object pose uncertainty. We evaluate these contributions on a manipulation platform consisting of a 7-DOF arm with a 4-DOF hand. This paper is accompanied by a video, which can also be downloaded at