Learning Task Priorities from Demonstrations

Bimanual operations in humanoids offer the possibility to carry out more than one manipulation task at the same time, which in turn introduces the problem of task prioritization. We address this problem from a learning from demonstration perspective, by extending the Task-Parameterized Gaussian Mixture Model (TP-GMM) to Jacobian and null space structures. The proposed approach is tested on bimanual skills but can be applied in any scenario where the prioritization between potentially conflicting tasks needs to be learned. We evaluate the proposed framework in: two different tasks with humanoids requiring the learning of priorities and a loco-manipulation scenario, showing that the approach can be exploited to learn the prioritization of multiple tasks in parallel.Comment: Accepted for publication at the IEEE Transactions on Robotic

Positioning two redundant arms for cooperative manipulation of objects

Bimanual manipulation of objects is receiving a lot of attention nowadays, but there is few literature addressing the design of the arms configuration. In this paper, we propose a way to analyze the relative positioning of two redundant arms, both equipped with spherical wrists, in order to obtain the best common workspace for grasping purposes. Considering the geometry of a robot with a spherical wrist, the Cartesian workspace can be discretized, with an easy representation of the feasible end-effector orientations at each point using bounding cones. After having characterized the workspace for one robot arm, we can evaluate how good each of the discretized poses relate with an identical arm in another position with a quality function that considers orientations. In the end, we obtain a quality value for each relative position of two arms, and we perform an optimization using genetic algorithms to obtain the best workspace for a cooperative task.Peer ReviewedPostprint (author’s final draft

A Certified-Complete Bimanual Manipulation Planner

Planning motions for two robot arms to move an object collaboratively is a difficult problem, mainly because of the closed-chain constraint, which arises whenever two robot hands simultaneously grasp a single rigid object. In this paper, we propose a manipulation planning algorithm to bring an object from an initial stable placement (position and orientation of the object on the support surface) towards a goal stable placement. The key specificity of our algorithm is that it is certified-complete: for a given object and a given environment, we provide a certificate that the algorithm will find a solution to any bimanual manipulation query in that environment whenever one exists. Moreover, the certificate is constructive: at run-time, it can be used to quickly find a solution to a given query. The algorithm is tested in software and hardware on a number of large pieces of furniture.Comment: 12 pages, 7 figures, 1 tabl

Integration of Dual-Arm Manipulation in a Passivity Based Whole-Body Controller for Torque-Controlled Humanoid Robots

This work presents an extension of balance control for torque-controlled humanoid robots. Within a non-strict task hierarchy, the controller allows the robot to use the feet endeffectors to balance, while the remaining hand end-effectors can be used to perform Dual-Arm manipulation. The controller generates a passive and compliance behaviour to regulate the location of the centre of mass (CoM), the orientation of the hip and the poses of each end-effector assigned to the task of interaction (in this case bi-manipulation). Then, an appropriate wrench (force and torque) is applied to each of the end-effectors employed for the task to achieve this purpose. Now, in this new controller, the essential requirement focuses on the fact that the desired wrench in the CoM is computed through the sum of the balancing and bi-manipulation wrenches. The bimanipulation wrenches are obtained through a new dynamic model that allows executing tasks of approaching the grip and manipulation of large objects compliantly. On the other hand, the feedback controller has been maintained but in combination with a bi-manipulation-oriented feedforward control to improve the performance in the object trajectory tracking. This controller is tested in different experiments with the robot TORO

Stabilize to Act: Learning to Coordinate for Bimanual Manipulation

Key to rich, dexterous manipulation in the real world is the ability to coordinate control across two hands. However, while the promise afforded by bimanual robotic systems is immense, constructing control policies for dual arm autonomous systems brings inherent difficulties. One such difficulty is the high-dimensionality of the bimanual action space, which adds complexity to both model-based and data-driven methods. We counteract this challenge by drawing inspiration from humans to propose a novel role assignment framework: a stabilizing arm holds an object in place to simplify the environment while an acting arm executes the task. We instantiate this framework with BimanUal Dexterity from Stabilization (BUDS), which uses a learned restabilizing classifier to alternate between updating a learned stabilization position to keep the environment unchanged, and accomplishing the task with an acting policy learned from demonstrations. We evaluate BUDS on four bimanual tasks of varying complexities on real-world robots, such as zipping jackets and cutting vegetables. Given only 20 demonstrations, BUDS achieves 76.9% task success across our task suite, and generalizes to out-of-distribution objects within a class with a 52.7% success rate. BUDS is 56.0% more successful than an unstructured baseline that instead learns a BC stabilizing policy due to the precision required of these complex tasks. Supplementary material and videos can be found at https://sites.google.com/view/stabilizetoact .Comment: Conference on Robot Learning, 202