Data-Driven Grasp Synthesis - A Survey

We review the work on data-driven grasp synthesis and the methodologies for sampling and ranking candidate grasps. We divide the approaches into three groups based on whether they synthesize grasps for known, familiar or unknown objects. This structure allows us to identify common object representations and perceptual processes that facilitate the employed data-driven grasp synthesis technique. In the case of known objects, we concentrate on the approaches that are based on object recognition and pose estimation. In the case of familiar objects, the techniques use some form of a similarity matching to a set of previously encountered objects. Finally for the approaches dealing with unknown objects, the core part is the extraction of specific features that are indicative of good grasps. Our survey provides an overview of the different methodologies and discusses open problems in the area of robot grasping. We also draw a parallel to the classical approaches that rely on analytic formulations.Comment: 20 pages, 30 Figures, submitted to IEEE Transactions on Robotic

Real-Time Online Re-Planning for Grasping Under Clutter and Uncertainty

We consider the problem of grasping in clutter. While there have been motion planners developed to address this problem in recent years, these planners are mostly tailored for open-loop execution. Open-loop execution in this domain, however, is likely to fail, since it is not possible to model the dynamics of the multi-body multi-contact physical system with enough accuracy, neither is it reasonable to expect robots to know the exact physical properties of objects, such as frictional, inertial, and geometrical. Therefore, we propose an online re-planning approach for grasping through clutter. The main challenge is the long planning times this domain requires, which makes fast re-planning and fluent execution difficult to realize. In order to address this, we propose an easily parallelizable stochastic trajectory optimization based algorithm that generates a sequence of optimal controls. We show that by running this optimizer only for a small number of iterations, it is possible to perform real time re-planning cycles to achieve reactive manipulation under clutter and uncertainty.Comment: Published as a conference paper in IEEE Humanoids 201

Active model learning and diverse action sampling for task and motion planning

The objective of this work is to augment the basic abilities of a robot by learning to use new sensorimotor primitives to enable the solution of complex long-horizon problems. Solving long-horizon problems in complex domains requires flexible generative planning that can combine primitive abilities in novel combinations to solve problems as they arise in the world. In order to plan to combine primitive actions, we must have models of the preconditions and effects of those actions: under what circumstances will executing this primitive achieve some particular effect in the world? We use, and develop novel improvements on, state-of-the-art methods for active learning and sampling. We use Gaussian process methods for learning the conditions of operator effectiveness from small numbers of expensive training examples collected by experimentation on a robot. We develop adaptive sampling methods for generating diverse elements of continuous sets (such as robot configurations and object poses) during planning for solving a new task, so that planning is as efficient as possible. We demonstrate these methods in an integrated system, combining newly learned models with an efficient continuous-space robot task and motion planner to learn to solve long horizon problems more efficiently than was previously possible.Comment: Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain. https://www.youtube.com/playlist?list=PLoWhBFPMfSzDbc8CYelsbHZa1d3uz-W_

Physics-based motion planning for grasping and manipulation

This thesis develops a series of knowledge-oriented physics-based motion planning algorithms for grasping and manipulation in cluttered an uncertain environments. The main idea is to use high-level knowledge-based reasoning to define the manipulation constraints that define the way how robot should interact with the objects in the environment. These interactions are modeled by incorporating the physics-based model of rigid body dynamics in planning. The first part of the thesis is focused on the techniques to integrate the knowledge with physics-based motion planning. The knowledge is represented in terms of ontologies, a prologbased knowledge inference process is introduced that defines the manipulation constraints. These constraints are used in the state validation procedure of sampling-based kinodynamic motion planners. The state propagator of the motion planner is replaced by a physics-engine that takes care of the kinodynamic and physics-based constraints. To make the interaction humanlike, a low-level physics-based reasoning process is introduced that dynamically varies the control bounds by evaluating the physical properties of the objects. As a result, power efficient motion plans are obtained. Furthermore, a framework has been presented to incorporate linear temporal logic within physics-based motion planning to handle complex temporal goals. The second part of this thesis develops physics-based motion planning approaches to plan in cluttered and uncertain environments. The uncertainty is considered in 1) objects’ poses due to sensing and due to complex robot-object or object-object interactions; 2) uncertainty in the contact dynamics (such as friction coefficient); 3) uncertainty in robot controls. The solution is framed with sampling-based kinodynamic motion planners that solve the problem in open-loop, i.e., it considers uncertainty while planning and computes the solution in such a way that it successfully moves the robot from the start to the goal configuration even if there is uncertainty in the system. To implement the above stated approaches, a knowledge-oriented physics-based motion planning tool is presented. It is developed by extending The Kautham Project, a C++ based tool for sampling-based motion planning. Finally, the current research challenges and future research directions to extend the above stated approaches are discussedEsta tesis desarrolla una serie de algoritmos de planificación del movimientos para la aprehensión y la manipulación de objetos en entornos desordenados e inciertos, basados en la física y el conocimiento. La idea principal es utilizar el razonamiento de alto nivel basado en el conocimiento para definir las restricciones de manipulación que definen la forma en que el robot debería interactuar con los objetos en el entorno. Estas interacciones se modelan incorporando en la planificación el modelo dinámico de los sólidos rígidos. La primera parte de la tesis se centra en las técnicas para integrar el conocimiento con la planificación del movimientos basada en la física. Para ello, se representa el conocimiento mediante ontologías y se introduce un proceso de razonamiento basado en Prolog para definir las restricciones de manipulación. Estas restricciones se usan en los procedimientos de validación del estado de los algoritmos de planificación basados en muestreo, cuyo propagador de estado se susituye por un motor basado en la física que tiene en cuenta las restricciones físicas y kinodinámicas. Además se ha implementado un proceso de razonamiento de bajo nivel que permite adaptar los límites de los controles aplicados a las propiedades físicas de los objetos. Complementariamente, se introduce un marco de desarrollo para la inclusión de la lógica temporal lineal en la planificación de movimientos basada en la física. La segunda parte de esta tesis extiende el enfoque a planificación del movimiento basados en la física en entornos desordenados e inciertos. La incertidumbre se considera en 1) las poses de los objetos debido a la medición y a las interacciones complejas robot-objeto y objeto-objeto; 2) incertidumbre en la dinámica de los contactos (como el coeficiente de fricción); 3) incertidumbre en los controles del robot. La solución se enmarca en planificadores kinodinámicos basados en muestro que solucionan el problema en lazo abierto, es decir que consideran la incertidumbre en la planificación para calcular una solución robusta que permita mover al robot de la configuración inicial a la final a pesar de la incertidumbre. Para implementar los enfoques mencionados anteriormente, se presenta una herramienta de planificación del movimientos basada en la física y guiada por el conocimiento, desarrollada extendiendo The Kautham Project, una herramienta implementada en C++ para la planificación de movimientos basada en muestreo. Finalmente, de discute los retos actuales y las futuras lineas de investigación a seguir para extender los enfoques presentados