Open World Assistive Grasping Using Laser Selection

Many people with motor disabilities are unable to complete activities of daily living (ADLs) without assistance. This paper describes a complete robotic system developed to provide mobile grasping assistance for ADLs. The system is comprised of a robot arm from a Rethink Robotics Baxter robot mounted to an assistive mobility device, a control system for that arm, and a user interface with a variety of access methods for selecting desired objects. The system uses grasp detection to allow previously unseen objects to be picked up by the system. The grasp detection algorithms also allow for objects to be grasped in cluttered environments. We evaluate our system in a number of experiments on a large variety of objects. Overall, we achieve an object selection success rate of 88% and a grasp detection success rate of 90% in a non-mobile scenario, and success rates of 89% and 72% in a mobile scenario

Grasping of Solid Industrial Objects Using 3D Registration

Robots allow industrial manufacturers to speed up production and to increase the product’s quality. This paper deals with the grasping of partially known industrial objects in an unstructured environment. The proposed approach consists of two main steps: (1) the generation of an object model, using multiple point clouds acquired by a depth camera from different points of view; (2) the alignment of the generated model with the current view of the object in order to detect the grasping pose. More specifically, the model is obtained by merging different point clouds with a registration procedure based on the iterative closest point (ICP) algorithm. Then, a grasping pose is placed on the model. Such a procedure only needs to be executed once, and it works even in the presence of objects only partially known or when a CAD model is not available. Finally, the current object view is aligned to the model and the final grasping pose is estimated. Quantitative experiments using a robot manipulator and three different real-world industrial objects were conducted to demonstrate the effectiveness of the proposed approach

A Practical Approach for Picking Items in an Online Shopping Warehouse

Commercially viable automated picking in unstructured environments by a robot arm remains a difficult challenge. The problem of robot grasp planning has long been around but the existing solutions tend to be limited when it comes to deploy them in open-ended realistic scenarios. Practical picking systems are called for that can handle the different properties of the objects to be manipulated, as well as the problems arising from occlusions and constrained accessibility. This paper presents a practical solution to the problem of robot picking in an online shopping warehouse by means of a novel approach that integrates a carefully selected method with a new strategy, the centroid normal approach (CNA), on a cost-effective dual-arm robotic system with two grippers specifically designed for this purpose: a two-finger gripper and a vacuum gripper. Objects identified in the scene point cloud are matched to the grasping techniques and grippers to maximize success. Extensive experimentation provides clues as to what are the reasons for success and failure. We chose as benchmark the scenario proposed by the 2017 Amazon Robotics Challenge, since it represents a realistic description of a retail shopping warehouse case; it includes many challenging constraints, such as a wide variety of different product items with a diversity of properties, which are also presented with restricted visibility and accessibility.This paper describes research conducted at the UJI Robotic Intelligence Laboratory. Support for this laboratory is provided in part by Ministerio de Economía y Competitividad (DPI2015-69041-R, DPI2017-89910-R), by Universitat Jaume I (P1-1B2014-52) and by Generalitat Valenciana (PROMETEO/2020/034). The first author was recipient of an Erasmus Mundus scholarship by the European Commission for the EMARO+ Master Program

Robotic manipulation of multiple objects as a POMDP

This paper investigates manipulation of multiple unknown objects in a crowded environment. Because of incomplete knowledge due to unknown objects and occlusions in visual observations, object observations are imperfect and action success is uncertain, making planning challenging. We model the problem as a partially observable Markov decision process (POMDP), which allows a general reward based optimization objective and takes uncertainty in temporal evolution and partial observations into account. In addition to occlusion dependent observation and action success probabilities, our POMDP model also automatically adapts object specific action success probabilities. To cope with the changing system dynamics and performance constraints, we present a new online POMDP method based on particle filtering that produces compact policies. The approach is validated both in simulation and in physical experiments in a scenario of moving dirty dishes into a dishwasher. The results indicate that: 1) a greedy heuristic manipulation approach is not sufficient, multi-object manipulation requires multi-step POMDP planning, and 2) on-line planning is beneficial since it allows the adaptation of the system dynamics model based on actual experience

Griff-in-die-Kiste - Neue Ansätze für ein klassisches Problem

The automation of handling tasks has been an important scientific topic since the development of the first industrial robots. The first step in the chain of scientific challenges to be solved is the automatic grasping of objects. One of the most famous examples in this context is the well known ”bin-picking” problem. To pick up objects, scrambled in a box is an easy task for humans, but its automation is very complex. Besides the localization of the object, meaning the estimation of the object’s pose (orientation and position), it has to be ensured that a collision free path can be found to safely grasp the objects. For over 50 years, researchers have published approaches towards generic solutions to this problem, but unfortunately no industry applicable, generic system has been developed yet. In this thesis, three different approaches to solve the bin-picking problem are described. More precisely, different solutions to the pose estimation problem are introduced, each paired with additional functionalities to complete it for application in a bin-picking station. It is described, how modern sensors can be used for efficient bin-picking as well as how classic sensor concepts can be applied for novel bin-picking techniques. Three complete systems are described and compared. First, 3D point clouds, generated using a laser scanner, are used as basis. Employing the known Random Sample Matching algorithm and modifications of it, paired with a very efficient depth map based collision avoidance mechanism results in a very robust bin-picking approach. In the second approach, all computations are done on depth maps. This allows the use of 2D image analysis techniques to fulfill the tasks and results in real time data analysis. Combined with force/torque and acceleration sensors, a near time optimal bin-picking system emerges. As a third option, surface normal maps are employed as a basis for pose estimation. In contrast to known approaches, the normal maps are not used for 3D data computation but directly for the object localization problem. This enables the application of a new class of sensors for bin-picking. All three methods are compared and advantages and disadvantages of each approach are discussed.Das automatisierte Handling von Objekten ist seit Entwicklung der ersten Roboter ein Forschungsthema. Der erste Schritt in diese Richtung ist das automatische Greifen von Objekten. Eines der berühmtesten Probleme in diesem Zusammenhang ist der "Griff-in-die-Kiste", oder "Bin-Picking". Frei angeordnete Objekte (Schüttgut) aus einer Kiste zu entnehmen stellt für Menschen keine schwierige Aufgabe dar, ist jedoch extrem komplex zu automatisieren. Neben der Objektlokalisierung, also dem Bestimmen der Position und der Orientierung, der Pose, des Objekts muss hier auch gewährleistet werden, dass eine kollisionsfreie Interaktion des Roboters mit dem Objekt möglich ist. Seit mehr als 50 Jahren veröffentlichen Forscher Ansätze, um einer generischen Lösung dieses Problems näher zu kommen. Dennoch ist Bin-Picking auch heute noch nicht vollständig gelöst. Diese Arbeit beschreibt daher drei neue, unterschiedliche Konzepte um das Bin-Picking-Problem zu lösen. Genauer gesagt werden Verfahren vorgestellt, die auf Basis unterschiedlicher Daten Objekte lokalisieren können. Die Arbeit beschreibt, wie moderne optische Sensoren effizient für das Bin-Picking eingesetzt werden können, aber auch, dass klassische Sensorkonzepte neuartige und effiziente Lösungen ermöglichen. Drei Systeme werden beschrieben und verglichen. Zunächst werden per 3D-Scanner aufgenommene Punktwolken als Basis genutzt und mittels Random Sample Matching Objektposen extrahiert. Die Kollisionsvermeidungsstrategie basiert auf Tiefenbildern, was die Berechnung sehr effizient macht. Als zweites wird die Lokalisierung direkt auf Tiefenbildern gerechnet. Dies ermöglicht den direkten Einsatz von 2d Bildverarbeitungsmethoden, was eine Greifposenbestimmung in Echtzeit ermöglicht. Verbunden mit Kraft-Momentensensorik entsteht so ein nahezu zeitoptimales Bin-Picking-System. Als dritte Möglichkeit werden Oberflächennormalenkarten als Basis zur Objektlokalisierung verwendet. Im Gegensatz zu herkömmlichen Ansätzen aus der Literatur werden diese Karten nicht zu 3d Daten umgerechnet sondern direkt zur Posenschätzung genutzt. Dies ermöglicht den Einsatz einer Klasse von Sensoren zum Bin-Picking die bisher nur in anderen Gebieten genutzt werden konnte. Alle drei Methoden werden miteinander verglichen und Vor- sowie Nachteile beleuchtet