Fast Object Pose Estimation Using Adaptive Threshold for Bin-Picking

Robotic bin-picking is a common process in modern manufacturing, logistics, and warehousing that aims to pick-up known or unknown objects with random poses out of a bin by using a robot-camera system. Rapid and accurate object pose estimation pipelines have become an escalating issue for robot picking in recent years. In this paper, a fast 6-DoF (degrees of freedom) pose estimation pipeline for random bin-picking is proposed in which the pipeline is capable of recognizing different types of objects in various cluttered scenarios and uses an adaptive threshold segment strategy to accelerate estimation and matching for the robot picking task. Particularly, our proposed method can be effectively trained with fewer samples by introducing the geometric properties of objects such as contour, normal distribution, and curvature. An experimental setup is designed with a Kinova 6-Dof robot and an Ensenso industrial 3D camera for evaluating our proposed methods with respect to four different objects. The results indicate that our proposed method achieves a 91.25% average success rate and a 0.265s average estimation time, which sufficiently demonstrates that our approach provides competitive results for fast objects pose estimation and can be applied to robotic random bin-picking tasks

Multiple object detection of workpieces based on fusion of deep learning and image processing

A workpiece detection method based on fusion of deep learning and image processing is proposed. Firstly, the workpiece bounding boxes are located in the workpiece images by YOLOv3, whose parameters are compressed by an improved convolutional neural network residual structure pruning strategy. Then, the workpiece images are cropped based on the bounding boxes with cropping biases. Finally, the contours and suitable gripping points of the workpieces are obtained through image processing. The experimental results show that mean Average Precision (mAP) is 98.60% for YOLOv3, and 99.38% for that one by pruning 50.89% of its parameters, and the inference time is shortened by 31.13%. Image processing effectively corrects the bounding boxes obtained by deep learning, and obtains workpiece contour and gripping point information

Collaborative and Cooperative Robotics Applications using Visual Perception

The objective of this Thesis is to develop novel integrated strategies for collaborative and cooperative robotic applications. Commonly, industrial robots operate in structured environments and in work-cell separated from human operators. Nowadays, collaborative robots have the capacity of sharing the workspace and collaborate with humans or other robots to perform complex tasks. These robots often operate in an unstructured environment, whereby they need sensors and algorithms to get information about environment changes. Advanced vision and control techniques have been analyzed to evaluate their performance and their applicability to industrial tasks. Then, some selected techniques have been applied for the first time to an industrial context. A Peg-in-Hole task has been chosen as first case study, since it has been extensively studied but still remains challenging: it requires accuracy both in the determination of the hole poses and in the robot positioning. Two solutions have been developed and tested. Experimental results have been discussed to highlight the advantages and disadvantages of each technique. Grasping partially known objects in unstructured environments is one of the most challenging issues in robotics. It is a complex task and requires to address multiple subproblems, in order to be accomplished, including object localization and grasp pose detection. Also for this class of issues some vision techniques have been analyzed. One of these has been adapted to be used in industrial scenarios. Moreover, as a second case study, a robot-to-robot object handover task in a partially structured environment and in the absence of explicit communication between the robots has been developed and validated. Finally, the two case studies have been integrated in two real industrial setups to demonstrate the applicability of the strategies to solving industrial problems

Interactive singulation of objects from a pile

Abstract—Interaction with unstructured groups of objects allows a robot to discover and manipulate novel items in cluttered environments. We present a framework for interactive singulation of individual items from a pile. The proposed framework provides an overall approach for tasks involving operation on multiple objects, such as counting, arranging, or sorting items in a pile. A perception module combined with pushing actions accumulates evidence of singulated items over multiple pile interactions. A decision module scores the likelihood of a single-item pile to a multiple-item pile based on the magnitude of motion and matching determined from the perception module. Three variations of the singulation framework were evaluated on a physical robot for an arrangement task. The proposed interactive singulation method with adaptive pushing reduces the grasp errors on non-singulated piles compared to alternative methods without the perception and decision modules. This work contributes the general pile interaction framework, a specific method for integrating perception and action plans with grasp decisions, and an experimental evaluation of the cost trade-offs for different singulation methods. I

On Sensor-Controlled Robotized One-off Manufacturing

A semi-automatic task oriented system structure has been developed and tested on an arc welding application. In normal industrial robot programming, the path is created and the process is based upon the decided path. Here a process-oriented method is proposed instead. It is natural to focus on the process, since the path is in reality a result of process needs. Another benefit of choosing process focus, is that it automatically leads us into task oriented thoughts, which in turn can be split in sub-tasks, one for each part of the process with similar process-characteristics. By carefully choosing and encapsulating the information needed to execute a sub-task, this component can be re-used whenever the actual subtask occurs. By using virtual sensors and generic interfaces to robots and sensors, applications built upon the system design do not change between simulation and actual shop floor runs. The system allows a mix of real- and simulated components during simulation and run-time

Component-based synthesis of motion planning algorithms

Combinatory Logic Synthesis generates data or runnable programs according to formal type specifications. Synthesis results are composed based on a user-specified repository of components, which brings several advantages for representing spaces of high variability. This work suggests strategies to manage the resulting variations by proposing a domain-specific brute-force search and a machine learning-based optimization procedure. The brute-force search involves the iterative generation and evaluation of machining strategies. In contrast, machine learning optimization uses statistical models to enable the exploration of the design space. The approaches involve synthesizing programs and meta-programs that manipulate, run, and evaluate programs. The methodologies are applied to the domain of motion planning algorithms, and they include the configuration of programs belonging to different algorithmic families. The study of the domain led to the identification of variability points and possible variations. Proof-of-concept repositories represent these variability points and incorporate them into their semantic structure. The selected algorithmic families involve specific computation steps or data structures, and corresponding software components represent possible variations. Experimental results demonstrate that CLS enables synthesis-driven domain-specific optimization procedures to solve complex problems by exploring spaces of high variability.Combinatory Logic Synthesis (CLS) generiert Daten oder lauffähige Programme anhand von formalen Typspezifikationen. Die Ergebnisse der Synthese werden auf Basis eines benutzerdefinierten Repositories von Komponenten zusammengestellt, was diverse Vorteile für die Beschreibung von Räumen mit hoher Variabilität mit sich bringt. Diese Arbeit stellt Strategien für den Umgang mit den resultierenden Variationen vor, indem eine domänen-spezifische Brute-Force Suche und ein maschinelles Lernverfahren für die Untersuchung eines Optimierungsproblems aufgezeigt werden. Die Brute-Force Suche besteht aus der iterativen Generierung und Evaluation von Frässtrategien. Im Gegensatz dazu nutzt der Optimierungsansatz statistische Modelle zur Erkundung des Entwurfsraums. Beide Ansätze synthetisieren Programme und Metaprogramme, welche Programme bearbeiten, ausführen und evaluieren. Diese Methoden werden auf die Domäne der Bewegungsplanungsalgorithmen angewendet und sie beinhalten die Konfiguration von Programmen, welche zu unterschiedlichen algorithmischen Familien gehören. Die Untersuchung der Domäne führte zur Identifizierung der Variabilitätspunkte und der möglichen Variationen. Entsprechende Proof of Concept Implementierungen in Form von Repositories repräsentieren jene Variabilitätspunkte und beziehen diese in ihre semantische Struktur ein. Die gewählten algorithmischen Familien sehen bestimmte Berechnungsschritte oder Datenstrukturen vor, und entsprechende Software Komponenten stellen mögliche Variationen dar. Versuchsergebnisse belegen, dass CLS synthese-getriebene domänenspezifische Optimierungsverfahren ermöglicht, welche komplexe Probleme durch die Exploration von Räumen hoher Variabilität lösen