Demonstration of Object Recognition Using DOPE Deep Learning Algorithm for Collaborative Robotics

When collaborating on a common task, passing, or receiving various objects such as tools between each other is one of the most common interaction methods among humans. Similarly, it is expected to be a common and important interaction method in a fluent and natural human-robot collaboration. This thesis studied human-robot-interaction in the context of unilateral robot-to-human handover task. More specifically, it focused on studying grasping an object using a state-of-the-art machine learning algorithm called Guided Uncertainty-Aware Policy Optimization (GUAPO). Within the broader scope of the whole GUAPO algorithm, it was limited to only demonstrating the object detection and pose estimation part of the task. In this case, it was implemented using an object pose estimation algorithm called Deep Object Pose Estimation (DOPE). DOPE is a deep learning approach to predict image key points from a large-enough set of training data of an object-of-interest. The challenge of having enough training data for teaching a supervised machine learning-based machine vision algorithm was tackled by creating a synthetic (computer generated) dataset. The dataset needed to represent the real-life scenario closely to beat the so-called reality-gap. This dataset was created with Unreal Engine 4 (UE4) and NVIDIA Deep learning Dataset Synthesizer (NDDS). During the experimental part, a 3D model of the object-of-interest was created using Blender and the object was imported into the created UE4 environment. NDDS was used to create and extract the training dataset for DOPE. DOPE’s functionality was successfully tested with a pre-trained network and then it was manually shown that it is possible to start training the DOPE algorithm with the dataset created. However, the lack of computing power became the limitation of this work, and it was not possible to train the DOPE algorithm enough to recognize the object-of-interest. The results prove this to be an effective way to approach training object recognition algorithms, albeit being technologically challenging to do from scratch, as knowledge of broad sets of software and programming skills are needed

Incremental Learning for Robot Perception through HRI

Scene understanding and object recognition is a difficult to achieve yet crucial skill for robots. Recently, Convolutional Neural Networks (CNN), have shown success in this task. However, there is still a gap between their performance on image datasets and real-world robotics scenarios. We present a novel paradigm for incrementally improving a robot's visual perception through active human interaction. In this paradigm, the user introduces novel objects to the robot by means of pointing and voice commands. Given this information, the robot visually explores the object and adds images from it to re-train the perception module. Our base perception module is based on recent development in object detection and recognition using deep learning. Our method leverages state of the art CNNs from off-line batch learning, human guidance, robot exploration and incremental on-line learning

Interactive multiple object learning with scanty human supervision

© 2016. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/We present a fast and online human-robot interaction approach that progressively learns multiple object classifiers using scanty human supervision. Given an input video stream recorded during the human robot interaction, the user just needs to annotate a small fraction of frames to compute object specific classifiers based on random ferns which share the same features. The resulting methodology is fast (in a few seconds, complex object appearances can be learned), versatile (it can be applied to unconstrained scenarios), scalable (real experiments show we can model up to 30 different object classes), and minimizes the amount of human intervention by leveraging the uncertainty measures associated to each classifier.; We thoroughly validate the approach on synthetic data and on real sequences acquired with a mobile platform in indoor and outdoor scenarios containing a multitude of different objects. We show that with little human assistance, we are able to build object classifiers robust to viewpoint changes, partial occlusions, varying lighting and cluttered backgrounds. (C) 2016 Elsevier Inc. All rights reserved.Peer ReviewedPostprint (author's final draft