Interactive Task Encoding System for Learning-from-Observation

We introduce a practical pipeline that interactively encodes multimodal human demonstrations for robot teaching. This pipeline is designed as an input system for a framework called Learning-from-Observation (LfO), which aims to program household robots with manipulative tasks through few-shots human demonstration without coding. While most previous LfO systems run with visual demonstration, recent research on robot teaching has shown the effectiveness of verbal instruction in making recognition robust and teaching interactive. To the best of our knowledge, however, no LfO system has yet been proposed that utilizes both verbal instruction and interaction, namely \textit{multimodal LfO}. This paper proposes the interactive task encoding system (ITES) as an input pipeline for multimodal LfO. ITES assumes that the user teaches step-by-step, pausing hand movements in order to match the granularity of human instructions with the granularity of robot execution. ITES recognizes tasks based on step-by-step verbal instructions that accompany the hand movements. Additionally, the recognition is made robust through interactions with the user. We test ITES on a real robot and show that the user can successfully teach multiple operations through multimodal demonstrations. The results suggest the usefulness of ITES for multimodal LfO. The source code is available at https://github.com/microsoft/symbolic-robot-teaching-interface.Comment: 7 pages, 10 figures. Last updated January 24st, 202

AR-Enhanced Human-Robot-Interaction - Methodologies, Algorithms, Tools

By using Augmented Reality in Human-Robot-Interaction scenariospropose it is possible to improve training, programming, maintenance and process monitoring. AR Enhanced Human Robot Interaction means it is possible to conduct activities not only in a training facility with physical robot(s) but also in a complete virtual environment. By using virtual environments only a computer and possibly Head Mounting Display is required. This will reduce the bottlenecks for with overbooked physical training facilities. Physical environment for the activities with robot(s) will still be required, however using also virtual environments will increase flexibility and human operator can focus on training more complicated tasks. (C) 2016 The Authors. Published by Elsevier B.V.Partially funded by FP7 EU project LIAA (http://www.project- leanautomation.eu/

Intelligent multi-sensor integrations

Growth in the intelligence of space systems requires the use and integration of data from multiple sensors. Generic tools are being developed for extracting and integrating information obtained from multiple sources. The full spectrum is addressed for issues ranging from data acquisition, to characterization of sensor data, to adaptive systems for utilizing the data. In particular, there are three major aspects to the project, multisensor processing, an adaptive approach to object recognition, and distributed sensor system integration