In situ interactive teaching of trustworthy robotic assistants

©2007 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Presented at the 2007 IEEE International Conference on Systems, Man and Cybernetics, October 7-10, 2007, Montréal.DOI: 10.1109/ICSMC.2007.4414025In this paper we discuss a method for transferring human knowledge to a robotic platform via teleoperation. The method combines unsupervised clustering and classification with interactive instruction to enable behavior capture in a transferable form. We discuss the approach in both simulation and robotic hardware platform to show the capability of the learning system. In this work we also present a definition and associated metric for trustworthiness, and relate this quantity to system performance. Improved performance and trustworthiness are motivations for our application of interactive learning, and we present results that indicate that these were indeed attained

Toward supervised reinforcement learning with partial states for social HRI

Social interacting is a complex task for which machine learning holds particular promise. However, as no sufficiently accurate simulator of human interactions exists today, the learning of social interaction strategies has to happen online in the real world. Actions executed by the robot impact on humans, and as such have to be carefully selected, making it impossible to rely on random exploration. Additionally, no clear reward function exists for social interactions. This implies that traditional approaches used for Reinforcement Learning cannot be directly applied for learning how to interact with the social world. As such we argue that robots will profit from human expertise and guidance to learn social interactions. However, as the quantity of input a human can provide is limited, new methods have to be designed to use human input more efficiently. In this paper we describe a setup in which we combine a framework called Supervised Progressively Autonomous Robot Competencies (SPARC), which allows safer online learning with Reinforcement Learning, with the use of partial states rather than full states to accelerate generalisation and obtain a usable action policy more quickly

Investigating the Effects of Robot Engagement Communication on Learning from Demonstration

Robot Learning from Demonstration (RLfD) is a technique for robots to derive policies from instructors' examples. Although the reciprocal effects of student engagement on teacher behavior are widely recognized in the educational community, it is unclear whether the same phenomenon holds true for RLfD. To fill this gap, we first design three types of robot engagement behavior (attention, imitation, and a hybrid of the two) based on the learning literature. We then conduct, in a simulation environment, a within-subject user study to investigate the impact of different robot engagement cues on humans compared to a "without-engagement" condition. Results suggest that engagement communication significantly changes the human's estimation of the robots' capability and significantly raises their expectation towards the learning outcomes, even though we do not run actual learning algorithms in the experiments. Moreover, imitation behavior affects humans more than attention does in all metrics, while their combination has the most profound influences on humans. We also find that communicating engagement via imitation or the combined behavior significantly improve humans' perception towards the quality of demonstrations, even if all demonstrations are of the same quality.Comment: Under revie

Additive Manufacturing for Nautical Design An Automated Approach to Marine Manufacturing

How can additive manufacturing (AM) technology be applied to automate the production of small marine vessels? For the past 50 years small (below 40 meters) marine vessel manufacturing has been dominated by moulded fiber-reinforced plastics (FRP). There are several shortcomings to this manufacturing method that affect both the formal outcome and the manufacturing process of boats built in FRP: 1) manufacturing requires the use of expensive moulds, 2) formal geometric freedom is limited by moulds which reduce the potential for customization, and 3) special assemblies and structural reinforcements must be moulded separately and joined using a time-consuming hand lay-up process. The use of AM may reduce cost of production by eliminating need for moulds, allow greater ease of customization, and improve worker safety by limiting exposure to harmful materials and chemicals. The purpose of this research project is to evaluate existing AM technology and assess its potential for application to small marine vessel manufacturing. The project aims to investigate new methods for generating novel AM tool paths and demonstrate through proof of concept that it may be possible to produce the complex topological surfaces and assemblies that are common in marine vessels using multi-bias additive manufacturing (MBAM). However, AM is a broad term that describes a variety of different ways to manufacture objects. As such, AM can be applied to marine manufacturing in a variety of different ways, in different phases of the manufacturing process, and to different extents. At the same time, building boats is a complex process that presents specific problems that must be addressed in any automation solution. Several marine vessel construction projects have already been completed using AM which can serve as case studies for understanding the opportunities and challenges for applying AM to the marine sector. A review of the current state of the technology and qualitative analysis (QA) of case studies provides a set of guidelines for designing a manufacturing method that may prove effective for producing small marine vessels using AM. The project relied on design-based research (DBR) to develop a series of experimental extruder prototypes for novel toolpath testing on excerpts from a small reference vessel. The combination of QA and DBR experimentation point to a manufacturing solution using articulated robotic manipulators and a continuous fiber thermoset plastic extruder using a modified version of the fused filament fabrication process. This kinematic solution can be extended with external linear or rotational axes and/or by mounting robotic manipulators within a large gantry. This will allow the extruder to approach the work using a wide range of orientations that will be optimal for both the geometry of marine vessels and the requirements of MBAM extrusion. Meanwhile, toolpath generation using the software Grasshopper with KukaPRC plugin demonstrated a proof of concept for creating MBAM toolpaths optimized for small marine vessels. While the method proved feasible for smaller excerpts there remain significant challenges to successful deployment of this manufacturing method that can only be addressed with additional research

Situated messages for asynchronous human-robot interaction

An ongoing issue in human robot interaction (HRI) is how people and robots communicate with one another. While there is considerable work in real-time human-robot communication, fairly little has been done in asynchronous realm. Our approach, which we call situated messages, lets humans and robots asynchronously exchange information by placing physical tokens – each representing a simple message – in meaningful physical locations of their shared environment. Using knowledge of the robot’s routines, a person can place a message token at a location, where the location is typically relevant to redirecting the robot’s behavior at that location. When the robot passes nearby that location, it detects the message and reacts accordingly. Similarly, robots can themselves place tokens at specific locations for people to read. Thus situated messages leverages embodied interaction, where token placement exploits the everyday practices and routines of both people and robots. We describe our working prototype, introduce application scenarios, explore message categories and usage patterns, and suggest future directions

Using Echo State Networks for Robot Navigation Behavior Acquisition

International audienceRobot Behavior Learning by Demonstration deals with the ability for a robot to learn a behavior from one or several demonstrations provided by a human teacher, possibly through tele-operation or imitation. This implies controllers that can address both (1) the feature selection problem related to a great amount of mostly irrelevant sensory data and (2) dealing with temporal sequences of demonstrations. Echo State Networks have been proposed recently for time series prediction and have been shown to perform remarkably well on this kind of data. In this paper, we introduce ESN to robot behavior acquisition in the scope of a mobile robot performing navigation tasks. ESN actually show comparable and even better performance with that of other algorithms from the literature in similar experimental conditions. Moreover, some properties regarding dynamics of ESN in the context of learning by demonstration are investigated

Pragmatic Frames for Teaching and Learning in Human-Robot interaction: Review and Challenges

Vollmer A-L, Wrede B, Rohlfing KJ, Oudeyer P-Y. Pragmatic Frames for Teaching and Learning in Human-Robot interaction: Review and Challenges. FRONTIERS IN NEUROROBOTICS. 2016;10: 10.One of the big challenges in robotics today is to learn from human users that are inexperienced in interacting with robots but yet are often used to teach skills flexibly to other humans and to children in particular. A potential route toward natural and efficient learning and teaching in Human-Robot Interaction (HRI) is to leverage the social competences of humans and the underlying interactional mechanisms. In this perspective, this article discusses the importance of pragmatic frames as flexible interaction protocols that provide important contextual cues to enable learners to infer new action or language skills and teachers to convey these cues. After defining and discussing the concept of pragmatic frames, grounded in decades of research in developmental psychology, we study a selection of HRI work in the literature which has focused on learning-teaching interaction and analyze the interactional and learning mechanisms that were used in the light of pragmatic frames. This allows us to show that many of the works have already used in practice, but not always explicitly, basic elements of the pragmatic frames machinery. However, we also show that pragmatic frames have so far been used in a very restricted way as compared to how they are used in human-human interaction and argue that this has been an obstacle preventing robust natural multi-task learning and teaching in HRI. In particular, we explain that two central features of human pragmatic frames, mostly absent of existing HRI studies, are that (1) social peers use rich repertoires of frames, potentially combined together, to convey and infer multiple kinds of cues; (2) new frames can be learnt continually, building on existing ones, and guiding the interaction toward higher levels of complexity and expressivity. To conclude, we give an outlook on the future research direction describing the relevant key challenges that need to be solved for leveraging pragmatic frames for robot learning and teaching