Measurement and analysis of interactive behavior in tutoring action with children and robots

Vollmer A-L. Measurement and analysis of interactive behavior in tutoring action with children and robots. Bielefeld: Universität Bielefeld; 2011.Robotics research is increasingly addressing the issue of enabling robots to learn in social interaction. In contrast to the traditional approach by which robots are programmed by experts and prepared for and restricted to one specific purpose, they are now envisioned as general-purpose machines that should be able to carry out different tasks and thus solve various problems in everyday environments. Robots which are able to learn novel actions in social interaction with a human tutor would have many advantages. Unexperienced users could "program" new skills for a robot simply by demonstrating them. Children are able to rapidly learn in social interaction. Modifications in tutoring behavior toward children ("motionese") are assumed to assist their learning processes. Similar to small children, robots do not have much experience of the world and thus could make use of this beneficial natural tutoring behavior if it was employed, when tutoring them. To achieve this goal, the thesis provides theoretical background on imitation learning as a central field of social learning, which has received much attention in robotics and develops new interdisciplinary methods to measure interactive behavior. Based on this background, tutoring behavior is examined in adult-child, adult-adult, and adult-robot interactions by applying the developed methods. The findings reveal that the learner’s feedback is a constituent part of the natural tutoring interaction and shapes the tutor’s demonstration behavior. The work provides an insightful understanding of interactional patterns and processes. From this it derives feedback strategies for human-robot tutoring interactions, with which a robot could prompt hand movement modifications during the tutor’s action demonstration by using its gaze, enabling robots to elicit advantageous modifications of the tutor’s behavior

A User Study on Robot Skill Learning Without a Cost Function: Optimization of Dynamic Movement Primitives via Naive User Feedback

Vollmer A-L, Hemion NJ. A User Study on Robot Skill Learning Without a Cost Function: Optimization of Dynamic Movement Primitives via Naive User Feedback. Frontiers in Robotics and AI. 2018;5: 77.Enabling users to teach their robots new tasks at home is a major challenge for research in personal robotics. This work presents a user study in which participants were asked to teach the robot Pepper a game of skill. The robot was equipped with a state-of-the-art skill learning method, based on dynamic movement primitives (DMPs). The only feedback participants could give was a discrete rating after each of Pepper's movement executions (“very good,” “good,” “average,” “not so good,” “not good at all”). We compare the learning performance of the robot when applying user-provided feedback with a version of the learning where an objectively determined cost via hand-coded cost function and external tracking system is applied. Our findings suggest that (a) an intuitive graphical user interface for providing discrete feedback can be used for robot learning of complex movement skills when using DMP-based optimization, making the tedious definition of a cost function obsolete; and (b) un-experienced users with no knowledge about the learning algorithm naturally tend to apply a working rating strategy, leading to similar learning performance as when using the objectively determined cost. We discuss insights about difficulties when learning from user provided feedback, and make suggestions how learning continuous movement skills from non-expert humans could be improved

Pragmatic Frames for Teaching and Learning in Human-Robot interaction: Review and Challenges

Vollmer A-L, Wrede B, Rohlfing KJ, Oudeyer P-Y. Pragmatic Frames for Teaching and Learning in Human-Robot interaction: Review and Challenges. FRONTIERS IN NEUROROBOTICS. 2016;10: 10.One of the big challenges in robotics today is to learn from human users that are inexperienced in interacting with robots but yet are often used to teach skills flexibly to other humans and to children in particular. A potential route toward natural and efficient learning and teaching in Human-Robot Interaction (HRI) is to leverage the social competences of humans and the underlying interactional mechanisms. In this perspective, this article discusses the importance of pragmatic frames as flexible interaction protocols that provide important contextual cues to enable learners to infer new action or language skills and teachers to convey these cues. After defining and discussing the concept of pragmatic frames, grounded in decades of research in developmental psychology, we study a selection of HRI work in the literature which has focused on learning-teaching interaction and analyze the interactional and learning mechanisms that were used in the light of pragmatic frames. This allows us to show that many of the works have already used in practice, but not always explicitly, basic elements of the pragmatic frames machinery. However, we also show that pragmatic frames have so far been used in a very restricted way as compared to how they are used in human-human interaction and argue that this has been an obstacle preventing robust natural multi-task learning and teaching in HRI. In particular, we explain that two central features of human pragmatic frames, mostly absent of existing HRI studies, are that (1) social peers use rich repertoires of frames, potentially combined together, to convey and infer multiple kinds of cues; (2) new frames can be learnt continually, building on existing ones, and guiding the interaction toward higher levels of complexity and expressivity. To conclude, we give an outlook on the future research direction describing the relevant key challenges that need to be solved for leveraging pragmatic frames for robot learning and teaching

Robot feedback shapes the tutor's presentation. How a robot's online gaze strategies lead to micro-adaptation of the human's conduct

Pitsch K, Vollmer A-L, Muehlig M. Robot feedback shapes the tutor's presentation. How a robot's online gaze strategies lead to micro-adaptation of the human's conduct. Interaction Studies. 2013;14(2):268-296.The paper investigates the effects of a humanoid robot's online feedback during a tutoring situation in which a human demonstrates how to make a frog jump across a table. Motivated by micro-analytic studies of adult-child-interaction, we investigated whether tutors react to a robot's gaze strategies while they are presenting an action. And if so, how they would adapt to them. Analysis reveals that tutors adjust typical "motionese" parameters (pauses, speed, and height of motion). We argue that a robot - when using adequate online feedback strategies - has at its disposal an important resource with which it could pro-actively shape the tutor's presentation and help generate the input from which it would benefit most. These results advance our understanding of robotic "Social Learning" in that they suggest to consider human and robot as one interactional learning system

Interactive Robot Task Learning: Human Teaching Proficiency with Different Feedback Approaches

The deployment of versatile robot systems in diverse environments requires intuitive approaches for humans to flexibly teach them new skills. In our present work, we investigate different user feedback types to teach a real robot a new movement skill. We compare feedback as star ratings on an absolute scale for single roll-outs versus preference-based feedback for pairwise comparisons with respective optimization algorithms (i.e., a variation of co-variance matrix adaptation -evolution strategy (CMA-ES) and random optimization) to teach the robot the game of skill cup-and-ball. In an experimental investigation with users, we investigated the influence of the feedback type on the user experience of interacting with the different interfaces and the performance of the learning systems. While there is no significant difference for the subjective user experience between the conditions, there is a significant difference in learning performance. The preference-based system learned the task quicker, but this did not influence the users’ evaluation of it. In a follow-up study, we confirmed that the difference in learning performance indeed can be attributed to the human users’ performance

Robots show us how to teach them: Feedback from robots shapes tutoring behavior during action learning

Vollmer A-L, Mühlig M, Steil JJ, et al. Robots show us how to teach them: Feedback from robots shapes tutoring behavior during action learning. PLoS ONE. 2014;9(3): e91349.Robot learning by imitation requires the detection of a tutor's action demonstration and its relevant parts. Current approaches implicitly assume a unidirectional transfer of knowledge from tutor to learner. The presented work challenges this predominant assumption based on an extensive user study with an autonomously interacting robot. We show that by providing feedback, a robot learner influences the human tutor's movement demonstrations in the process of action learning. We argue that the robot's feedback strongly shapes how tutors signal what is relevant to an action and thus advocate a paradigm shift in robot action learning research toward truly interactive systems learning in and benefiting from interaction

An Alternative to Mapping a Word onto a Concept in Language Acquisition: Pragmatic Frames

Rohlfing K, Wrede B, Vollmer A-L, Oudeyer P-Y. An Alternative to Mapping a Word onto a Concept in Language Acquisition: Pragmatic Frames. FRONTIERS IN PSYCHOLOGY. 2016;7: 470.The classic mapping metaphor posits that children learn a word by mapping it onto a concept of an object or event. However, we believe that a mapping metaphor cannot account for word learning, because even though children focus attention on objects, they do not necessarily remember the connection between the word and the referent unless it is framed pragmatically, that is, within a task. Our theoretical paper proposes an alternative mechanism for word learning. Our main premise is that word learning occurs as children accomplish a goal in cooperation with a partner. We follow Bruner's (1983) idea and further specify pragmatic frames as the learning units that drive language acquisition and cognitive development. These units consist of a sequence of actions and verbal behaviors that are co-constructed with a partner to achieve a joint goal. We elaborate on this alternative, offer some initial parametrizations of the concept, and embed it in current language learning approaches

Tutoring in adult-child-interaction: On the loop of the tutor's action modification and the recipient's gaze

Pitsch K, Vollmer A-L, Rohlfing K, Fritsch J, Wrede B. Tutoring in adult-child-interaction: On the loop of the tutor's action modification and the recipient's gaze. Interaction Studies. 2014;15(1):55-98.Research of tutoring in parent-infant interaction has shown that tutors - when presenting some action - modify both their verbal and manual performance for the learner (‘motherese’, ‘motionese’). Investigating the sources and effects of the tutors’ action modifications, we suggest an interactional account of ‘motionese’. Using video-data from a semi-experimental study in which parents taught their 8 to 11 month old infants how to nest a set of differently sized cups, we found that the tutors’ action modifications (in particular: high arches) functioned as an orienting device to guide the infant’s visual attention (gaze). Action modification and the recipient’s gaze can be seen to have a reciprocal sequential relationship and to constitute a constant loop of mutual adjustments. Implications are discussed for developmental research and for robotic ‘Social Learning’. We argue that a robot system could use on-line feedback strategies (e.g. gaze) to pro-actively shape a tutor’s action presentation as it emerges

Studying the Co-Construction of Interaction Protocols in Collaborative Tasks with Humans

International audienceIn interaction, humans align and effortlessly create common ground in communication, allowing efficient collabora- tion in widely diverse contexts. Robots are still far away from being able to adapt in such a flexible manner with non-expert humans to complete collaborative tasks. Challenges include the capability to understand unknown feedback or guidance signals, to make sense of what they refer to depending on their timing and context, and to agree on how to organize the interaction into roles and turns. As a first step in approaching this issue, we investigate here the processes used by humans to negotiate a protocol of interaction when they do not already share one. We introduce a new experimental setup, where two humans have to collaborate to solve a task. The channels of communication they can use are constrained and force them to invent and agree on a shared interaction protocol in order to solve the task. These constraints allow us to analyze how a communication protocol is progressively established through the interplay and history of individual actions. We report preliminary results obtained from a pilot study, and discuss how the understanding of strategies used by humans could be useful to achieve more flexible HRI

Studying the Co-Construction of Interaction Protocols in Collaborative Tasks with Humans

International audienceIn interaction, humans align and effortlessly create common ground in communication, allowing efficient collabora- tion in widely diverse contexts. Robots are still far away from being able to adapt in such a flexible manner with non-expert humans to complete collaborative tasks. Challenges include the capability to understand unknown feedback or guidance signals, to make sense of what they refer to depending on their timing and context, and to agree on how to organize the interaction into roles and turns. As a first step in approaching this issue, we investigate here the processes used by humans to negotiate a protocol of interaction when they do not already share one. We introduce a new experimental setup, where two humans have to collaborate to solve a task. The channels of communication they can use are constrained and force them to invent and agree on a shared interaction protocol in order to solve the task. These constraints allow us to analyze how a communication protocol is progressively established through the interplay and history of individual actions. We report preliminary results obtained from a pilot study, and discuss how the understanding of strategies used by humans could be useful to achieve more flexible HRI