Ghost-in-the-Machine reveals human social signals for human-robot interaction

© 2015 Loth, Jettka, Giuliani and de Ruiter. We used a new method called "Ghost-in-the-Machine" (GiM) to investigate social interactions with a robotic bartender taking orders for drinks and serving them. Using the GiM paradigm allowed us to identify how human participants recognize the intentions of customers on the basis of the output of the robotic recognizers. Specifically, we measured which recognizer modalities (e.g., speech, the distance to the bar) were relevant at different stages of the interaction. This provided insights into human social behavior necessary for the development of socially competent robots. When initiating the drink-order interaction, the most important recognizers were those based on computer vision. When drink orders were being placed, however, the most important information source was the speech recognition. Interestingly, the participants used only a subset of the available information, focussing only on a few relevant recognizers while ignoring others. This reduced the risk of acting on erroneous sensor data and enabled them to complete service interactions more swiftly than a robot using all available sensor data. We also investigated socially appropriate response strategies. In their responses, the participants preferred to use the same modality as the customer's requests, e.g., they tended to respond verbally to verbal requests. Also, they added redundancy to their responses, for instance by using echo questions. We argue that incorporating the social strategies discovered with the GiM paradigm in multimodal grammars of human-robot interactions improves the robustness and the ease-of-use of these interactions, and therefore provides a smoother user experience

Automatic detection of service initiation signals used in bars

Loth S, Huth K, de Ruiter J. Automatic detection of service initiation signals used in bars. Frontiers in Psychology. 2013;4: 557.Recognizing the intention of others is important in all social interactions, especially in the service domain. Enabling a bartending robot to serve customers is particularly challenging as the system has to recognize the social signals produced by customers and respond appropriately. Detecting whether a customer would like to order is essential for the service encounter to succeed. This detection is particularly challenging in a noisy environment with multiple customers. Thus, a bartending robot has to be able to distinguish between customers intending to order, chatting with friends or just passing by. In order to study which signals customers use to initiate a service interaction in a bar, we recorded real-life customer-staff interactions in several German bars. These recordings were used to generate initial hypotheses about the signals customers produce when bidding for the attention of bar staff. Two experiments using snapshots and short video sequences then tested the validity of these hypothesized candidate signals. The results revealed that bar staff responded to a set of two non-verbal signals: first, customers position themselves directly at the bar counter and, secondly, they look at a member of staff. Both signals were necessary and, when occurring together, sufficient. The participants also showed a strong agreement about when these cues occurred in the videos. Finally, a signal detection analysis revealed that ignoring a potential order is deemed worse than erroneously inviting customers to order. We conclude that (a) these two easily recognizable actions are sufficient for recognizing the intention of customers to initiate a service interaction, but other actions such as gestures and speech were not necessary, and (b) the use of reaction time experiments using natural materials is feasible and provides ecologically valid results

Confidence in uncertainty: Error cost and commitment in early speech hypotheses

© 2018 Loth et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Interactions with artificial agents often lack immediacy because agents respond slower than their users expect. Automatic speech recognisers introduce this delay by analysing a user’s utterance only after it has been completed. Early, uncertain hypotheses of incremental speech recognisers can enable artificial agents to respond more timely. However, these hypotheses may change significantly with each update. Therefore, an already initiated action may turn into an error and invoke error cost. We investigated whether humans would use uncertain hypotheses for planning ahead and/or initiating their response. We designed a Ghost-in-the-Machine study in a bar scenario. A human participant controlled a bartending robot and perceived the scene only through its recognisers. The results showed that participants used uncertain hypotheses for selecting the best matching action. This is comparable to computing the utility of dialogue moves. Participants evaluated the available evidence and the error cost of their actions prior to initiating them. If the error cost was low, the participants initiated their response with only suggestive evidence. Otherwise, they waited for additional, more confident hypotheses if they still had time to do so. If there was time pressure but only little evidence, participants grounded their understanding with echo questions. These findings contribute to a psychologically plausible policy for human-robot interaction that enables artificial agents to respond more timely and socially appropriately under uncertainty

Player agency in interactive narrative: audience, actor & author

The question motivating this review paper is, how can computer-based interactive narrative be used as a constructivist learn- ing activity? The paper proposes that player agency can be used to link interactive narrative to learner agency in constructivist theory, and to classify approaches to interactive narrative. The traditional question driving research in interactive narrative is, ‘how can an in- teractive narrative deal with a high degree of player agency, while maintaining a coherent and well-formed narrative?’ This question derives from an Aristotelian approach to interactive narrative that, as the question shows, is inherently antagonistic to player agency. Within this approach, player agency must be restricted and manip- ulated to maintain the narrative. Two alternative approaches based on Brecht’s Epic Theatre and Boal’s Theatre of the Oppressed are reviewed. If a Boalian approach to interactive narrative is taken the conflict between narrative and player agency dissolves. The question that emerges from this approach is quite different from the traditional question above, and presents a more useful approach to applying in- teractive narrative as a constructivist learning activity

An emotion and memory model for social robots : a long-term interaction

In this thesis, we investigate the role of emotions and memory in social robotic companions. In particular, our aim is to study the effect of an emotion and memory model towards sustaining engagement and promoting learning in a long-term interaction. Our Emotion and Memory model was based on how humans create memory under various emotional events/states. The model enabled the robot to create a memory account of user's emotional events during a long-term child-robot interaction. The robot later adapted its behaviour through employing the developed memory in the following interactions with the users. The model also had an autonomous decision-making mechanism based on reinforcement learning to select behaviour according to the user preference measured through user's engagement and learning during the task. The model was implemented on the NAO robot in two different educational setups. Firstly, to promote user's vocabulary learning and secondly, to inform how to calculate area and perimeter of regular and irregular shapes. We also conducted multiple long-term evaluations of our model with children at the primary schools to verify its impact on their social engagement and learning. Our results showed that the behaviour generated based on our model was able to sustain social engagement. Additionally, it also helped children to improve their learning. Overall, the results highlighted the benefits of incorporating memory during child-Robot Interaction for extended periods of time. It promoted personalisation and reflected towards creating a child-robot social relationship in a long-term interaction

The tertiary education institution of the future towards 2030: scenarios for skills transformation

The research methodology used in this research was comprised of Inayatullah’s Six Pillars of Futures Studies, in which emphasis was placed on scenario planning and the creation of alternative scenarios for the tertiary education institutions in South Africa towards 2030. An environmental scan revealed the drivers of change in the education sector and in the world of work. Deepening of the future of education was done through Causal Layered Analysis (CLA) to facilitate the discerning of issues from various viewpoints in the creation and expansion of transformative stories so as to provide a window into possible futures for skills transformation. The four scenarios for the tertiary education institution of the future, namely “Stairway to Heaven”, “Highway to Hell”, “Bat out of Hell” and “Still Raining” were developed. These scenarios can be used as departure points by tertiary education providers to make strides towards the Global Sustainable Development Education 2030 targets and the attainment of South Africa’s Vision 2030 targets contained in the National Development Plan. Equally important, these scenarios make known what was previously unknown, exploring the possible and impossible, and encouraging new, innovative thinking for decision-makers. The “Stairway to Heaven” scenario supplies a future in which all stakeholders approve of and embrace the mandate of providing relevant skills and job readiness in a fast-changing world, and the benefits are maximised for all involved through co-creation. It is a scenario where industry, tertiary institutions and society have decided that the purpose of education should be lifelong learning for a viable, productive and sustainable world. The desired future of tertiary education is set against a backdrop of public and private sector collaboration, with the aim of turning the nation into an excellent hub for skills transformation. Furthermore, the scenario provides some insight on the vital measures required to embrace the innovation and the appropriate pedagogy. This research was motivated by the need to shine a light on the 21st century learner, rapidly obsoleting skills, no-collar worker, skills of the future, learning futures, and possible predictions about what new jobs may come into existence so that educationists can better prepare for the future. This research offers solutions on how institutions can prepare students for future jobs, especially considering the rapid changes in jobs and the unprecedented demise of certain jobs. The research closes a research gap through creating scenarios that offer various stakeholders in the tertiary education sector different insights and analysis into a number of interpretations of the potential paths that they can follow. The scenario application culminated in the formulation and creation of a “future vision of the tertiary education institution in South Africa towards 2030”, delivering a platform for skills transformation that will deliver adaptable workers, and sustainable and inclusive progress for all South Africans. To bring transformation into the present and design the future that embraces skills transformation, it is invaluable to interrogate the roles and choices that stakeholders of the educational sector make in determining the preferred future. The approach of this research makes it clear that, as the new world of work transpires, policymakers, students, labour, educational leaders, captains of industry and workers must proactively manage the workforce transitions. The focal issue is to discover the appropriate tools that will establish the confidence necessary to create the preferred future for skills transformation in tertiary institutions. This research has laid a platform for co-creation with various stakeholders in an effort to visualise a tertiary institution that contributes to skills development. The vision must accept that the South African jobs and skills historical profile is different from that of industrialised countries. Alternatively, the nation should respond to the double-barrelled challenge of participating in a high skills’ competitive environment on a global scale, as well as a local context that creates low-wage, blue-collar jobs to absorb the large numbers who are unemployed. The challenge is even greater for South Africa, because the economy – if highly service-oriented, with a big informal sector and a quality postgraduate education offering – is supported by a basic education system that is not producing enough critical thinkers who are equipped for university and work life. Thus, the system requires a double transformation to ensure student-centredness and meet the needs of a future worker

Rights and virtues: the groundwork of a virtue-based theory of rights

The dissertation investigates whether virtue ethics can provide the normative ground for the justification of rights. Most justificatory accounts of rights consist in different explanations of the function(s) of rights. On the view I will defend, rights have a plurality of functions and one of the main functions of rights is to make the right-holder more virtuous. The idea that the possession of rights leads to the development of virtues, called the function of virtue acquisition, is the core of a virtue-based justification of rights elaborated in this dissertation. Based on Leif Wenar’s Kind-Desire Theory of rights and Nancy E. Snow’s Minimal Virtue of the Folk, I argue that there are two general types of rights, enabling rights and protective rights, and that both types are essential to the development of virtues. Enabling rights have the function of virtue acquisition because they allow the right-holder to strive for approximation of ideal role models while performing duties and other tasks. Such models are called heuristic models of virtues for they define what virtues are and how these virtues should be expressed in action. Some protective rights, namely promissory rights and the power-right to promise, behave like enabling rights when it comes to their ability to develop virtues. Other protective rights, especially children’s rights, make right-holders more virtues via protecting their essential needs. The dissertation succeeds in elaborating the virtue-based justification of many rights, hence it suggests that people hold and should have rights because rights make them better people