Progressivity for Voice Interface Design

Drawing from Conversation Analysis (CA), we examine how the orientation towards progressivity in talk---keeping things moving---might help us better understand and design for voice interactions. We introduce progressivity by surveying its explication in CA, and then look at how a strong preference for progressivity in conversation works out practically in sequences of voice interaction recorded in people's homes. Following \citeauthor{sti06}'s work on progressivity, we find our data shows: how non-answer responses impede progress; how accounts offered for non-answer responses can lead to recovery; how participants work to receive answers; and how, ultimately, moving the interaction forwards does not necessarily involve a fitted answer, but other kinds of responses as well. We discuss the wider potential of applying progressivity to evaluate and understand voice interactions, and consider what designers of voice experiences might do to design for progressivity. Our contribution is a demonstration of the progressivity principle and its interactional features, which also points towards the need for specific kinds of future developments in speech technology

Emotion and mood blending in embodied artificial agents: expressing affective states in the mini social robot

Robots that are devised for assisting and interacting with humans are becoming fundamental in many applications, including in healthcare, education, and entertainment. For these robots, the capacity to exhibit affective states plays a crucial role in creating emotional bonding with the user. In this work, we present an affective architecture that grounds biological foundations to shape the affective state of the Mini social robot in terms of mood and emotion blending. The affective state depends upon the perception of stimuli in the environment, which influence how the robot behaves and affectively communicates with other peers. According to research in neuroscience, mood typically rules our affective state in the long run, while emotions do it in the short term, although both processes can overlap. Consequently, the model that is presented in this manuscript deals with emotion and mood blending towards expressing the robot's internal state to the users. Thus, the primary novelty of our affective model is the expression of: (i) mood, (ii) punctual emotional reactions to stimuli, and (iii) the decay that mood and emotion undergo with time. The system evaluation explored whether users can correctly perceive the mood and emotions that the robot is expressing. In an online survey, users evaluated the robot's expressions showing different moods and emotions. The results reveal that users could correctly perceive the robot's mood and emotion. However, emotions were more easily recognized, probably because they are more intense affective states and mainly arise as a stimuli reaction. To conclude the manuscript, a case study shows how our model modulates Mini's expressiveness depending on its affective state during a human-robot interaction scenario.The research leading to these results has received funding from the projects Robots sociales para estimulación física, cognitiva y afectiva de mayores (ROSES) RTI2018-096338-B-I00 funded by Agencia Estatal de Investigación (AEI), Ministerio de Ciencia, Innovación y Universidades and RoboCity2030-DIH-CM, Madrid Robotics Digital Innovation Hub, S2018/NMT-4331, funded by "Programas de Actividades I+D en la Comunidad de Madrid" and cofunded by Structural Funds of the EU. Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature

Automatic Context-Driven Inference of Engagement in HMI: A Survey

An integral part of seamless human-human communication is engagement, the process by which two or more participants establish, maintain, and end their perceived connection. Therefore, to develop successful human-centered human-machine interaction applications, automatic engagement inference is one of the tasks required to achieve engaging interactions between humans and machines, and to make machines attuned to their users, hence enhancing user satisfaction and technology acceptance. Several factors contribute to engagement state inference, which include the interaction context and interactants' behaviours and identity. Indeed, engagement is a multi-faceted and multi-modal construct that requires high accuracy in the analysis and interpretation of contextual, verbal and non-verbal cues. Thus, the development of an automated and intelligent system that accomplishes this task has been proven to be challenging so far. This paper presents a comprehensive survey on previous work in engagement inference for human-machine interaction, entailing interdisciplinary definition, engagement components and factors, publicly available datasets, ground truth assessment, and most commonly used features and methods, serving as a guide for the development of future human-machine interaction interfaces with reliable context-aware engagement inference capability. An in-depth review across embodied and disembodied interaction modes, and an emphasis on the interaction context of which engagement perception modules are integrated sets apart the presented survey from existing surveys

Towards long-term social child-robot interaction: using multi-activity switching to engage young users

Social robots have the potential to provide support in a number of practical domains, such as learning and behaviour change. This potential is particularly relevant for children, who have proven receptive to interactions with social robots. To reach learning and therapeutic goals, a number of issues need to be investigated, notably the design of an effective child-robot interaction (cHRI) to ensure the child remains engaged in the relationship and that educational goals are met. Typically, current cHRI research experiments focus on a single type of interaction activity (e.g. a game). However, these can suffer from a lack of adaptation to the child, or from an increasingly repetitive nature of the activity and interaction. In this paper, we motivate and propose a practicable solution to this issue: an adaptive robot able to switch between multiple activities within single interactions. We describe a system that embodies this idea, and present a case study in which diabetic children collaboratively learn with the robot about various aspects of managing their condition. We demonstrate the ability of our system to induce a varied interaction and show the potential of this approach both as an educational tool and as a research method for long-term cHRI

Modeling Human-Robot-Interaction based on generic Interaction Patterns

Peltason J. Modeling Human-Robot-Interaction based on generic Interaction Patterns. Bielefeld: Bielefeld University; 2014

Video Conferencing and Multimodal Expression of Voice: Children's Communication in a Second Language Using Skype

This thesis focuses on how voice is experienced and expressed in a telecollaborative project using Skype to connect two groups of English language learners of primary age across two different countries. Voice is understood as a social semiotic phenomenon which takes as its base the ideas of Bakhtin (1986) and Goffman (1981) and is expanded to include multimodal forms of expression through the work of Kress (2003). This social semiotic notion of voice is synthesised with a framework of mediated action from Vygotsky (1978) and Wertsch (1991). The theoretical view of voice frames a small-scale qualitative study on how voice is expressed materially involving tools such as verbal language, body language, technology, and the spatial and temporal characteristics within which the communication takes place. As this is an area that has not been widely researched, a methodology had to be designed to analyse the video recorded data and a framework based on Scollon and Scollon’s (2003) concept of geosemiotics was developed. This method of analysis investigates how language is materially assembled through interaction with others in the physical world around us. It has been rooted in a social constructivist paradigm to shed light on how multimodal expressions of voice through Skype can support children’s second language use. The study shows that webcam-mediated online communication creates particular sets of conditions which affect the ways children are able to express their voice. Some points of divergence from familiar patterns of communication include how children use different spaces to negotiate different ways of being together, the multimodal ways in which children are able to express their voices and the diverse ways in which interpersonal distances can be represented and manipulated to manage conversations. The implications drawn out in the conclusion should initiate wider discussion in early childhood education and second language learning practice and research concerning the importance of adopting a multimodal perspective on how children express voice to support their communication in video conferencing environments