Online backchannel synthesis evaluation with the switching Wizard of Oz

In this paper, we evaluate a backchannel synthesis algorithm in an online conversation between a human speaker and a virtual listener. We adopt the Switching Wizard of Oz (SWOZ) approach to assess behavior synthesis algorithms online. A human speaker watches a virtual listener that is either controlled by a human listener or by an algorithm. The source switches at random intervals. Speakers indicate when they feel they are no longer talking to a human listener. Analysis of these responses reveals patterns of inappropriate behavior in terms of quantity and timing of backchannels

Online behavior evaluation with the switching wizard of Oz

Advances in animation and sensor technology allow us to engage in face-to-face conversations with virtual agents [1]. One major challenge is to generate the virtual agent’s appropriate, human-like behavior contingent with that of the human conversational partner. Models of (nonverbal) behavior are pre-dominantly learned from corpora of dialogs between human subjects [2], or based on simple observations from literature (e.g. [3,4,5,6]

The effect of multiple modalities on the perception of a listening agent

Listening agents are IVAs that display attentive listening behavior to a human speaker. The research into listening agents has mainly focused on (1) automatically timing listener responses; and (2) investigating the perceptual quality of listening behavior. Both issues have predominantly been addressed in an offline fashion, e.g. based on controlled animations that were rated by human observers. This allows for the systematic investigation of variables such as the quantity, type and timing of listening behaviors. However, there is a trade-off between the control and the realism of the stimuli. The display of head movement and facial expressions makes the animated listening behavior more realistic but hinders the investigation of specific behavior such as the timing of a backchannel. To migitate these problems, the Switching Wizard of Oz (SWOZ) framework was introduced in [1]. In online speaker-listener dialogs, a human listener and a behavior synthesis algorithm simultaneously generate backchannel timings. The listening agent is animated based on one of the two sources, which is switched at random time intervals. Speakers are asked to press a button whenever they think the behavior is not human-like. As both human and algorithm have the same limited means of expression, these judgements can solely be based on aspects of the behavior such as the quantity and timing of backchannels. In [1], the listening agent only showed head nods. In the current experiment, we investigate the effect of adding facial expressions. Facial expressions such as smiles and frowns are known to function as backchannels as they can be regarded as a signal of understanding and attention

Conversational collection of grandparents' stories

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2000.Includes bibliographical references (p. 79-81).The act of sharing stories, which often characterizes the interactions between grandparents and grandchildren, exerts a profound influence on both the child listener and the grandparent teller. Unfortunately, opportunities for such sharing are rare for the many extended families who are geographically separated, and the stories go untold. Simple methods such as tape recorders or memory books can be difficult to work with, as they do not provide the powerful feedback that an active and interested listener can give. Computer-based systems have the potential to model this feedback, but in order to be effective at evoking stories, the interface must move away from keyboard and monitor and must be grounded in an understanding of conversation. This work argues that an effective story-eliciting system for grandparents must be based on a model of conversational behavior, must provide a comfortable and story-evoking environment, and that the ideal interface is an autonomous animated character. I present GrandChair, a system which can elicit, record, index, and play back grandparents' stories within an interaction model based on face-to-face conversation, and couched in an environment designed to be comfortable and story-evoking. Tellers sit in a comfortable rocking chair and tell stories with the assistance of a conversational agent on a screen, who takes the form of a child, to help them tailor their stories to a child audience, and prompts them with stories, questions, and video clips from their previous interactions.by Jennifer Smith.S.M

Measuring, analysing and artificially generating head nodding signals in dyadic social interaction

Social interaction involves rich and complex behaviours where verbal and non-verbal signals are exchanged in dynamic patterns. The aim of this thesis is to explore new ways of measuring and analysing interpersonal coordination as it naturally occurs in social interactions. Specifically, we want to understand what different types of head nods mean in different social contexts, how they are used during face-to-face dyadic conversation, and if they relate to memory and learning. Many current methods are limited by time-consuming and low-resolution data, which cannot capture the full richness of a dyadic social interaction. This thesis explores ways to demonstrate how high-resolution data in this area can give new insights into the study of social interaction. Furthermore, we also want to demonstrate the benefit of using virtual reality to artificially generate interpersonal coordination to test our hypotheses about the meaning of head nodding as a communicative signal. The first study aims to capture two patterns of head nodding signals – fast nods and slow nods – and determine what they mean and how they are used across different conversational contexts. We find that fast nodding signals receiving new information and has a different meaning than slow nods. The second study aims to investigate a link between memory and head nodding behaviour. This exploratory study provided initial hints that there might be a relationship, though further analyses were less clear. In the third study, we aim to test if interactive head nodding in virtual agents can be used to measure how much we like the virtual agent, and whether we learn better from virtual agents that we like. We find no causal link between memory performance and interactivity. In the fourth study, we perform a cross-experimental analysis of how the level of interactivity in different contexts (i.e., real, virtual, and video), impacts on memory and find clear differences between them

Modelling a conversational agent (Botocrates) for promoting critical thinking and argumentation skills

Students in higher education institutions are often advised to think critically, yet without being guided to do so. The study investigated the use of a conversational agent (Botocrates) for supporting critical thinking and academic argumentation skills. The overarching research questions were: can a conversational agent support critical thinking and academic argumentation skills? If so, how? The study was carried out in two stages: modelling and evaluating Botocrates' prototype. The prototype was a Wizard-of-Oz system where a human plays Botocrates' role by following a set of instructions and knowledge-base to guide generation of responses. Both stages were conducted at the School of Education at the University of Leeds. In the first stage, the study analysed 13 logs of online seminars in order to define the tasks and dialogue strategies needed to be performed by Botocrates. The study identified two main tasks of Botocrates: providing answers to students' enquiries and engaging students in the argumentation process. Botocrates’ dialogue strategies and contents were built to achieve these two tasks. The novel theoretical framework of the ‘challenge to explain’ process and the notion of the ‘constructive expansion of exchange structure’ were produced during this stage and incorporated into Botocrates’ prototype. The aim of the ‘challenge to explain’ process is to engage users in repeated and constant cycles of reflective thinking processes. The ‘constructive expansion of exchange structure’ is the practical application of the ‘challenge to explain’ process. In the second stage, the study used the Wizard-of-Oz (WOZ) experiments and interviews to evaluate Botocrates’ prototype. 7 students participated in the evaluation stage and each participant was immediately interviewed after chatting with Botocrates. The analysis of the data gathered from the WOZ and interviews showed encouraging results in terms of students’ engagement in the process of argumentation. As a result of the role of ‘critic’ played by Botocrates during the interactions, users actively and positively adopted the roles of explainer, clarifier, and evaluator. However, the results also showed negative experiences that occurred to users during the interaction. Improving Botocrates’ performance and training users could decrease users’ unsuccessful and negative experiences. The study identified the critical success and failure factors related to achieving the tasks of Botocrates