An End-to-End Conversational Style Matching Agent

We present an end-to-end voice-based conversational agent that is able to engage in naturalistic multi-turn dialogue and align with the interlocutor's conversational style. The system uses a series of deep neural network components for speech recognition, dialogue generation, prosodic analysis and speech synthesis to generate language and prosodic expression with qualities that match those of the user. We conducted a user study (N=30) in which participants talked with the agent for 15 to 20 minutes, resulting in over 8 hours of natural interaction data. Users with high consideration conversational styles reported the agent to be more trustworthy when it matched their conversational style. Whereas, users with high involvement conversational styles were indifferent. Finally, we provide design guidelines for multi-turn dialogue interactions using conversational style adaptation

Towards a socially adaptive digital playground

We are working towards a socially adaptive digital playground for children. To this end, we are looking into nonverbal synchrony and other social signals as a measure of social behaviour and into ways to alter game dynamics to trigger and inhibit certain social behaviours. Our first results indicate that we can indeed influence social behaviours in a digital playground by changing game dynamics. Furthermore, our first results show that we will be able to sense some of these social behaviours using only computer vision techniques. I propose an iterative method for working towards a socially adaptive digital playground

Understanding Engagement within the Context of a Safety Critical Game

One of the most frequent arguments for deploying serious games is that they provide an engaging format for student learning. However, engagement is often equated with enjoyment, which may not be the most relevant conceptualization in safety-critical settings, such as law enforcement and healthcare. In these contexts, the term ‘serious’ does not only relate to the non-entertainment purpose of the game but also the environment simulated by the game. In addition, a lack of engagement in a safety critical training setting can have serious ethical implications, leading to significant real-world impacts. However, evaluations of safety-critical games (SCGs) rarely provide an in-depth consideration of player experience. Thus, in relation to simulation game-based training, we are left without a clear understanding of what sort of experience players are having, what factors influence their engagement and how their engagement relates to learning. In order to address these issues, this paper reports on the mixed-method evaluation of a SCG that was developed to support police training. The findings indicate that engagement is supported by the experience situational relevance, due to the player’s experience of real-world authenticity, targeted feedback mechanisms and learning challenges