Exploring miscommunication and collaborative behaviour in human-robot interaction

This paper presents the first step in designing a speech-enabled robot that is capable of natural management of miscommunication. It describes the methods and results of two WOz studies, in which dyads of naïve participants interacted in a collaborative task. The first WOz study explored human miscommunication management. The second study investigated how shared visual space and monitoring shape the processes of feedback and communication in task-oriented interactions. The results provide insights for the development of human-inspired and robust natural language interfaces in robots

17 ways to say yes:Toward nuanced tone of voice in AAC and speech technology

People with complex communication needs who use speech-generating devices have very little expressive control over their tone of voice. Despite its importance in human interaction, the issue of tone of voice remains all but absent from AAC research and development however. In this paper, we describe three interdisciplinary projects, past, present and future: The critical design collection Six Speaking Chairs has provoked deeper discussion and inspired a social model of tone of voice; the speculative concept Speech Hedge illustrates challenges and opportunities in designing more expressive user interfaces; the pilot project Tonetable could enable participatory research and seed a research network around tone of voice. We speculate that more radical interactions might expand frontiers of AAC and disrupt speech technology as a whole

Designing and Implementing Embodied Agents: Learning from Experience

In this paper, we provide an overview of part of our experience in designing and implementing some of the embodied agents and talking faces that we have used for our research into human computer interaction. We focus on the techniques that were used and evaluate this with respect to the purpose that the agents and faces were to serve and the costs involved in producing and maintaining the software. We discuss the function of this research and development in relation to the educational programme of our graduate students

Designing Women: Essentializing Femininity in AI Linguistics

Since the eighties, feminists have considered technology a force capable of subverting sexism because of technology’s ability to produce unbiased logic. Most famously, Donna Haraway’s “A Cyborg Manifesto” posits that the cyborg has the inherent capability to transcend gender because of its removal from social construct and lack of loyalty to the natural world. But while humanoids and artificial intelligence have been imagined as inherently subversive to gender, current artificial intelligence perpetuates gender divides in labor and language as their programmers imbue them with traits considered “feminine.” A majority of 21st century AI and humanoids are programmed to fit female stereotypes as they fulfill emotional labor and perform pink-collar tasks, whether through roles as therapists, query-fillers, or companions. This paper examines four specific chat-based AI --ELIZA, XiaoIce, Sophia, and Erica-- and examines how their feminine linguistic patterns are used to maintain the illusion of emotional understanding in regards to the tasks that they perform. Overall, chat-based AI fails to subvert gender roles, as feminine AI are relegated to the realm of emotional intelligence and labor

CreaTable Content and Tangible Interaction in Aphasia

Multimedia digital content (combining pictures, text and music) is ubiquitous. The process of creating such content using existing tools typically requires complex, language-laden interactions which pose a challenge for users with aphasia (a language impairment following brain injury). Tangible interactions offer a potential means to address this challenge, however, there has been little work exploring their potential for this purpose. In this paper, we present CreaTable – a platform that enables us to explore tangible interaction as a means of supporting digital content creation for people with aphasia. We report details of the co-design of CreaTable and findings from a digital creativity workshop. Workshop findings indicated that CreaTable enabled people with aphasia to create something they would not otherwise have been able to. We report how users’ aphasia profiles affected their experience, describe tensions in collaborative content creation and provide insight into more accessible content creation using tangibles

Modelling and analysing user views of telecommunications services

User views of calls are modelled by behaviour trees, which are synchronised to form a network of users. High level presentations of the models are given using process algebra and an explicit theory of features, including precedences. These precedences abstractly encapsulate the possible state spaces which result from different combinations of features. The high level presentation supports incremental development of features and testing and experimentation through animation. Interactions which are not detected during the experimentation phase may be found through static analysis of the high level presentation, through dynamic analysis of the under-lying low level transition system, and through verification of temporal properties through model-checking. In each case, interactions are resolved through manipulation of the feature precedences

An Exploratory Study into the Accessibility of a Multi-User Virtual World for Young People with Aphasia

This paper describes an exploratory study into the accessibility of the virtual world Second Life for two young people with aphasia. Aphasia is a communicative disorder most commonly caused by a stroke. It affects both written and spoken language, is frequently accompanied by right-sided paralysis and people with aphasia can experience isolation and social exclusion. Multi-user virtual worlds are a potential source of fun and contact with others, but how accessible are such worlds to those with communication issues? We report an investigation into the accessibility and potential of Second Life for people with aphasia. This was accomplished through a critique and an empirical study involving two young people: Ann was in her mid twenties and Bob in his early thirties. They were selected because both were comfortable with computer technologies before their strokes and each continues to use them, albeit in a more limited capacity. We discuss implications of the results for people with aphasia interacting with multi-user virtual worlds

An End-to-End Conversational Style Matching Agent

We present an end-to-end voice-based conversational agent that is able to engage in naturalistic multi-turn dialogue and align with the interlocutor's conversational style. The system uses a series of deep neural network components for speech recognition, dialogue generation, prosodic analysis and speech synthesis to generate language and prosodic expression with qualities that match those of the user. We conducted a user study (N=30) in which participants talked with the agent for 15 to 20 minutes, resulting in over 8 hours of natural interaction data. Users with high consideration conversational styles reported the agent to be more trustworthy when it matched their conversational style. Whereas, users with high involvement conversational styles were indifferent. Finally, we provide design guidelines for multi-turn dialogue interactions using conversational style adaptation