The Role of Modular Robotics in Mediating Nonverbal Social Exchanges

This paper outlines the use of modular robotics to encourage and facilitate non-verbal communication during therapeutic intervention in dementia care. A set of new socially interactive modular robotic devices called Rolling Pins (RPs) have been designed and developed to assist the therapist in interacting with dementia affected patients. The RPs are semi-transparent plastic tubes capable of measuring their orientation and the speed of their rotation; at a local level they have three types of feedback: RGB light, sound and vibration. The peculiarity of the RPs is that they are able to communicate with each other or with other devices equipped with the same radio communication technology. The RPs are usually used in pairs, as the local feedback of an RP can be set depending not only on its own speed and orientation, but also on the speed and the orientation of the peer RP. The system is not used as a therapeutic tool per se but as facilitator and mediator of social dynamics during normal therapy to counteract social isolation that can result in dementia through the loss of social skills. An experiment is reported showing that using the RPs the patients participated in the activity, coordinating their behaviour with the therapist and imitating the same interaction patterns generated by the therapist

Applying the “human-dog interaction” metaphor in human-robot interaction: a co-design practice engaging healthy retired adults in China

This research adopts a Deweyan pragmatist approach and “research through design” methods to explore the use of human-dog interaction as a model for developing human-robot interaction. This research asks two questions: (1) In what way could the human-dog interaction model inform the design of social robots to meet the needs of older adults? (2) What role could aesthetic, functional and behavioural aspects of the human-dog interaction play in older adults’ interaction with social robots? Driven by the pragmatist approach, this thesis uses the dog-human interaction model as a metaphor in this thesis. The research carried out four studies in two parts. The first part of the practice includes two explorative studies to identify aspects of human-dog interaction that could inform the design of social robots for older adults. Study 1 explores aspects of human-dog interaction that could inform the design of human-robot interaction for retired adults. Study 2 explores a group of healthy retired adults’ attitudes and preferences toward social/assistive robots in China. The findings suggest that, first, the pairing and training process provides a framework for building personalised social robots in terms of form, function, interaction, and stakeholders involved. Second, the cooperative interaction between a human and a guide dog provides insights for building social robots that take on leading roles in interactions. The robot-as-dog metaphor offers a new perspective to rethink the design process of social robots based on the role dog trainer, owner, and the dog plays in human-dog interaction. In the second part of the practice, two more studies are conducted to articulate the usefulness of the designer-as-trainer-metaphor, and the personalisation-astraining-metaphor, using participatory co-designing methods. Engaging both retired adult participants and roboticists as co-designers to investigate further how aesthetic aspects, functional features, and interactive behaviours characterising dog-human interaction could inform how older adults can interact with social robots. Study 3 involved co-designing a robot probe with roboticists and later deploying it in a participant’s home using the Wizard of Oz method. The personalisation-as-training metaphor helps facilitate a critical discussion for the interdisciplinary co-design process. It broadens the design space when addressing the technical limitation of the probe’s camera through reflection-in-action. Study 4 engages the retired adults as co-designers to envision what characteristics they would like robots to have, with attention to the robot’s form, the functions that the robot can perform and how the robot interacts with users. The study applies techniques such as sketching and storyboarding to understand how retired adults make sense of these core elements that are key to developing social/assistive robots for positive ageing. This thesis makes two main contributions to knowledge in human-robot interaction and interaction design research. Firstly, it provides an applied example using the robot-as-dog metaphor as a tool to probe human-robot interactions in a domestic context. Secondly, to show dog-human interaction model is applicable to different levels of abstraction for the co-designing process that involves the roboticists and the end-users. The outcome shows a reflective practice that engages metaphors to facilitate communication across disciplines in the co-design process

The Case for Public Interventions during a Pandemic

Funding Information: This work has been supported by Marie Skłodowska Curie Actions ITN AffecTech (ERC H2020 Project 1059 ID: 722022). Publisher Copyright: © 2022 by the authors.Within the field of movement sensing and sound interaction research, multi-user systems have gradually gained interest as a means to facilitate an expressive non-verbal dialogue. When tied with studies grounded in psychology and choreographic theory, we consider the qualities of interaction that foster an elevated sense of social connectedness, non-contingent to occupying one’s personal space. Upon reflection of the newly adopted social distancing concept, we orchestrate a technological intervention, starting with interpersonal distance and sound at the core of interaction. Materialised as a set of sensory face-masks, a novel wearable system was developed and tested in the context of a live public performance from which we obtain the user’s individual perspectives and correlate this with patterns identified in the recorded data. We identify and discuss traits of the user’s behaviour that were accredited to the system’s influence and construct four fundamental design considerations for physically distanced sound interaction. The study concludes with essential technical reflections, accompanied by an adaptation for a pervasive sensory intervention that is finally deployed in an open public space.publishersversionpublishe

Producing Acoustic-Prosodic Entrainment in a Robotic Learning Companion to Build Learner Rapport

abstract: With advances in automatic speech recognition, spoken dialogue systems are assuming increasingly social roles. There is a growing need for these systems to be socially responsive, capable of building rapport with users. In human-human interactions, rapport is critical to patient-doctor communication, conflict resolution, educational interactions, and social engagement. Rapport between people promotes successful collaboration, motivation, and task success. Dialogue systems which can build rapport with their user may produce similar effects, personalizing interactions to create better outcomes. This dissertation focuses on how dialogue systems can build rapport utilizing acoustic-prosodic entrainment. Acoustic-prosodic entrainment occurs when individuals adapt their acoustic-prosodic features of speech, such as tone of voice or loudness, to one another over the course of a conversation. Correlated with liking and task success, a dialogue system which entrains may enhance rapport. Entrainment, however, is very challenging to model. People entrain on different features in many ways and how to design entrainment to build rapport is unclear. The first goal of this dissertation is to explore how acoustic-prosodic entrainment can be modeled to build rapport. Towards this goal, this work presents a series of studies comparing, evaluating, and iterating on the design of entrainment, motivated and informed by human-human dialogue. These models of entrainment are implemented in the dialogue system of a robotic learning companion. Learning companions are educational agents that engage students socially to increase motivation and facilitate learning. As a learning companion’s ability to be socially responsive increases, so do vital learning outcomes. A second goal of this dissertation is to explore the effects of entrainment on concrete outcomes such as learning in interactions with robotic learning companions. This dissertation results in contributions both technical and theoretical. Technical contributions include a robust and modular dialogue system capable of producing prosodic entrainment and other socially-responsive behavior. One of the first systems of its kind, the results demonstrate that an entraining, social learning companion can positively build rapport and increase learning. This dissertation provides support for exploring phenomena like entrainment to enhance factors such as rapport and learning and provides a platform with which to explore these phenomena in future work.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Design Strategies for Adaptive Social Composition: Collaborative Sound Environments

In order to develop successful collaborative music systems a variety of subtle interactions need to be identified and integrated. Gesture capture, motion tracking, real-time synthesis, environmental parameters and ubiquitous technologies can each be effectively used for developing innovative approaches to instrument design, sound installations, interactive music and generative systems. Current solutions tend to prioritise one or more of these approaches, refining a particular interface technology, software design or compositional approach developed for a specific composition, performer or installation environment. Within this diverse field a group of novel controllers, described as ‘Tangible Interfaces’ have been developed. These are intended for use by novices and in many cases follow a simple model of interaction controlling synthesis parameters through simple user actions. Other approaches offer sophisticated compositional frameworks, but many of these are idiosyncratic and highly personalised. As such they are difficult to engage with and ineffective for groups of novices. The objective of this research is to develop effective design strategies for implementing collaborative sound environments using key terms and vocabulary drawn from the available literature. This is articulated by combining an empathic design process with controlled sound perception and interaction experiments. The identified design strategies have been applied to the development of a new collaborative digital instrument. A range of technical and compositional approaches was considered to define this process, which can be described as Adaptive Social Composition. Dan Livingston