Designing Sound for Social Robots: Advancing Professional Practice through Design Principles

Sound is one of the core modalities social robots can use to communicate with the humans around them in rich, engaging, and effective ways. While a robot's auditory communication happens predominantly through speech, a growing body of work demonstrates the various ways non-verbal robot sound can affect humans, and researchers have begun to formulate design recommendations that encourage using the medium to its full potential. However, formal strategies for successful robot sound design have so far not emerged, current frameworks and principles are largely untested and no effort has been made to survey creative robot sound design practice. In this dissertation, I combine creative practice, expert interviews, and human-robot interaction studies to advance our understanding of how designers can best ideate, create, and implement robot sound. In a first step, I map out a design space that combines established sound design frameworks with insights from interviews with robot sound design experts. I then systematically traverse this space across three robot sound design explorations, investigating (i) the effect of artificial movement sound on how robots are perceived, (ii) the benefits of applying compositional theory to robot sound design, and (iii) the role and potential of spatially distributed robot sound. Finally, I implement the designs from prior chapters into humanoid robot Diamandini, and deploy it as a case study. Based on a synthesis of the data collection and design practice conducted across the thesis, I argue that the creation of robot sound is best guided by four design perspectives: fiction (sound as a means to convey a narrative), composition (sound as its own separate listening experience), plasticity (sound as something that can vary and adapt over time), and space (spatial distribution of sound as a separate communication channel). The conclusion of the thesis presents these four perspectives and proposes eleven design principles across them which are supported by detailed examples. This work contributes an extensive body of design principles, process models, and techniques providing researchers and designers with new tools to enrich the way robots communicate with humans

Human Machine Interfaces for Teleoperators and Virtual Environments

In Mar. 1990, a meeting organized around the general theme of teleoperation research into virtual environment display technology was conducted. This is a collection of conference-related fragments that will give a glimpse of the potential of the following fields and how they interplay: sensorimotor performance; human-machine interfaces; teleoperation; virtual environments; performance measurement and evaluation methods; and design principles and predictive models

Improving the Speech Intelligibility By Cochlear Implant Users

In this thesis, we focus on improving the intelligibility of speech for cochlear implants (CI) users. As an auditory prosthetic device, CI can restore hearing sensations for most patients with profound hearing loss in both ears in a quiet background. However, CI users still have serious problems in understanding speech in noisy and reverberant environments. Also, bandwidth limitation, missing temporal fine structures, and reduced spectral resolution due to a limited number of electrodes are other factors that raise the difficulty of hearing in noisy conditions for CI users, regardless of the type of noise. To mitigate these difficulties for CI listener, we investigate several contributing factors such as the effects of low harmonics on tone identification in natural and vocoded speech, the contribution of matched envelope dynamic range to the binaural benefits and contribution of low-frequency harmonics to tone identification in quiet and six-talker babble background. These results revealed several promising methods for improving speech intelligibility for CI patients. In addition, we investigate the benefits of voice conversion in improving speech intelligibility for CI users, which was motivated by an earlier study showing that familiarity with a talker’s voice can improve understanding of the conversation. Research has shown that when adults are familiar with someone’s voice, they can more accurately – and even more quickly – process and understand what the person is saying. This theory identified as the “familiar talker advantage” was our motivation to examine its effect on CI patients using voice conversion technique. In the present research, we propose a new method based on multi-channel voice conversion to improve the intelligibility of transformed speeches for CI patients

Brain Tumour Classification Using Deep Features from Structural MRI

C

Workshops of the Sixth International Brain–Computer Interface Meeting: brain–computer interfaces past, present, and future

Brain–computer interfaces (BCI) (also referred to as brain–machine interfaces; BMI) are, by definition, an interface between the human brain and a technological application. Brain activity for interpretation by the BCI can be acquired with either invasive or non-invasive methods. The key point is that the signals that are interpreted come directly from the brain, bypassing sensorimotor output channels that may or may not have impaired function. This paper provides a concise glimpse of the breadth of BCI research and development topics covered by the workshops of the 6th International Brain–Computer Interface Meeting

Object-based radio : effects on production and audience experience

This thesis analyses the benefits of using object-based audio as a production and delivery format in order to enable new audience experiences. This is achieved though a series of case studies, each focusing on a different user experience enabled by the use of object-based audio. Each study considers the impact of using object-based audio on the creative process, production workflow and audience experience.The first study analyses the audience’s use of the ability to personalise the mix of a live football match. It demonstrates that there was not a single audio mix favoured by all, and the ability to change the mix was valued by the audience. While listeners did adjust the mix initially, they tended to leave it at that setting and did not interact much once they made their initial selection. While there were three favoured mixes, over 50% of listeners did not choose one of these three mixes, indicating that only offering three options would not satisfy everyone.Modes of listening model the ways listeners deconstruct complex sound scenes into foreground and background categories ascribing different salience to foreground and background sounds. The second study uses this model to inform a series of card sorting exercises which result in similar foreground and background categories. However, rather than being unimportant, background sounds were present to convey ancillary information or to affect emotional responses and foreground sounds to expose plot or story events. This study demonstrated that this grouping was a meaningful categorisation for broadcast sound and evaluated how beneficial allowing different foreground and background audio mixes would be for audiences. It contains analysis of audio objects in the context of foreground and background sounds based on the opinions of the content creators. It also includes subjective testing of audience preferences for different mixes of foreground verses background audio levels across five different genres and four different loudspeaker layouts. It shows that there is no clustering of listeners based on their preference of foreground vs background balances. It also shows that there is significant variation of foreground and background balance preference between loudspeaker layouts.The final study goes beyond tailoring audio levels, balances and loudspeaker layouts and analyses the benefit to audiences of being able to adapt the story of a drama in order to set it in a location that is familiar to the listener. It shows that being able to set a radio drama in the location where the listening is taking place improves audience’s enjoyment of the programme. 75% of listeners who experienced the tailored version of the drama reported liking the story, compared with 65% of listeners who experienced a non-tailored version.The three studies also analyse the impact of object-based content creation on production workflows by documenting the challenges faced and discussing possible solutions. For example, providing writers with constraints when they are designing dynamic content and allowing sound designers time to develop trust in the technology when mixing content for multiple loudspeaker layouts.The original contribution to knowledge is to establish a new listening model applicable to constructed and designed sound experiences based on functional analysis of audio objects. This work also establishes, for the first time, a framework for the definition of an audio object based on the creator’s intended range of audience experiences. In addition the thesis also provides insights into how audiences interact with object-based content experiences and insights about audience attitudes towards using personal data to personalise object-based content experiences. Each study addresses the potential advantages of delivering object-based audio, assess any impact on the quality of the audience’s experience and analyses the challenges faced by production in the creation of these new experiences