Using movement sonification to alter body perception and promote physical activity in physically inactive people

Mención Internacional en el título de doctorWorldwide, one out of four adults are not physically active enough. Supporting people to be physically active through technology remains thus an important challenge in the field of Human-Computer Interaction (HCI). Some technologies have tried to tackle this challenge of increasing physical activity (PA) by using sensing devices for monitoring the amount and quality of PA and providing motivational feedback on it. However, such technologies provide very limited support to physically inactive users: while users are aware of their physical inactivity level, they are frequently incapable of acting on these problems by themselves. Among the reasons for it are negative perceptions about one’s body (e.g., feelings of body tiredness or weakness in self-esteem) which may act as psychological barriers to PA. This research project aims to address this limitation by employing an approach that, through movement sonification (i.e., real-time auditory feedback on body movement), exploits bottom-up multisensory mechanisms related to BPs to ultimately support PA. This thesis presents the design, development, and evaluation of SoniShoes and SoniBand, two wearable technological devices with a gesture-sound palette that allows for a range of body movement sonifications aimed to alter BPs. These prototypes aim at changing BPs, and in turn emotional state and movement behavior, to address psychological barriers related to the perception of one’s body, and ultimately impact positively on people’s adherence to PA. First, this work proposes to organize knowledge through a taxonomy of the barriers to PA related to body perception (BP), which follows a process of four steps to inform the design of the movement-sound palette: (1) Identification, (2) Extraction and clustering of attributes, (3) Definition of instructions or considerations, and (4) Strategies. The first two steps allowed the identification and grouping of barriers to PA that are related to BPs, with inputs from a literature review, a survey, and a focus group with HCI experts. The third and fourth steps allowed defining the body features and dimensions to act upon, to finally propose movement sonification strategies that have the potential to tackle the barriers. Second, several movement-sound mappings, based on metaphors, are presented. Movements were selected from exercises included in guidelines for becoming more physically active (e.g., walking). The mappings of these movements into sounds were implemented in SoniShoes and SoniBand prototypes. They were evaluated through an iterative process, starting with an exploratory study that tested for the first time the potential of the proposed mappings to change BPs. In this first study, participants were asked to think aloud about their experiences using the first prototype of SoniShoes (from MagicShoes project), by describing their body sensations and sound characteristics during the exercise. Results suggested the potential of movement sonification to alter BP through movement sonification and informed the design of the subsequent studies and prototypes. This exploratory study was followed by quantitative and qualitative studies aimed to understand how to design movement sonifications and wearable devices integrating them to facilitate PA by tackling barriers related to BP. The quantitative studies were controlled laboratory studies, in which different versions of SoniShoes and SoniBand prototypes were evaluated, and which results led to further iterations of the prototypes. The results of these quantitative evaluations revealed movement-sound mappings that can lead to changes in feelings about the body (e.g., feeling lighter or less tired), feelings about the movement (e.g., having more movement control over the movement), and emotional feelings (e.g., having more comfort, motivation to complete the exercise, or feeling happier) during PA. Results also showed effects of sound on movement behavior, such as effects in movement deceleration/acceleration and stance time, and proprioceptive awareness. Furthermore, two qualitative studies were carried out, which involved using the SoniBand prototype for several days and in two different contexts of use, laboratory and home. The aim of these studies was two-fold. First, elucidating the effects that particular metaphorical sonifications’ qualities and characteristics have on people’s perception of their own body and their PA. Second, understanding how the observed effects may be specific to physically inactive (vs. active) populations. The results revealed specific connections between properties of the movement sonifications (e.g., gradual or frequency changes) on the one hand, and particular body feelings (e.g., feeling strong) and aspects of PA (e.g., repetitions) on the other hand, but effects seem to vary according to the PA-level of the populations. Finally, the findings, contributions, and principles for the design of movement sonifications and wearable technology to promote PA through acting upon BP are discussed, finishing by considering implications for potential interventions and applications supporting PA, as well as opportunities opened for future research.En todo el mundo, uno de cada cuatro adultos no es lo suficientemente activo físicamente. Por ello, ayudar a las personas a ser físicamente activas a través de la tecnología sigue siendo un reto importante en el campo de “Human-Computer Interaction” (HCI). Algunas tecnologías han tratado de abordar el reto de aumentar la actividad física (PA) mediante el uso de dispositivos de detección para controlar la cantidad y la calidad de la PA y proporcionar retroalimentación motivacional al respecto. Sin embargo, estas tecnologías proporcionan una ayuda muy limitada a los usuarios físicamente inactivos: aunque los usuarios son conscientes de su nivel de inactividad física, a menudo son incapaces de actuar por sí mismos sobre estos problemas. Entre las razones están las percepciones negativas sobre el propio cuerpo (por ejemplo, la sensación de cansancio corporal o el no sentirse capaces) que pueden actuar como barreras psicológicas para la PA. Este proyecto de investigación pretende abordar esta limitación empleando un enfoque que, a través de la sonificación del movimiento (es decir, la retroalimentación auditiva en tiempo real sobre el movimiento del cuerpo), explota los mecanismos “bottom-up” multisensoriales relacionados con las percepciones del cuerpo (BPs) para apoyar la PA. Esta tesis presenta el diseño, el desarrollo y la evaluación de “SoniShoes” y “SoniBand”, dos dispositivos tecnológicos vestibles con una paleta de gestos y sonidos que permiten una serie de sonificaciones del movimiento corporal destinadas a modificar las BPs. Estos prototipos tienen como objetivo cambiar las BPs, y a su vez el estado emocional y el comportamiento de movimiento, para abordar las barreras psicológicas relacionadas con la BP, y en última instancia impactar positivamente en la adherencia de las personas a la PA. En primer lugar, este trabajo propone organizar el conocimiento a través de una taxonomía de las barreras a la PA relacionadas con la BP, que sigue un proceso de cuatro pasos para informar el diseño de la paleta de movimiento-sonido: (1) Identificación, (2) Extracción y agrupación de atributos, (3) Definición de instrucciones o consideraciones, y (4) Estrategias. Los dos primeros pasos permitieron identificar y agrupar las barreras a la PA relacionadas con los BP, con aportaciones de una revisión bibliográfica, una encuesta y un grupo de discusión con expertos en HCI. El tercero y cuarto paso permitió definir las características y dimensiones corporales sobre las que actuar, para finalmente proponer estrategias de sonificación del movimiento que tienen el potencial de abordar las barreras. En segundo lugar, se presentan varios mapeos de movimiento-sonido, basados en metáforas. Los movimientos se seleccionaron a partir de ejercicios incluidos en las guías para ser más activos físicamente (por ejemplo, caminar). Los mapeos de estos movimientos en sonidos se implementaron en los prototipos “SoniShoes” y “SoniBand”. Se evaluaron a través de un proceso iterativo, comenzando con un estudio exploratorio que probó por primera vez el potencial de los mapeos propuestos para cambiar los BP. En este primer estudio, se pidió a los participantes que pensaran en voz alta sobre sus experiencias utilizando el primer prototipo de “SoniShoes” (llamado “MagicShoes”), describiendo sus sensaciones corporales y las características del sonido durante el ejercicio. Los resultados mostraron el potencial de la sonificación del movimiento para alterar la BP a través de la sonificación del movimiento e informaron el diseño de los estudios y prototipos posteriores. A este estudio exploratorio le siguieron estudios cuantitativos y cualitativos destinados a comprender cómo diseñar sonificaciones del movimiento y dispositivos vestibles que las integren para facilitar la PA abordando las barreras relacionadas con la BP. Los estudios cuantitativos fueron estudios de laboratorio controlados, en los que se evaluaron diferentes versiones de los prototipos “SoniShoes” y “SoniBand”, y cuyos resultados condujeron a nuevas iteraciones de los prototipos. Los resultados de estas evaluaciones cuantitativas mostraron que existen mapeos de movimiento-sonido que pueden provocar cambios en las sensaciones sobre el cuerpo (por ejemplo, sentirse más ligero o menos cansado), en las sensaciones sobre el movimiento (por ejemplo, tener más control sobre el movimiento) y en las sensaciones emocionales (por ejemplo, tener más comodidad, motivación para completar el ejercicio o sentirse más feliz) durante la PA. Los resultados también mostraron los efectos del sonido en el comportamiento del movimiento, como los efectos en la desaceleración/aceleración del movimiento y el tiempo de postura, y la conciencia propioceptiva. Además, se llevaron a cabo dos estudios cualitativos, en los que se utilizó el prototipo “SoniBand” durante varios días y en dos contextos de uso diferentes, el laboratorio y el hogar. El objetivo de estos estudios era doble. En primer lugar, dilucidar los efectos que determinadas cualidades y características de las sonificaciones con metáforas tienen en la percepción que las personas tienen de su propio cuerpo y de su PA. En segundo lugar, comprender cómo los efectos observados pueden ser específicos de las poblaciones físicamente inactivas (vs. las activas). Los resultados revelaron conexiones específicas entre las propiedades de las sonificaciones de movimiento (por ejemplo, los cambios graduales o de frecuencia) por un lado, y las sensaciones corporales particulares (por ejemplo, sentirse fuerte) y los aspectos de la PA (por ejemplo, las repeticiones) por otro lado, pero los efectos parecen variar según el nivel de PA de las poblaciones. Por último, se discuten los hallazgos, las contribuciones y las guías de diseño de sonificación de movimiento y tecnología vestible para promover la PA a través de la actuación sobre la BP, para finalmente considerar las implicaciones para las posibles intervenciones y aplicaciones de apoyo a la PA, así como las oportunidades abiertas para futuras investigaciones.I owe thanks to “MAGIC SHOES” (PSI2016-79004-R and BES-2017-080471) and “CROSS-COLAB” (PGC2018-101884-B-I00) projects that funded my research. Thanks to “MAGIC OUTFIT” (PID2019-105579RB-I00) for letting me be part of the team and project.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: Paloma Martínez Fernández.- Secretario: Domna Banakou.- Vocal: Mar González Franc

Coaching Imagery to Athletes with Aphantasia

We administered the Plymouth Sensory Imagery Questionnaire (Psi-Q) which tests multi-sensory imagery, to athletes (n=329) from 9 different sports to locate poor/aphantasic (baseline scores <4.2/10) imagers with the aim to subsequently enhance imagery ability. The low imagery sample (n=27) were randomly split into two groups who received the intervention: Functional Imagery Training (FIT), either immediately, or delayed by one month at which point the delayed group were tested again on the Psi-Q. All participants were tested after FIT delivery and six months post intervention. The delayed group showed no significant change between baseline and the start of FIT delivery but both groups imagery score improved significantly (p=0.001) after the intervention which was maintained six months post intervention. This indicates that imagery can be trained, with those who identify as having aphantasia (although one participant did not improve on visual scores), and improvements maintained in poor imagers. Follow up interviews (n=22) on sporting application revealed that the majority now use imagery daily on process goals. Recommendations are given for ways to assess and train imagery in an applied sport setting

Perceptual abnormalities in amputees: phantom pain, mirror-touch synaesthesia and referred tactile sensations

It is often reported that after amputation people experience "a constant or inconstant... sensory ghost... faintly felt at time, but ready to be called up to [their] perception" (Mitchell, 1866). Perceptual abnormalities have been highlighted in amputees, such as sensations in the phantom when being stroked elsewhere (Ramachandran et al., 1992) or when observing someone in pain (Giummarra and Bradshaw, 2008). This thesis explored the perceptual changes that occur following amputation whist focusing on pain, vision and touch. A sample of over 100 amputees were recruited through the National Health Service. Despite finding no difference in phantom pain based on physical amputation details or nonpainful perceptual phenomena, results from Paper 1 indicated that phantom pain may be more intense, with sensations occurring more frequently, in amputees whose pain was triggerinduced. The survey in Paper 2 identified a group of amputees who in losing a limb acquired mirror-touch synaesthesia. Higher levels of empathy found in mirror-touch amputees might mean that some people are predisposed to develop synaesthesia, but that it takes sensory loss to bring dormant cross-sensory interactions into consciousness. Although the mirror-system may reach supra-threshold levels in some amputees, the experiments in Paper 3 suggested a relatively intact mirror-system in amputees overall. Specifically, in a task of apparent biological motion, amputees showed a similar, although weaker, pattern of results to normalbodied participants. The results of Paper 4 showed that tactile spatial acuity on the face was also largely not affected by amputation, as no difference was found between the sides ipsilateral and contralateral to the stump. In Paper 5 cross-modal cuing was used to investigate whether referred tactile sensations could prime a visually presented target in space occupied by the phantom limb. We conclude that perception is only moderately affected in most amputees, but that in some the sensory loss causes normally sub-threshold processing to enhance into conscious awareness

Participation as media: a compositional system for staging participation with reflective scenography

The practice-led research develops a compositional system for staging participation within reflective scenographies, and suggests an artistic concept of 'participation as media', which propose the participatory involvement as compositional material in itself. The research takes a starting point in the author's expert practice as a performer and director, and identifies key compositional problems from analysis of previous productions of participatory artworks. The practice-led research processes were organised into two laboratory events, a series of method investigations, and the production of two participatory installation artworks Mirror-Zone-Site and Zen- Sofa Arrangement. The approach is to rethink theatre as a complex communicational system of reflective operations, and to recognise performer technique as several simultaneously working levels of self-referential communicative operations, that can be staged as a participatory condition by reflective scenography. From a compositional perspective the question is how to externalise the performer's technique as abstracted mediating structures, and implement them by the use of responsive and mediating technology embedded in the reflective operations of a scenography. The compositional system consists of design parameters, compositional strategies, and a postprogressive dramaturgy. The design parameters framing, channelling, and coupling, organise a calibration of the staged feedback operations. The compositional strategies, which derive from practices of performer technique, organise scenarios of introvert, extrovert and social referencing operations. The postprogressive dramaturgy informs the performative engagement of the participant as a process of experiential narrativation. The system enables a capability to navigate the compositional process into the complex creation of participatory engagement as a media in itself, and enables a structured overview on the compositional process, argued in an interdisciplinary context. The research investigates events that involve the visitor in the realisation of the work, to an extent where the media of the artwork is the activity of participation in itself and the participatory engagement forms a main site of the emergence of the artwork. Through the visitor's acts of participation, she releases the potential of the artwork, and as such, occupies a crucial position in the constitution of the work These artworks are suggested to stage the participant in structures of communication and include her as an operator in a communication device

'Rapport and Subversion: Mesmer's Treatment of Paradis and Its Influence On the Fiction of E.T.A. Hoffmann' and The Cost of Light (a novel)

This is a thesis in two parts, a novel and a critical project. Both are about Maria-Theresavon Paradis, a blind and possibly 'hysterical'pianist and composer, and Doctor Franz Anton Mesmer, the founder of Mesmerism, who temporarily managed to cure her with this controversial treatment. In the novel, The Cost of Light, Paradis's confessions to Mesmer allow me to show that her possible hysterical blindness arises from a complex cluster of motives, for example, a refusal to see, and too great a desire to witness, forbidden sights. In this way, the novel demonstrates the malleable nature of eighteenth-century hysteria, which, for Paradis, transforms itself constantly in relation to her psychosocial conditions and familial constraints. Her sudden blindness, her miraculous recovery of sight at Doctor Mesmer's hands and then its subsequent loss could thereforeallbe considered hysterical symptoms. The related critical project, 'Rapportand Subversion: Mesmer's Treatment of Paradis and Its Influence on the Fiction of E.T.A. Hoffmann'is divided into three chapters. 'Chapter 1: The Blind Musician'explores the nature of Paradis'spossibly hysterical blindness and suggests that,for her, blindness may well have had subversive and creative compensations. Her experiences are compared to those of the blind harpist, CharlottaSeuerling,and to those of Beethoven. 'Chapter 2: The Miracle Doctor'argues that the relationship between Mesmer and his famous patient and between othermesmerists and their subjects is one of power and control, reinforcing normativeeighteenth-century gender roles. It examines the effectiveness of Mesmerism in treating Paradis's blindness, and argues that her possible hysteria, and other factors, were potent subversive forces in undermining Mesmer's treatment. 'Chapter3: The Mesmerised Writer'establishes how E.T.A. Hoffmann's characters in'The Sandman','Councillor Krespel'and 'New Year's Eve Adventure'operate as paradigms for the Paradis/Mesmer relationship. This chapter also showshowHoffmann's awareness of Mesmerisminfluenced his short stories, 'Automata','The Magnetiser'and 'The Golden Pot.

Visual impairment: its impact upon and implications for aesthetic experience

With this research programme, I will be looking at how visually impaired people interpret the sensory inputs that artwork evokes together with the spatial environment that visually impaired people engage with. The study intertwines concepts of aesthetics that have specific relevance for visually impaired people, together with the processes and concepts associated with vision. The study refers to some common beliefs regarding blindness and provides some evidence of links between art and blindness. The study reflects upon how human cognitive processes are different for blind people, the use of verbal description used by visually impaired people and comments upon the logical reasoning processes developed by people with sight loss. Finally the study teases out methods of media manipulation, the interplay of different sensory stimulus and the control that visually impaired people endeavour to exert over an unseen environment. The nature of this research will be developed into a programme which explores and revisits the central themes of study using a system of concentric evolution. (See methodology section.) As a result, this 'intertwining study' will examine the values of each strand of research and will provide data regarding the aesthetic understanding and creative processes used by people with visual impairment, together with an appreciation of the methods blind people engage with to understand and use spatial properties