Designing a gesture-sound wearable system to motivate physical activity by altering body perception

People, through their bodily actions, engage in sensorimotor loops that connect them to the world and to their own bodies. People's brains integrate the incoming sensory information to form mental representations of their body appearance and capabilities. Technology provides exceptional opportunities to tweak sensorimotor loops and provide people with different experiences of their bodies. We recently showed that real-time sound feedback on one's movement (sonic avatar) can be used for sensory alteration of people's body perception, and in turn provoke enhanced motor behaviour, confidence and motivation for physical activity (PA) in people while increasing their positive emotions towards their own bodies. Here we describe the design process of a wearable prototype that aims to investigate how we can overcome known body-perception-related psychological barriers to PA by employing action-sound loops. The prototype consists of sensors that capture people's bodily actions and a gesture-sound palette that allows different action-sound mappings. Grounded in neuroscientific, clinical and sports psychology studies on body perception and PA, the ultimate design aim is to enhance PA in inactive populations by provoking changes on their bodily experience

Effects of pitch and musical sounds on body-representations when moving with sound

The effects of music on bodily movement and feelings, such as when people are dancing or engaged in physical activity, are well-documented¿people may move in response to the sound cues, feel powerful, less tired. How sounds and bodily movements relate to create such effects? Here we deconstruct the problem and investigate how different auditory features affect people's body-representation and feelings even when paired with the same movement. In three experiments, participants executed a simple arm raise synchronised with changing pitch in simple tones (Experiment 1), rich musical sounds (Experiment 2) and within different frequency ranges (Experiment 3), while we recorded indirect and direct measures on their movement, body-representations and feelings. Changes in pitch influenced people's general emotional state as well as the various bodily dimensions investigated¿movement, proprioceptive awareness and feelings about one's body and movement. Adding harmonic content amplified the differences between ascending and descending sounds, while shifting the absolute frequency range had a general effect on movement amplitude, bodily feelings and emotional state. These results provide new insights in the role of auditory and musical features in dance and exercise, and have implications for the design of sound-based applications supporting movement expression, physical activity, or rehabilitation.We acknowledge funding by the Spanish Agencia Estatal de Investigación (PID2019-105579RB-I00/AEI/10.13039/501100011033) and the European Research Council Grant (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 101002711). JL is funded by the Ministry of Economy, Industry and Competitivity of Spain (doctoral training Grant BES-2017-080471). OD is funded by the Volkswagen Foundation (Co-Sense grant). FB is partially funded by the ELEMENT project (ANR-18-CE33-0002)

Interactive sonification to assist children with autism during motor therapeutic interventions

Interactive sonification is an effective tool used to guide individuals when practicing movements. Little research has shown the use of interactive sonification in supporting motor therapeutic interventions for children with autism who exhibit motor impairments. The goal of this research is to study if children with autism understand the use of interactive sonification during motor therapeutic interventions, its potential impact of interactive sonification in the development of motor skills in children with autism, and the feasibility of using it in specialized schools for children with autism. We conducted two deployment studies in Mexico using Go-with-the-Flow, a framework to sonify movements previously developed for chronic pain rehabilitation. In the first study, six children with autism were asked to perform the forward reach and lateral upper-limb exercises while listening to three different sound structures (i.e., one discrete and two continuous sounds). Results showed that children with autism exhibit awareness about the sonification of their movements and engage with the sonification. We then adapted the sonifications based on the results of the first study, for motor therapy of children with autism. In the next study, nine children with autism were asked to perform upper-limb lateral, cross-lateral, and push movements while listening to five different sound structures (i.e., three discrete and two continues) designed to sonify the movements. Results showed that discrete sound structures engage the children in the performance of upper-limb movements and increase their ability to perform the movements correctly. We finally propose design considerations that could guide the design of projects related to interactive sonification