BendableSound: An Elastic Multisensory Surface Using Touch-based interactions to Assist Children with Severe Autism During Music Therapy

Neurological Music Therapy uses live music to improve the sensorimotor regulation of children with severe autism. However, they often lack musical training and their impairments limit their interactions with musical instruments. In this paper, we present our co-design work that led to the BendableSound prototype: an elastic multisensory surface encouraging users to practice coordination movements when touching a fabric to play sounds. We present the results of a formative study conducted with 18 teachers showing BendableSound was perceived as “usable” and “attractive”. Then, we present a deployment study with 24 children with severe autism showing BendableSound is “easy to use” and may potentially have therapeutic benefits regarding attention and motor development. We propose a set of design insights that could guide the design of natural user interfaces, particularly elastic multisensory surfaces. We close with a discussion and directions for future work

Interactions of children and young adults using large-scale elastic displays

Elastic displays provide a unique and intuitive interaction and could be deployed at large-scale. As an emerging technology, open questions about the benefits large-scale elastic displays offer over rigid displays and their potential application to our everyday lives. In this paper, we present an overview of a 4-year project. First, we describe the development of a large-scale elastic display, called BendableSound. Second, we explain the results of a laboratory study, showing the elastic display has a better user and sensory experience than a rigid one. Third, we describe the results of two deployment studies showing how BendableSound could support the therapeutic practices of children with autism and the early development of toddlers. We close discussing open challenges to study the untapped potential of elastic displays in pervasive computing

Furthering Development of Smart Fabrics to Improve the Accessibility of Music Therapy

In this paper, we present the design and development of HarmonicThreads, a smart, cost-effective fabric augmented by generative machine learning algorithms to create music in real time according to the user\u27s interaction. In this manner, we hypothesize that individuals with sensory differences could take advantage of the fabric\u27s flexibility, the music will adapt according to users\u27 interaction, and the affordable hardware we propose will make it more accessible. We follow a design thinking methodology using data from a multidisciplinary team in Mexico and the United States. Then we will close this paper by discussing challenges in developing accessible smart fabrics in different contexts

Touch or Touchless? Evaluating Usability of Interactive Displays for Persons with Autistic Spectrum Disorders

Interactive public displays have been exploited and studied for engaging interaction in several previous studies. In this context, applications have been focused on supporting learning or entertainment activities, specifically designed for people with special needs. This includes, for example, those with Autism Spectrum Disorders (ASD). In this paper, we present a comparison study aimed at understanding the difference in terms of usability, effectiveness, and enjoyment perceived by users with ASD between two interaction modalities usually supported by interactive displays: touch-based and touchless gestural interaction. We present the outcomes of a within-subject setup involving 8 ASD users (age 18-25 y.o., IQ 40-60), based on the use of two similar user interfaces, differing only by the interaction modality. We show that touch interaction provides higher usability level and results in more effective actions, although touchless interaction is more effective in terms of enjoyment and engagemen

Innovative Computer Technology in music based interventions for individuals with autism - Moving beyond traditional interactive music therapy techniques

Individuals with autism spectrum disorder (ASD) who exhibit developmental lim- itations in social-emotional interaction and communication, are widely reported to respond positively to music therapy interventions that incorporate active and im- provisational techniques. The fundamental elements of music have been shown to bypass both cognitive and language impairments to help facilitate communication skills, form social relationships and promote cognitive progression. In recent years, computer technologies such as mobile devices and interactive motion capture systems are being developed to provide new approaches to music interaction. When included into new or existing music interventions, digital technology can enhance the music playing experience and motivate children to interact differently with their environ- ment through providing novel and engaging approaches to music interaction and creativity. This review highlights the core symptoms of ASD and presents evidence that supports the use of music therapy techniques within the autistic population, underpinned by neurological studies reporting on benefits of musical intervention. Further to this, it is explained how new technologies such as mobile devices and multi-sensory computer systems can move beyond traditional music instruments to add additional dimensions to music therapy sessions. Finally suggestions of future research are made alongside considerations in designing new and novel auditory interventions

XR, music and neurodiversity: design and application of new mixed reality technologies that facilitate musical intervention for children with autism spectrum conditions

This thesis, accompanied by the practice outputs,investigates sensory integration, social interaction and creativity through a newly developed VR-musical interface designed exclusively for children with a high-functioning autism spectrum condition (ASC).The results aim to contribute to the limited expanse of literature and research surrounding Virtual Reality (VR) musical interventions and Immersive Virtual Environments (IVEs) designed to support individuals with neurodevelopmental conditions. The author has developed bespoke hardware, software and a new methodology to conduct field investigations. These outputs include a Virtual Immersive Musical Reality Intervention (ViMRI) protocol, a Supplemental Personalised, immersive Musical Experience(SPiME) programme, the Assisted Real-time Three-dimensional Immersive Musical Intervention System’ (ARTIMIS) and a bespoke (and fully configurable) ‘Creative immersive interactive Musical Software’ application (CiiMS). The outputs are each implemented within a series of institutional investigations of 18 autistic child participants. Four groups are evaluated using newly developed virtual assessment and scoring mechanisms devised exclusively from long-established rating scales. Key quantitative indicators from the datasets demonstrate consistent findings and significant improvements for individual preferences (likes), fear reduction efficacy, and social interaction. Six individual case studies present positive qualitative results demonstrating improved decision-making and sensorimotor processing. The preliminary research trials further indicate that using this virtual-reality music technology system and newly developed protocols produces notable improvements for participants with an ASC. More significantly, there is evidence that the supplemental technology facilitates a reduction in psychological anxiety and improvements in dexterity. The virtual music composition and improvisation system presented here require further extensive testing in different spheres for proof of concept

Creative Haptic Interface Design for the Aging Population

Audiovisual human-computer-interfaces still make up the majority of content to the public; however, haptic interfaces offer unique advantage over the dominant information infrastructure, particularly for users with a disability or diminishing cognitive and physical skills like the elderly. The tactile sense allows users to integrate new, unobstructive channels for digital information into their sensorium, one that is less likely to be overwhelmed compared to vision and audition. Haptics research focus on the development of hardware, improving resolution, modality, and fidelity of the actuators. Despite the technological limitations, haptic interfaces are shown to reinforce physical skill acquisition, therapy, and communication. This chapter will present key characteristics intuitive tactile interfaces should capture for elderly end-users; sample projects will showcase unique applications and designs that identify the limitations of the UI

Studying neuroanatomy using MRI

The study of neuroanatomy using imaging enables key insights into how our brains function, are shaped by genes and environment, and change with development, aging, and disease. Developments in MRI acquisition, image processing, and data modelling have been key to these advances. However, MRI provides an indirect measurement of the biological signals we aim to investigate. Thus, artifacts and key questions of correct interpretation can confound the readouts provided by anatomical MRI. In this review we provide an overview of the methods for measuring macro- and mesoscopic structure and inferring microstructural properties; we also describe key artefacts and confounds that can lead to incorrect conclusions. Ultimately, we believe that, though methods need to improve and caution is required in its interpretation, structural MRI continues to have great promise in furthering our understanding of how the brain works