A Qualitative Analysis of Haptic Feedback in Music Focused Exercises

We present the findings of a pilot-study that analysed the role of haptic feedback in a musical context. To examine the role of haptics in Digital Musical Instrument (DMI) design an experiment was formulated to measure the users’ perception of device usability across four separate feedback stages: fully haptic (force and tactile combined), constant force only, vibrotactile only, and no feedback. The study was piloted over extended periods with the intention of exploring the application and integration of DMIs in real-world musical contexts. Applying a music orientated analysis of this type enabled the investigative process to not only take place over a comprehensive period, but allowed for the exploration of DMI integration in everyday compositional practices. As with any investigation that involves creativity, it was important that the participants did not feel rushed or restricted. That is, they were given sufficient time to explore and assess the different feedback types without constraint. This provided an accurate and representational set of qualitative data for validating the participants’ experience with the different feedback types they were presented with

A qualitative analysis of haptic feedback in music focused exercises new interfaces for musical expression

We present the findings of a pilot-study that analysed the role of haptic feedback in a musical context. To closely examine the role of haptics in Digital Musical Instrument (DMI) design an experiment was formulated to measure the users’ perception of device usability across four separate feedback stages: fully haptic (force and tactile combined), constant force only, vibrotactile only, and no feedback. The study was piloted over extended periods with the intention of exploring the application and integration of DMIs in real-world musical contexts. Applying a music orientated analysis of this type enabled the investigative process to not only take place over a comprehensive period, but allowed for the exploration of DMI integration in everyday compositional and explorative practices. As with any investigation that involves creativity, it was important that the participants did not feel rushed or restricted. That is, they were given sufficient time to explore and assess the different feedback types without constraint. This provided an accurate and representational set of qualitative data for validating the participants’ experience with the different feedback types they were presented with

BendableSound: An Elastic Multisensory Surface Using Touch-based interactions to Assist Children with Severe Autism During Music Therapy

Neurological Music Therapy uses live music to improve the sensorimotor regulation of children with severe autism. However, they often lack musical training and their impairments limit their interactions with musical instruments. In this paper, we present our co-design work that led to the BendableSound prototype: an elastic multisensory surface encouraging users to practice coordination movements when touching a fabric to play sounds. We present the results of a formative study conducted with 18 teachers showing BendableSound was perceived as “usable” and “attractive”. Then, we present a deployment study with 24 children with severe autism showing BendableSound is “easy to use” and may potentially have therapeutic benefits regarding attention and motor development. We propose a set of design insights that could guide the design of natural user interfaces, particularly elastic multisensory surfaces. We close with a discussion and directions for future work

Rhythmic Haptic Cueing for Gait Rehabilitation of Hemiparetic Stroke and Brain Injury Survivors

This thesis explores the gait rehabilitation of hemiparetic stroke and brain injury survivors by a process of haptic entrainment to rhythmic cues. Entrainment to auditory metronomes is known to improve gait; this thesis presents the first systematic study of entrainment for gait rehabilitation via the haptic modality. To investigate this approach, a multi-limb metronome capable of delivering a steady, isochronous haptic rhythm to alternating legs was developed, purpose-built for gait rehabilitation, together with appropriate software for monitoring and assessing gait. A formative observational study, carried out at a specialised neurological centre, supplemented by discussions with physiotherapists and neuropsychologists, was used to focus the scope on hemiparetic stroke and brain injury. A second formative study used a technology probe approach to explore the behaviour of hemiparetic participants under haptic cueing using a pre-existing prototype. Qualitative data was collected by observation of, and discussion with, participants and health professionals. In preparation for a quantitative gait study, a formal experiment was carried out to identify a workable range for haptic entrainment. This led to the creation of a procedure to screen out those with cognitive difficulties entraining to a rhythm, regardless of their walking ability. The final study was a quantitative gait study combining temporal and spatial data on haptically cued participants with hemiparetic stroke and brain injury. Gait characteristics were measured before, during and after cueing. All successfully screened participants were able to synchronise their steps to a haptically presented rhythm. For a substantial proportion of participants, an immediate (though not necessarily lasting) improvement of temporal gait characteristics was found during cueing. Some improvements over baseline occurred immediately afterwards, rather than during, haptic cueing. Design issues and trade-offs are identified, and interactions between perception, sensory deficit, attention, memory, cognitive load and haptic entrainment are noted

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD

“Musical Exercise” for people with visual impairments: A preliminary study with the blindfolded

Performing independent physical exercise is critical to maintain one\u27s good health, but it is specifically hard for people with visual impairments. To address this problem, we have developed a Musical Exercise platform for people with visual impairments so that they can perform exercise in a good form consistently. We designed six different conditions, including blindfolded or visual without audio conditions, and blindfolded or visual with two different types of audio feedback (continuous vs. discrete) conditions. Eighteen sighted participants participated in the experiment, by doing two exercises - squat and wall sit with all six conditions. The results show that Musical Exercise is a usable exercise assistance system without any adverse effect on exercise completion time or perceived workload. Also, the results show that with a specific sound design (i.e., discrete), participants in the blindfolded condition can do exercise as consistently as participants in the non-blindfolded condition. This implies that not all sounds equally work and thus, care is required to refine auditory displays. Potentials and limitations of Musical Exercise and future works are discussed with the results