Haptics for the development of fundamental rhythm skills, including multi-limb coordination

This chapter considers the use of haptics for learning fundamental rhythm skills, including skills that depend on multi-limb coordination. Different sensory modalities have different strengths and weaknesses for the development of skills related to rhythm. For example, vision has low temporal resolution and performs poorly for tracking rhythms in real-time, whereas hearing is highly accurate. However, in the case of multi-limbed rhythms, neither hearing nor sight are particularly well suited to communicating exactly which limb does what and when, or how the limbs coordinate. By contrast, haptics can work especially well in this area, by applying haptic signals independently to each limb. We review relevant theories, including embodied interaction and biological entrainment. We present a range of applications of the Haptic Bracelets, which are computer-controlled wireless vibrotactile devices, one attached to each wrist and ankle. Haptic pulses are used to guide users in playing rhythmic patterns that require multi-limb coordination. One immediate aim of the system is to support the development of practical rhythm skills and multi-limb coordination. A longer-term goal is to aid the development of a wider range of fundamental rhythm skills including recognising, identifying, memorising, retaining, analysing, reproducing, coordinating, modifying and creating rhythms – particularly multi-stream (i.e. polyphonic) rhythmic sequences. Empirical results are presented. We reflect on related work, and discuss design issues for using haptics to support rhythm skills. Skills of this kind are essential not just to drummers and percussionists but also to keyboards players, and more generally to all musicians who need a firm grasp of rhythm

Investigate the Piano Learning Rate with Haptic Actuators in Mixed Reality

With mixed reality (MR) becoming widely available, it could enhance learning because special equipment like musical instruments or access to instructors will be less of a concern. Furthermore, passive haptic learning systems to learn piano are promising research subjects. We combine both trends of MR and haptic learning to build a piano learning application. Through a study with diverse participants, we evaluate the piano application. The study results show the potentiality of the on-skin actuators and we hope our work could foster the future iterations of the actuators for a fun and effective learning environment

Tilt simulation : virtual reality based upper extremity stroke rehabilitation

The primary objective of this research is to design a recreational rehabilitation videogame that interactively encourages purposeful upper extremity gross motor movements. The simulation is also capable of continuous game modification to fit changing therapy goals, to match the needs of the players, and to provide continued motivation while capturing the interactive repetition. This thesis explains the design and features of this latest simulation - Tilt. Tilt uses physics to develop an engaging training experience and provides a realistic approach to virtual reality simulation including friction, elasticity and collisions between objects. It is designed to train upper extremity function as a unit involving multiple modalities simultaneously, either unilaterally or bilaterally. It is the latest addition to the NJIT Robot Assisted Virtual Rehabilitation (RAVR) system. It Employs the Cyber Glove and Flock of Birds systems to interface with the real world. This allows training motor function of patients that come to use in day to day life like making use of hands, fingers and shoulders to pick small objects on table, moving them and placing them elsewhere

BendableSound: An Elastic Multisensory Surface Using Touch-based interactions to Assist Children with Severe Autism During Music Therapy

Neurological Music Therapy uses live music to improve the sensorimotor regulation of children with severe autism. However, they often lack musical training and their impairments limit their interactions with musical instruments. In this paper, we present our co-design work that led to the BendableSound prototype: an elastic multisensory surface encouraging users to practice coordination movements when touching a fabric to play sounds. We present the results of a formative study conducted with 18 teachers showing BendableSound was perceived as “usable” and “attractive”. Then, we present a deployment study with 24 children with severe autism showing BendableSound is “easy to use” and may potentially have therapeutic benefits regarding attention and motor development. We propose a set of design insights that could guide the design of natural user interfaces, particularly elastic multisensory surfaces. We close with a discussion and directions for future work

Mobile Application to Train Chord-playing on Keyboard Instrument

Music played by a band is mostly annotated in chord charts, unlike complete notations for classical music. Beginner keyboardists playing in a band often struggle to catch up with only chord charts as reference because a single chord can be played in multiple ways on a keyboard instrument. To improve, practice is needed. However, the problems with practice chord-playing are the lack of training material, swayed practice goals as well as the immobility of instrument. Hence, the project has the objectives to: explore the multi-touch capabilities of tablet devices for music training, develop a structured and goal-oriented virtual keyboard mobile application to achieve chord-playing training; conduct a user perception study for the developed application on potential users; and conduct a usability study on the developed application. The application is targeted at beginner keyboardists to improve their existing skills in playing basic chords single-handedly, through a two-octave keyboard interface. The application was successfully developed with 4 prototype iterations. The user perception study showed that beginner keyboardists generally accept using mobile technology to train their chord-playing skills. The usability study showed that the developed application is quite a usable application

Towards a radical life. The social and aesthetic themes in Helen Keller’s activity in defence of humanity

Edyta Nieduziak, Towards a radical life. The social and aesthetic themes in Helen Keller’s activity in defence of humanity. Interdisciplinary Contexts of Special Pedagogy, no. 25, Poznań 2019. Pp. 229-254. Adam Mickiewicz University Press. ISSN 2300-391X. DOI: https://doi.org/10.14746/ikps.2019.25.10 The article is an attempt to describe three biographical motifs of H. Keller, referring to aesthetic experiences (R. Ingarden) caused by various experiences: tactile in contact with sculpture, haptic in contact with music, and literary. The starting point to consider, however, is the category of humanity in the sense of M.S. Archer. The reflexivity characteristic of humanity, combined with the aesthetic experience, makes the themes describe the non-aesthetic experiences of H. Keller. In the analysis, the author used H. Keller’s correspondence, her works, biographical sources, photographs and video recordings.Edyta Nieduziak, Towards a radical life. The social and aesthetic themes in Helen Keller’s activity in defence of humanity. Interdisciplinary Contexts of Special Pedagogy, no. 25, Poznań 2019. Pp. 229-254. Adam Mickiewicz University Press. ISSN 2300-391X. DOI: https://doi.org/10.14746/ikps.2019.25.10 The article is an attempt to describe three biographical motifs of H. Keller, referring to aesthetic experiences (R. Ingarden) caused by various experiences: tactile in contact with sculpture, haptic in contact with music, and literary. The starting point to consider, however, is the category of humanity in the sense of M.S. Archer. The reflexivity characteristic of humanity, combined with the aesthetic experience, makes the themes describe the non-aesthetic experiences of H. Keller. In the analysis, the author used H. Keller’s correspondence, her works, biographical sources, photographs and video recordings

Enhanced Learning of Jazz Chords with a Projector Based Piano Keyboard Augmentation

Published version: https://www.springer.com/gp/book/9783030353421acceptedVersio