Good vibrations: Guiding body movements with vibrotactile feedback

We describe the ongoing development of a system to support the teaching of good posture and bowing technique to novice violin players. Using an inertial motion capture system we can track in real-time a player’s bowing action and how it deviates from a target trajectory set by their music teacher. The system provides real-time vibrotactile feedback on the correctness of the student’s posture and bowing action. We present the findings of an initial study that shows that vibrotactile feedback can guide arm movements in one and two dimension pointing tasks. The advantages of vibrotactile feedback for teaching basic bowing technique to novice violin players are that it does not place demands on the students’ visual and auditory systems which are already heavily involved in the activity of music making, and is understood with little training

MusicJacket: the efficacy of real-time vibrotactile feedback for learning to play the violin

This research investigates the potential for vibrotactile feedback to enhance motor learning in the context of playing the violin. A prototype has been built which delivers vibrotactile feedback to the arms to indicate to a novice player how to correctly hold the violin and how to bow in a straight manner. This prototype was tested in a pilot user study with four complete beginners. Observations showed improvements in three of the four players whilst receiving the feedback. We also discuss the pros and cons of using negative feedback to enhance learning

Supporting sit-to-stand rehabilitation using smartphone sensors and arduino haptic feedback modules

The aim of this project is to design and build a system to aid patients in their rehabilitation after suffering a stroke. A stroke is one of the most serious conditions that an individual can suffer from, and the rehabilitation is often a long and difficult process. For many with movement effects, the sit-to-stand exercise is an important step in rehabilitation. The focus of this ongoing project is to create a system to assist sit-to-stand rehabilitation through the use of haptic feedback on balance. In this poster we present our initial prototype using standard smartphone accelerometers linked wirelessly to Arduino based vibration feedback modules mounted on the patients' legs. Initial feedback on the prototype is promising

Augmenting Sensorimotor Control Using “Goal-Aware” Vibrotactile Stimulation during Reaching and Manipulation Behaviors

We describe two sets of experiments that examine the ability of vibrotactile encoding of simple position error and combined object states (calculated from an optimal controller) to enhance performance of reaching and manipulation tasks in healthy human adults. The goal of the first experiment (tracking) was to follow a moving target with a cursor on a computer screen. Visual and/or vibrotactile cues were provided in this experiment, and vibrotactile feedback was redundant with visual feedback in that it did not encode any information above and beyond what was already available via vision. After only 10 minutes of practice using vibrotactile feedback to guide performance, subjects tracked the moving target with response latency and movement accuracy values approaching those observed under visually guided reaching. Unlike previous reports on multisensory enhancement, combining vibrotactile and visual feedback of performance errors conferred neither positive nor negative effects on task performance. In the second experiment (balancing), vibrotactile feedback encoded a corrective motor command as a linear combination of object states (derived from a linear-quadratic regulator implementing a trade-off between kinematic and energetic performance) to teach subjects how to balance a simulated inverted pendulum. Here, the tactile feedback signal differed from visual feedback in that it provided information that was not readily available from visual feedback alone. Immediately after applying this novel “goal-aware” vibrotactile feedback, time to failure was improved by a factor of three. Additionally, the effect of vibrotactile training persisted after the feedback was removed. These results suggest that vibrotactile encoding of appropriate combinations of state information may be an effective form of augmented sensory feedback that can be applied, among other purposes, to compensate for lost or compromised proprioception as commonly observed, for example, in stroke survivors

The haptic iPod: passive learning of multi-limb rhythm skills

Recent experiments showed that the use of haptic vibrotactile devices can support the learning of multi-limb rhythms [Holland et al., 2010]. These experiments centred on a tool called the Haptic Drum Kit, which uses vibrotactiles attached to wrists and ankles, together with a computer system that controls them, and a midi drum kit. The system uses haptic signals in real time, relying on human entrainment mechanisms [Clayton, Sager and Will, 2004] rather than stimulus response, to support the user in playing multi-limbed rhythms. In the present paper, we give a preliminary report on a new experiment, that aims to examine whether passive learning of multi-limb rhythms can occur through the silent playback of rhythmic stimuli via haptics when the subject is focusing on other tasks. The prototype system used for this new experiment is referred to as the Haptic iPod.Paper presented at the Workshop: When Words Fail: What can Music Interaction tell us about HCI? at BCS HCI Conference 2011, Newcastle, U

Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit

This paper introduces and explores a tool known as the Haptic Drum Kit. The Haptic Drum Kit employs four computer-controlled vibrotactile devices, one attached to each limb via the wrists and ankles. In the mode of use discussed in this paper, haptic pulses are used to guide the playing, on a drum kit, of rhythmic patterns that require multi-limb co-ordination. The immediate aim is to foster rhythm skills and multi-limb coordination. A broader aim is to systematically develop skills in recognizing, identifying, memorizing, retaining, analyzing, reproducing and composing monophonic and polyphonic rhythms. We consider the implications of three different theories for this approach: the work of the music educator Dalcroze (1865-1950 [1]; the entrainment theory of human rhythm perception and production [2,3]; and sensory motor contingency theory [4]. In this paper we introduce the Haptic Drum Kit; consider the implications of the above theories for this approach; report on a design study; and identify and discuss a variety of emerging design issues. As part of the design study, audio and haptic guidance was compared for five people learning to play polyphonic drum patterns of varying complexity. The results indicate that beginning drummers are able to learn intricate drum patterns from the haptic stimuli alone, although haptic plus audio is the mode of presentation preferred by subjects

Do you want to be a superhero? Boosting emotional states with the booth

Educational practitioners have stressed the relevance of providing learners with a set of 21th century skills that will allow them to face current life challenges. Among others this includes communication and social skills such as public speaking, argumentation, negotiation, etc. Besides the acquisition of knowledge and techniques, these skills have the special characteristic that their performance is usually conducted under emotionally charged and stressful situations. Recent advances in technology have allowed the creation of digital applications to support learners with the development of techniques for this type of skills. However, supporting learners on the preparation of a mindset that allows them to perform well under emotionally charged circumstances is an area that technology enhanced learning has practically overlooked. To examine this gap, we developed the Booth, an application designed to get learners into a powerful and resourceful emotional state. In this article we present a two-step user study. Results of the first evaluation show that the use of the Booth induced a positive emotional state on users. Results from the second step suggest that using the Booth helps learners to emotionally prepare for public speaking. (DIPF/Orig.

The Haptic Bracelets: learning multi-limb rhythm skills from haptic stimuli while reading

The Haptic Bracelets are a system designed to help people learn multi-limbed rhythms (which involve multiple simultaneous rhythmic patterns) while they carry out other tasks. The Haptic Bracelets consist of vibrotactiles attached to each wrist and ankle, together with a computer system to control them. In this chapter, we report on an early empirical test of the capabilities of this system, and consider de-sign implications. In the pre-test phase, participants were asked to play a series of multi-limb rhythms on a drum kit, guided by audio recordings. Participants’ per-formances in this phase provided a base reference for later comparisons. During the following passive learning phase, away from the drum kit, just two rhythms from the set were silently 'played' to each subject via vibrotactiles attached to wrists and ankles, while participants carried out a 30-minute reading comprehen-sion test. Different pairs of rhythms were chosen for different subjects to control for effects of rhythm complexity. In each case, the two rhythms were looped and alternated every few minutes. In the final phase, subjects were asked to play again at the drum kit the complete set of rhythms from the pre-test, including, of course, the two rhythms to which they had been passively exposed. Pending analysis of quantitative data focusing on accuracy, timing, number of attempts and number of errors, in this chapter we present preliminary findings based on participants’ sub-jective evaluations. Most participants thought that the technology helped them to understand rhythms and to play rhythms better, and preferred haptic to audio to find out which limb to play when. Most participants indicated that they would pre-fer using a combination of haptics and audio for learning rhythms to either mo-dality on its own. Replies to open questions were analysed to identify design is-sues, and implications for design improvements were considered

The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment

Objective and subjective measures of performance in virtual reality environments increase as more sensory cues are delivered and as simulation fidelity increases. Some cues (colour or sound) are easier to present than others (object weight, vestibular cues) so that substitute cues can be used to enhance informational content in a simulation at the expense of simulation fidelity. This study evaluates how substituting cues in one modality by alternative cues in another modality affects subjective and objective performance measures in a highly immersive virtual reality environment. Participants performed a wheel change in a virtual reality (VR) environment. Auditory, haptic and visual cues, signalling critical events in the simulation, were manipulated in a factorial design. Subjective ratings were recorded via questionnaires. The time taken to complete the task was used as an objective performance measure. The results show that participants performed best and felt an increased sense of immersion and involvement, collectively referred to as 'presence', when substitute multimodal sensory feedback was provided. Significant main effects of audio and tactile cues on task performance and on participants' subjective ratings were found. A significant negative relationship was found between the objective (overall completion times) and subjective (ratings of presence) performance measures. We conclude that increasing informational content, even if it disrupts fidelity, enhances performance and user's overall experience. On this basis we advocate the use of substitute cues in VR environments as an efficient method to enhance performance and user experience