Haptics for the development of fundamental rhythm skills, including multi-limb coordination

This chapter considers the use of haptics for learning fundamental rhythm skills, including skills that depend on multi-limb coordination. Different sensory modalities have different strengths and weaknesses for the development of skills related to rhythm. For example, vision has low temporal resolution and performs poorly for tracking rhythms in real-time, whereas hearing is highly accurate. However, in the case of multi-limbed rhythms, neither hearing nor sight are particularly well suited to communicating exactly which limb does what and when, or how the limbs coordinate. By contrast, haptics can work especially well in this area, by applying haptic signals independently to each limb. We review relevant theories, including embodied interaction and biological entrainment. We present a range of applications of the Haptic Bracelets, which are computer-controlled wireless vibrotactile devices, one attached to each wrist and ankle. Haptic pulses are used to guide users in playing rhythmic patterns that require multi-limb coordination. One immediate aim of the system is to support the development of practical rhythm skills and multi-limb coordination. A longer-term goal is to aid the development of a wider range of fundamental rhythm skills including recognising, identifying, memorising, retaining, analysing, reproducing, coordinating, modifying and creating rhythms – particularly multi-stream (i.e. polyphonic) rhythmic sequences. Empirical results are presented. We reflect on related work, and discuss design issues for using haptics to support rhythm skills. Skills of this kind are essential not just to drummers and percussionists but also to keyboards players, and more generally to all musicians who need a firm grasp of rhythm

Reaching Performance in Heathy Individuals and Stroke Survivors Improves after Practice with Vibrotactile State Feedback

Stroke causes deficits of cognition, motor, and/or somatosensory functions. These deficits degrade the capability to perform activities of daily living (ADLs). Many research investigations have focused on mitigating the motor deficits of stroke through motor rehabilitation. However, somatosensory deficits are common and may contribute importantly to impairments in the control of functional arm movement. This dissertation advances the goal of promoting functional motor recovery after stroke by investigating the use of a vibrotactile feedback (VTF) body-machine interface (BMI). The VTF BMI is intended to improve control of the contralesional arm of stroke survivors by delivering supplemental limb-state feedback to the ipsilesional arm, where somatosensory feedback remains intact. To develop and utilize a VTF BMI, we first investigated how vibrotactile stimuli delivered on the arm are perceived and discriminated. We determined that stimuli are better perceived sequentially than those delivered simultaneously. Such stimuli can propagate up to 8 cm from the delivery site, so future applications should consider adequate spacing between stimulation sites. We applied these findings to create a multi-channel VTF interface to guide the arm in the absence of vision. In healthy people, we found that short-term practice, less than 2.5 hrs, allows for small improvements in the accuracy of horizontal planar reaching. Long-term practice, about 10 hrs, engages motor learning such that the accuracy and efficiency of reaching is improved and cognitive loading of VTF-guided reaching is reduced. During practice, participants adopted a movement strategy whereby BMI feedback changed in just one channel at a time. From this observation, we sought to develop a practice paradigm that might improve stroke survivors’ learning of VTF-guided reaching without vision. We investigated the effects of practice methods (whole practice vs part practice) in stroke survivors’ capability to make VTF-guided arm movements. Stroke survivors were able to improve the accuracy of VTF-guided reaching with practice, however there was no inherent differences between practice methods. In conclusion, practice on VTF-guided 2D reaching can be used by healthy people and stroke survivors. Future studies should investigate long-term practice in stroke survivors and their capability to use VTF BMIs to improve performance of unconstrained actions, including ADLs

Vibrotactile Guidance Cues For Target Identification

The purpose of this dissertation was to establish how vibrotactile guidance cues can be used to improve marksmanship. This work originated in an effort to provide covert communication, navigation, and weapon aiming cues for infantrymen. It is predominantly an application-driven investigation rather than driven a priori by specific theoretical predictions from models of human performance. Three experiments are presented. Experiment 1 established the affect on initial response to vibrotactile guidance cues of tactor placements on the palmer versus dorsal surface of the hand, and targets appearing left versus right of center. Results suggest that tactile cues provided on the left side of the medial line of the hand afford moving the hand to the left, while tactile cues provided on the right side of the medial line afford moving the hand to the right. Experiment 2 established the affect of continuous relative distance cues and on- versus off-target vibrotactile stimuli on reaction time and accuracy for target selection. Results indicated an interaction between the pulse rate of vibrotactile stimuli and the method used to highlight an on-target condition; the suppressed target condition was superior to the enhanced target condition when the pulse rate increased as the cursor moved closer to a target. Experiment 3 established if there are performance differences between discrete and continuous distance information for target selection, and investigated the interaction between the near-target pulse rate and on-target cues. Results indicate that maximizing the difference between near-target guidance cues and on-target cues reduces the target selection time, particularly when the near-target pulse rates are fast (ISI = 10 msec). The results also suggest that, as with vision, the vibrotactile off-target guidance cues are not necessary during the whole target selection task. Rather, the guidance cues can be provided only during the initial pop-up condition and during the sub-movements closing on the target

Between-Tactor Display Using Dynamic Tactile Stimuli

Display of illusory vibration locations between physical vibrotactile motors (tactors) placed on the skin has the potential to reduce the number of tactors in distributed tactile displays. This paper presents a between-tactor display method that uses dynamic tactile stimuli to generate illusory vibration locations. A belt with only 6 vibration motors displays 24 targets consisting of on-tactor and between-tactor locations. On-tactor locations are represented by simply vibrating the relevant single tactor. Between-tactor locations are displayed by adjusting the relative vibration amplitudes of two adjacent motors, with either (1) constant vibration amplitudes or (2) perturbed vibration amplitudes (creating local illusory motion). User testing showed that perturbations improve recognition accuracy for in-between tactor localization

Augmenting the Spatial Perception Capabilities of Users Who Are Blind

People who are blind face a series of challenges and limitations resulting from their lack of being able to see, forcing them to either seek the assistance of a sighted individual or work around the challenge by way of a inefficient adaptation (e.g. following the walls in a room in order to reach a door rather than walking in a straight line to the door). These challenges are directly related to blind users' lack of the spatial perception capabilities normally provided by the human vision system. In order to overcome these spatial perception related challenges, modern technologies can be used to convey spatial perception data through sensory substitution interfaces. This work is the culmination of several projects which address varying spatial perception problems for blind users. First we consider the development of non-visual natural user interfaces for interacting with large displays. This work explores the haptic interaction space in order to find useful and efficient haptic encodings for the spatial layout of items on large displays. Multiple interaction techniques are presented which build on prior research (Folmer et al. 2012), and the efficiency and usability of the most efficient of these encodings is evaluated with blind children. Next we evaluate the use of wearable technology in aiding navigation of blind individuals through large open spaces lacking tactile landmarks used during traditional white cane navigation. We explore the design of a computer vision application with an unobtrusive aural interface to minimize veering of the user while crossing a large open space. Together, these projects represent an exploration into the use of modern technology in augmenting the spatial perception capabilities of blind users

Vibrotactile biofeedback devices in Parkinson’s disease: a narrative review

Parkinson’s disease (PD) is often associated with a vast list of gait-associated disabilities, for which there is still a limited pharmacological/surgical treatment efficacy. Therefore, alternative approaches have emerged as vibrotactile biofeedback systems (VBS). This review aims to focus on the technologies supporting VBS and identify their effects on improving gait-associated disabilities by verifying how VBS were applied and validated with end-users. It is expected to furnish guidance to researchers looking to enhance the effectiveness of future vibrotactile cueing systems. The use of vibrotactile cues has proved to be relevant and attractive, as positive results have been obtained in patients’ gait performance, suitability in any environment, and easy adherence. There seems to be a preference in developing VBS to mitigate freezing of gait, to improve balance, to overcome the risk of fall, and a prevalent use to apply miniaturized wearable actuators and sensors. Most studies implemented a biofeedback loop able to provide rescue strategies during or after the detection of a gait-associated disability. However, there is a need of more clinical evidence and inclusion of experimental sessions to evaluate if the biofeedback was effectively integrated into the patients’ motor system.This work was supported by FCT national funds, under the national support to R&D units grant, through the reference project UIDB/04436/2020 and UIDP/04436/2020, and under the Reference Scholarship under Grant SFRH/BD/136569/2018

A wearable biofeedback device to improve motor symptoms in Parkinson’s disease

Dissertação de mestrado em Engenharia BiomédicaThis dissertation presents the work done during the fifth year of the course Integrated Master’s in Biomedical Engineering, in Medical Electronics. This work was carried out in the Biomedical & Bioinspired Robotic Devices Lab (BiRD Lab) at the MicroElectroMechanics Center (CMEMS) established at the University of Minho. For validation purposes and data acquisition, it was developed a collaboration with the Clinical Academic Center (2CA), located at Braga Hospital. The knowledge acquired in the development of this master thesis is linked to the motor rehabilitation and assistance of abnormal gait caused by a neurological disease. Indeed, this dissertation has two main goals: (1) validate a wearable biofeedback system (WBS) used for Parkinson's disease patients (PD); and (2) develop a digital biomarker of PD based on kinematic-driven data acquired with the WBS. The first goal aims to study the effects of vibrotactile biofeedback to play an augmentative role to help PD patients mitigate gait-associated impairments, while the second goal seeks to bring a step advance in the use of front-end algorithms to develop a biomarker of PD based on inertial data acquired with wearable devices. Indeed, a WBS is intended to provide motor rehabilitation & assistance, but also to be used as a clinical decision support tool for the classification of the motor disability level. This system provides vibrotactile feedback to PD patients, so that they can integrate it into their normal physiological gait system, allowing them to overcome their gait difficulties related to the level/degree of the disease. The system is based on a user- centered design, considering the end-user driven, multitasking and less cognitive effort concepts. This manuscript presents all steps taken along this dissertation regarding: the literature review and respective critical analysis; implemented tech-based procedures; validation outcomes complemented with results discussion; and main conclusions and future challenges.Esta dissertação apresenta o trabalho realizado durante o quinto ano do curso Mestrado Integrado em Engenharia Biomédica, em Eletrónica Médica. Este trabalho foi realizado no Biomedical & Bioinspired Robotic Devices Lab (BiRD Lab) no MicroElectroMechanics Center (CMEMS) estabelecido na Universidade do Minho. Para efeitos de validação e aquisição de dados, foi desenvolvida uma colaboração com Clinical Academic Center (2CA), localizado no Hospital de Braga. Os conhecimentos adquiridos no desenvolvimento desta tese de mestrado estão ligados à reabilitação motora e assistência de marcha anormal causada por uma doença neurológica. De facto, esta dissertação tem dois objetivos principais: (1) validar um sistema de biofeedback vestível (WBS) utilizado por doentes com doença de Parkinson (DP); e (2) desenvolver um biomarcador digital de PD baseado em dados cinemáticos adquiridos com o WBS. O primeiro objetivo visa o estudo dos efeitos do biofeedback vibrotáctil para desempenhar um papel de reforço para ajudar os pacientes com PD a mitigar as deficiências associadas à marcha, enquanto o segundo objetivo procura trazer um avanço na utilização de algoritmos front-end para biomarcar PD baseado em dados inerciais adquiridos com o dispositivos vestível. De facto, a partir de um WBS pretende-se fornecer reabilitação motora e assistência, mas também utilizá-lo como ferramenta de apoio à decisão clínica para a classificação do nível de deficiência motora. Este sistema fornece feedback vibrotáctil aos pacientes com PD, para que possam integrá-lo no seu sistema de marcha fisiológica normal, permitindo-lhes ultrapassar as suas dificuldades de marcha relacionadas com o nível/grau da doença. O sistema baseia-se numa conceção centrada no utilizador, considerando o utilizador final, multitarefas e conceitos de esforço menos cognitivo. Portanto, este manuscrito apresenta todos os passos dados ao longo desta dissertação relativamente a: revisão da literatura e respetiva análise crítica; procedimentos de base tecnológica implementados; resultados de validação complementados com discussão de resultados; e principais conclusões e desafios futuros

Intention recognition for dynamic role exchange in haptic collaboration

In human-computer collaboration involving haptics, a key issue that remains to be solved is to establish an intuitive communication between the partners. Even though computers are widely used to aid human operators in teleoperation, guidance, and training, because they lack the adaptability, versatility, and awareness of a human, their ability to improve efficiency and effectiveness in dynamic tasks is limited. We suggest that the communication between a human and a computer can be improved if it involves a decision-making process in which the computer is programmed to infer the intentions of the human operator and dynamically adjust the control levels of the interacting parties to facilitate a more intuitive interaction setup. In this paper, we investigate the utility of such a dynamic role exchange mechanism, where partners negotiate through the haptic channel to trade their control levels on a collaborative task. We examine the energy consumption, the work done on the manipulated object, and the joint efficiency in addition to the task performance. We show that when compared to an equal control condition, a role exchange mechanism improves task performance and the joint efficiency of the partners. We also show that augmenting the system with additional informative visual and vibrotactile cues, which are used to display the state of interaction, allows the users to become aware of the underlying role exchange mechanism and utilize it in favor of the task. These cues also improve the users sense of interaction and reinforce his/her belief that the computer aids with the execution of the task. © 2013 IEEE

Principles and Guidelines for Advancement of Touchscreen-Based Non-visual Access to 2D Spatial Information

Graphical materials such as graphs and maps are often inaccessible to millions of blind and visually-impaired (BVI) people, which negatively impacts their educational prospects, ability to travel, and vocational opportunities. To address this longstanding issue, a three-phase research program was conducted that builds on and extends previous work establishing touchscreen-based haptic cuing as a viable alternative for conveying digital graphics to BVI users. Although promising, this approach poses unique challenges that can only be addressed by schematizing the underlying graphical information based on perceptual and spatio-cognitive characteristics pertinent to touchscreen-based haptic access. Towards this end, this dissertation empirically identified a set of design parameters and guidelines through a logical progression of seven experiments. Phase I investigated perceptual characteristics related to touchscreen-based graphical access using vibrotactile stimuli, with results establishing three core perceptual guidelines: (1) a minimum line width of 1mm should be maintained for accurate line-detection (Exp-1), (2) a minimum interline gap of 4mm should be used for accurate discrimination of parallel vibrotactile lines (Exp-2), and (3) a minimum angular separation of 4mm should be used for accurate discrimination of oriented vibrotactile lines (Exp-3). Building on these parameters, Phase II studied the core spatio-cognitive characteristics pertinent to touchscreen-based non-visual learning of graphical information, with results leading to the specification of three design guidelines: (1) a minimum width of 4mm should be used for supporting tasks that require tracing of vibrotactile lines and judging their orientation (Exp-4), (2) a minimum width of 4mm should be maintained for accurate line tracing and learning of complex spatial path patterns (Exp-5), and (3) vibrotactile feedback should be used as a guiding cue to support the most accurate line tracing performance (Exp-6). Finally, Phase III demonstrated that schematizing line-based maps based on these design guidelines leads to development of an accurate cognitive map. Results from Experiment-7 provide theoretical evidence in support of learning from vision and touch as leading to the development of functionally equivalent amodal spatial representations in memory. Findings from all seven experiments contribute to new theories of haptic information processing that can guide the development of new touchscreen-based non-visual graphical access solutions

Engineering data compendium. Human perception and performance. User's guide

The concept underlying the Engineering Data Compendium was the product of a research and development program (Integrated Perceptual Information for Designers project) aimed at facilitating the application of basic research findings in human performance to the design and military crew systems. The principal objective was to develop a workable strategy for: (1) identifying and distilling information of potential value to system design from the existing research literature, and (2) presenting this technical information in a way that would aid its accessibility, interpretability, and applicability by systems designers. The present four volumes of the Engineering Data Compendium represent the first implementation of this strategy. This is the first volume, the User's Guide, containing a description of the program and instructions for its use