Personalising Vibrotactile Displays through Perceptual Sensitivity Adjustment

Haptic displays are commonly limited to transmitting a discrete set of tactile motives. In this paper, we explore the transmission of real-valued information through vibrotactile displays. We simulate spatial continuity with three perceptual models commonly used to create phantom sensations: the linear, logarithmic and power model. We show that these generic models lead to limited decoding precision, and propose a method for model personalization adjusting to idiosyncratic and spatial variations in perceptual sensitivity. We evaluate this approach using two haptic display layouts: circular, worn around the wrist and the upper arm, and straight, worn along the forearm. Results of a user study measuring continuous value decoding precision show that users were able to decode continuous values with relatively high accuracy (4.4% mean error), circular layouts performed particularly well, and personalisation through sensitivity adjustment increased decoding precision

A brain-computer interface with vibrotactile biofeedback for haptic information

<p>Abstract</p> <p>Background</p> <p>It has been suggested that Brain-Computer Interfaces (BCI) may one day be suitable for controlling a neuroprosthesis. For closed-loop operation of BCI, a tactile feedback channel that is compatible with neuroprosthetic applications is desired. Operation of an EEG-based BCI using only <it>vibrotactile feedback</it>, a commonly used method to convey haptic senses of contact and pressure, is demonstrated with a high level of accuracy.</p> <p>Methods</p> <p>A Mu-rhythm based BCI using a motor imagery paradigm was used to control the position of a virtual cursor. The cursor position was shown visually as well as transmitted haptically by modulating the intensity of a vibrotactile stimulus to the upper limb. A total of six subjects operated the BCI in a two-stage targeting task, receiving only vibrotactile biofeedback of performance. The location of the vibration was also systematically varied between the left and right arms to investigate location-dependent effects on performance.</p> <p>Results and Conclusion</p> <p>Subjects are able to control the BCI using only vibrotactile feedback with an average accuracy of 56% and as high as 72%. These accuracies are significantly higher than the 15% predicted by random chance if the subject had no voluntary control of their Mu-rhythm. The results of this study demonstrate that vibrotactile feedback is an effective biofeedback modality to operate a BCI using motor imagery. In addition, the study shows that placement of the vibrotactile stimulation on the biceps ipsilateral or contralateral to the motor imagery introduces a significant bias in the BCI accuracy. This bias is consistent with a drop in performance generated by stimulation of the contralateral limb. Users demonstrated the capability to overcome this bias with training.</p

Augmenting Sensorimotor Control Using “Goal-Aware” Vibrotactile Stimulation during Reaching and Manipulation Behaviors

We describe two sets of experiments that examine the ability of vibrotactile encoding of simple position error and combined object states (calculated from an optimal controller) to enhance performance of reaching and manipulation tasks in healthy human adults. The goal of the first experiment (tracking) was to follow a moving target with a cursor on a computer screen. Visual and/or vibrotactile cues were provided in this experiment, and vibrotactile feedback was redundant with visual feedback in that it did not encode any information above and beyond what was already available via vision. After only 10 minutes of practice using vibrotactile feedback to guide performance, subjects tracked the moving target with response latency and movement accuracy values approaching those observed under visually guided reaching. Unlike previous reports on multisensory enhancement, combining vibrotactile and visual feedback of performance errors conferred neither positive nor negative effects on task performance. In the second experiment (balancing), vibrotactile feedback encoded a corrective motor command as a linear combination of object states (derived from a linear-quadratic regulator implementing a trade-off between kinematic and energetic performance) to teach subjects how to balance a simulated inverted pendulum. Here, the tactile feedback signal differed from visual feedback in that it provided information that was not readily available from visual feedback alone. Immediately after applying this novel “goal-aware” vibrotactile feedback, time to failure was improved by a factor of three. Additionally, the effect of vibrotactile training persisted after the feedback was removed. These results suggest that vibrotactile encoding of appropriate combinations of state information may be an effective form of augmented sensory feedback that can be applied, among other purposes, to compensate for lost or compromised proprioception as commonly observed, for example, in stroke survivors

Psychometric characterization of incidental feedback sources during grasping with a hand prosthesis

Background A prosthetic system should ideally reinstate the bidirectional communication between the user’s brain and its end effector by restoring both motor and sensory functions lost after an amputation. However, current commercial prostheses generally do not incorporate somatosensory feedback. Even without explicit feedback, grasping using a prosthesis partly relies on sensory information. Indeed, the prosthesis operation is characterized by visual and sound cues that could be exploited by the user to estimate the prosthesis state. However, the quality of this incidental feedback has not been objectively evaluated. Methods In this study, the psychometric properties of the auditory and visual feedback of prosthesis motion were assessed and compared to that of a vibro-tactile interface. Twelve able-bodied subjects passively observed prosthesis closing and grasping an object, and they were asked to discriminate (experiment I) or estimate (experiment II) the closing velocity of the prosthesis using visual (VIS), acoustic (SND), or combined (VIS?+?SND) feedback. In experiment II, the subjects performed the task also with a vibrotactile stimulus (VIB) delivered using a single tactor. The outcome measures for the discrimination and estimation experiments were just noticeable difference (JND) and median absolute estimation error (MAE), respectively. Results The results demonstrated that the incidental sources provided a remarkably good discrimination and estimation of the closing velocity, significantly outperforming the vibrotactile feedback. Using incidental sources, the subjects could discriminate almost the minimum possible increment/decrement in velocity that could be commanded to the prosthesis (median JND?<?2% for SND and VIS?+?SND). Similarly, the median MAE in estimating the prosthesis velocity randomly commanded from the full working range was also low, i.e., approximately 5% in SND and VIS?+?SND. Conclusions Since the closing velocity is proportional to grasping force in state-of-the-art myoelectric prostheses, the results of the present study imply that the incidental feedback, when available, could be usefully exploited for grasping force control. Therefore, the impact of incidental feedback needs to be considered when designing a feedback interface in prosthetics, especially since the quality of estimation using supplemental sources (e.g., vibration) can be worse compared to that of the intrinsic cues

Design and Assessment of Vibrotactile Biofeedback and Instructional Systems for Balance Rehabilitation Applications.

Sensory augmentation, a type of biofeedback, is a technique for supplementing or reinforcing native sensory inputs. In the context of balance-related applications, it provides users with additional information about body motion, usually with respect to the gravito-inertial environment. Multiple studies have demonstrated that biofeedback, regardless of the feedback modality (i.e., vibrotactile, electrotactile, auditory), decreases body sway during real-time use within a laboratory setting. However, in their current laboratory-based form, existing vibrotactile biofeedback devices are not appropriate for use in clinical and/or home-based rehabilitation settings due to the expense, size, and operating complexity of the instrumentation required. This dissertation describes the design, development, and preliminary assessment of two technologies that support clinical and home-based balance rehabilitation training. The first system provides vibrotactile-based instructional motion cues to a trainee based on the measured difference between the expert’s and trainee’s motions. The design of the vibrotactile display is supported by a study that characterizes the non-volitional postural responses to vibrotactile stimulation applied to the torso. This study shows that vibration applied individually by tactors over the internal oblique and erector spinae muscles induces a postural shift of the order of one degree oriented in the direction of the stimulation. Furthermore, human performance is characterized both experimentally and theoretically when the expert–trainee error thresholds and nature of the control signal are varied. The results suggest that expert–subject cross-correlation values were maximized and position errors and time delays were minimized when the controller uses a 0.5 error threshold and proportional plus derivative feedback control signal, and that subject performance decreases as motion speed and complexity increase. The second system provides vibrotactile biofeedback about body motion using a cell phone. The system is capable of providing real-time vibrotactile cues that inform corrective trunk tilt responses. When feedback is available, both healthy subjects and those with vestibular involvement significantly reduce their anterior-posterior or medial-lateral root-mean-square body sway, have significantly smaller elliptical area fits to their sway trajectory, spend a significantly greater mean percentage time within the no feedback zone, and show a significantly greater A/P or M/L mean power frequency.Ph.D.Mechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/91546/1/channy_1.pd

Doctor of Philosophy

dissertationWhen interacting with objects, humans utilize their sense of touch to provide information about the object and surroundings. However, in video games, virtual reality, and training exercises, humans do not always have information available through the sense of touch. Several types of haptic feedback devices have been created to provide touch information in these scenarios. This dissertation describes the use of tactile skin stretch feedback to provide cues that convey direction information to a user. The direction cues can be used to guide a user or provide information about the environment. The tactile skin stretch feedback devices described herein provide feedback directly to the hands, just as in many real life interactions involving the sense of touch. The devices utilize a moving tactor (actuated skin contact surface, also called a contactor) and surrounding material to give the user a sense of the relative motion. Several game controller prototypes with skin stretch feedback embedded into the device to interface with the fingers were constructed. Experiments were conducted to evaluate user performance in moving the joysticks to match the direction of the stimulus. These experiments investigated stimulus masking effects with both skin stretch feedback and vibrotactile feedback. A controller with feedback on the thumb joysticks was found to have higher user accuracy. Next, precision grip and power grip skin stretch feedback devices were created to investigate cues to convey motion in a three-dimensional space. Experiments were conducted to compare the two devices and to explore user accuracy in identifying different direction cue types. The precision grip device was found to be superior in communicating direction cues to users in four degrees of freedom. Finally, closed-loop control was implemented to guide users to a specific location and orientation within a three-dimensional space. Experiments were conducted to improve controller feedback which in turn improved user performance. Experiments were also conducted to investigate the feasibility of providing multiple cues in succession, in order to guide a user with multiple motions of the hand. It was found that users can successfully reach multiple target locations and orientations in succession

A Review of Non-Invasive Haptic Feedback stimulation Techniques for Upper Extremity Prostheses

A sense of touch is essential for amputees to reintegrate into their social and work life. The design of the next generation of the prostheses will have the ability to effectively convey the tactile information between the amputee and the artificial limbs. This work reviews non-invasive haptic feedback stimulation techniques to convey the tactile information from the prosthetic hand to the amputee’s brain. Various types of actuators that been used to stimulate the patient’s residual limb for different types of artificial prostheses in previous studies have been reviewed in terms of functionality, effectiveness, wearability and comfort. The non-invasive hybrid feedback stimulation system was found to be better in terms of the stimulus identification rate of the haptic prostheses’ users. It can be conclude that integrating hybrid haptic feedback stimulation system with the upper limb prostheses leads to improving its acceptance among users

The Effect of Vibrotactile Feedback on Healthy People and People with Vestibular Disorders during Dual-task Conditions

Vibrotactile feedback (VTF) has been shown to improve balance performance in healthy people and people with vestibular disorders in a single-task experimental condition. However, typical balance activities occur in a multi-task environment. Dual-task performance can degrade with age and in people with vestibular disorders. It is unclear if the ability to use VTF might be affected by dual-task conditions in different age groups and people with vestibular disorders. The purposes of this dissertation are to investigate in healthy young and older adults, and people with vestibular disorders: 1) balance performance in a dual-task paradigm under various sensory conditions while using VTF, 2) reaction time during dual-task performance under different sensory conditions while using VTF, and 3) the effect of testing duration and visit on VTF use. Three study visits were included in this dissertation study: one screening visit and two experimental visits. Twenty younger and twenty older subjects were recruited in the first study to determine if VTF was affected by age. Seven people with unilateral vestibular hypofunction (UVH) and seven age-matched controls were recruited in the second study to investigate the effect of vestibular dysfunction. The results showed that young and older adults use VTF differently, depending on the underlying sensory integration balance task. Older adults increased postural sway during fixed platform conditions, but both young and older adults decreased postural sway during sway-referenced platform conditions. Reaction times on the secondary cognitive tasks increased more while using the VTF in older adults compared with young adults. This finding suggested that using VTF requires greater attention in older adults. The trial duration and visit also affected postural sway performance while VTF was applied. Similar postural sway results were found when comparing people with UVH and age-matched controls. However, no group difference was found between people with UVH and age-matched controls in the magnitude of postural sway, which suggested that people with UVH were able to use VTF under dual-task conditions similar to normal adults. Our data also indicated that people with UVH require more attentional resources to perform secondary cognitive tasks while using VTF

Vibrotactile Sensory Augmentation and Machine Learning Based Approaches for Balance Rehabilitation

Vestibular disorders and aging can negatively impact balance performance. Currently, the most effective approach for improving balance is exercise-based balance rehabilitation. Despite its effectiveness, balance rehabilitation does not always result in a full recovery of balance function. In this dissertation, vibrotactile sensory augmentation (SA) and machine learning (ML) were studied as approaches for further improving balance rehabilitation outcomes. Vibrotactile SA provides a form of haptic cues to complement and/or replace sensory information from the somatosensory, visual and vestibular sensory systems. Previous studies have shown that people can reduce their body sway when vibrotactile SA is provided; however, limited controlled studies have investigated the retention of balance improvements after training with SA has ceased. The primary aim of this research was to examine the effects of supervised balance rehabilitation with vibrotactile SA. Two studies were conducted among people with unilateral vestibular disorders and healthy older adults to explore the use of vibrotactile SA for therapeutic and preventative purposes, respectively. The study among people with unilateral vestibular disorders provided six weeks of supervised in-clinic balance training. The findings indicated that training with vibrotactile SA led to additional body sway reduction for balance exercises with head movements, and the improvements were retained for up to six months. Training with vibrotactile SA did not lead to significant additional improvements in the majority of the clinical outcomes except for the Activities-specific Balance Confidence scale. The study among older adults provided semi-supervised in-home balance rehabilitation training using a novel smartphone balance trainer. After completing eight weeks of balance training, participants who trained with vibrotactile SA showed significantly greater improvements in standing-related clinical outcomes, but not in gait-related clinical outcomes, compared with those who trained without SA. In addition to investigating the effects of long-term balance training with SA, we sought to study the effects of vibrotactile display design on people’s reaction times to vibrational cues. Among the various factors tested, the vibration frequency and tactor type had relatively small effects on reaction times, while stimulus location and secondary cognitive task had relatively large effects. Factors affected young and older adults’ reaction times in a similar manner, but with different magnitudes. Lastly, we explored the potential for ML to inform balance exercise progression for future applications of unsupervised balance training. We mapped body motion data measured by wearable inertial measurement units to balance assessment ratings provided by physical therapists. By training a multi-class classifier using the leave-one-participant-out cross-validation method, we found approximately 82% agreement among trained classifier and physical therapist assessments. The findings of this dissertation suggest that vibrotactile SA can be used as a rehabilitation tool to further improve a subset of clinical outcomes resulting from supervised balance rehabilitation training. Specifically, individuals who train with a SA device may have additional confidence in performing balance activities and greater postural stability, which could decrease their fear of falling and fall risk, and subsequently increase their quality of life. This research provides preliminary support for the hypothesized mechanism that SA promotes the central nervous system to reweight sensory inputs. The preliminary outcomes of this research also provide novel insights for unsupervised balance training that leverage wearable technology and ML techniques. By providing both SA and ML-based balance assessment ratings, the smart wearable device has the potential to improve individuals’ compliance and motivation for in-home balance training.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/143901/1/baotian_1.pd