A Review of Non-Invasive Haptic Feedback stimulation Techniques for Upper Extremity Prostheses

A sense of touch is essential for amputees to reintegrate into their social and work life. The design of the next generation of the prostheses will have the ability to effectively convey the tactile information between the amputee and the artificial limbs. This work reviews non-invasive haptic feedback stimulation techniques to convey the tactile information from the prosthetic hand to the amputee’s brain. Various types of actuators that been used to stimulate the patient’s residual limb for different types of artificial prostheses in previous studies have been reviewed in terms of functionality, effectiveness, wearability and comfort. The non-invasive hybrid feedback stimulation system was found to be better in terms of the stimulus identification rate of the haptic prostheses’ users. It can be conclude that integrating hybrid haptic feedback stimulation system with the upper limb prostheses leads to improving its acceptance among users

Haptic wearables as sensory replacement, sensory augmentation and trainer - a review

Sensory impairments decrease quality of life and can slow or hinder rehabilitation. Small, computationally powerful electronics have enabled the recent development of wearable systems aimed to improve function for individuals with sensory impairments. The purpose of this review is to synthesize current haptic wearable research for clinical applications involving sensory impairments. We define haptic wearables as untethered, ungrounded body worn devices that interact with skin directly or through clothing and can be used in natural environments outside a laboratory. Results of this review are categorized by degree of sensory impairment. Total impairment, such as in an amputee, blind, or deaf individual, involves haptics acting as sensory replacement; partial impairment, as is common in rehabilitation, involves haptics as sensory augmentation; and no impairment involves haptics as trainer. This review found that wearable haptic devices improved function for a variety of clinical applications including: rehabilitation, prosthetics, vestibular loss, osteoarthritis, vision loss and hearing loss. Future haptic wearables development should focus on clinical needs, intuitive and multimodal haptic displays, low energy demands, and biomechanical compliance for long-term usage

A hybrid haptic stimulation prosthetic wearable device to recover the missing sensation of the upper limb amputees

A hybrid haptic feedback stimulation system that is capable in sensing the contact pressure, the surface texture, and the temperature, simultaneously, was designed for a prosthetic hand to provide a tactile sensation to amputation patients. In addition, the haptic system was developed to enable the prosthetic’s users to implement withdrawal reflexes due to the thermal noxious stimulus in a quick manner. The re-sensation is achieved by non-invasively stimulating the skin of the patients’ residual limbs, based on the type and the level of tactile signals provided by the sensory system of the prostheses. Accordingly, three stages of design and development were performed to satisfy the research methodology. A vibrotactile prosthetic device, which is designed for the detection of contact pressure and surface texture in upper extremity, represents. While, the design of a novel wearable hybrid pressure-vibration haptic feedback stimulation device for conveying the tactile information regarding the contact pressure between the prosthetic hand and the grasped objects represents the second methodology stage. Lastly, the third stage was achieved by designing a novel hybrid pressure-vibration-temperature feedback stimulation system to provide a huge information regarding the prostheses environment to the users without brain confusing or requiring long pre-training. The main contribution of this work is the development and evaluation of the first step of a novel approach for a lightweight, 7 Degrees-Of-Freedom (DOF) tactile prosthetic arm to perform an effective as well as fast object manipulation and grasping. Furthermore, this study investigates the ability to convey the tactile information about the contact pressure, surface texture, and object temperature to the amputees with high identification accuracy by mean of using the designed hybrid pressure-vibration-temperature feedback wearable device. An evaluation of sensation and response has been conducted on forty healthy volunteers to evaluate the ability of the haptic system to stimulate the human nervous system. The results in term of Stimulus Identification Rate (SIR) show that all the volunteers were correctly able to discriminate the sensation of touch, start of touch, end of touch, and grasping objects. While 94%, 96%, 97%, and 95.24% of the entire stimuli were successfully identified by the volunteers during the experiments of slippage, pressure level, surface texture, and temperature, respectively. The position tracking controller system was designed to synchronize the movements of the volunteers’ elbow joints and the prosthetic’s elbow joint to record the withdrawal reflexes. The results verified the ability of the haptic system to excite the human brain at the abnormal noxious stimulus and enable the volunteers to perform a quick withdrawal reflex within 0.32 sec. The test results and the volunteers' response established evidence that amputees are able to recover their sense of the contact pressure, the surface texture, and the object temperature as well as to perform thermal withdrawal reflexes using the solution developed in this work

On Neuromechanical Approaches for the Study of Biological Grasp and Manipulation

Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for the design and construction of robotic systems. The past 20 years have seen several conceptual advances in both fields and the quest to unify them. Chief among them is the reluctant recognition that their underlying fundamental mechanisms may actually share limited common ground, while exhibiting many fundamental differences. This recognition is particularly liberating because it allows us to resolve and move beyond multiple paradoxes and contradictions that arose from the initial reasonable assumption of a large common ground. Here, we begin by introducing the perspective of neuromechanics, which emphasizes that real-world behavior emerges from the intimate interactions among the physical structure of the system, the mechanical requirements of a task, the feasible neural control actions to produce it, and the ability of the neuromuscular system to adapt through interactions with the environment. This allows us to articulate a succinct overview of a few salient conceptual paradoxes and contradictions regarding under-determined vs. over-determined mechanics, under- vs. over-actuated control, prescribed vs. emergent function, learning vs. implementation vs. adaptation, prescriptive vs. descriptive synergies, and optimal vs. habitual performance. We conclude by presenting open questions and suggesting directions for future research. We hope this frank assessment of the state-of-the-art will encourage and guide these communities to continue to interact and make progress in these important areas

Distributed Sensing and Stimulation Systems Towards Sense of Touch Restoration in Prosthetics

Modern prostheses aim at restoring the functional and aesthetic characteristics of the lost limb. To foster prosthesis embodiment and functionality, it is necessary to restitute both volitional control and sensory feedback. Contemporary feedback interfaces presented in research use few sensors and stimulation units to feedback at most two discrete feedback variables (e.g. grasping force and aperture), whereas the human sense of touch relies on a distributed network of mechanoreceptors providing high-fidelity spatial information. To provide this type of feedback in prosthetics, it is necessary to sense tactile information from artificial skin placed on the prosthesis and transmit tactile feedback above the amputation in order to map the interaction between the prosthesis and the environment. This thesis proposes the integration of distributed sensing systems (e-skin) to acquire tactile sensation, and non-invasive multichannel electrotactile feedback and virtual reality to deliver high-bandwidth information to the user. Its core focus addresses the development and testing of close-loop sensory feedback human-machine interface, based on the latest distributed sensing and stimulation techniques for restoring the sense of touch in prosthetics. To this end, the thesis is comprised of two introductory chapters that describe the state of art in the field, the objectives and the used methodology and contributions; as well as three studies distributed over stimulation system level and sensing system level. The first study presents the development of close-loop compensatory tracking system to evaluate the usability and effectiveness of electrotactile sensory feedback in enabling real-time close-loop control in prosthetics. It examines and compares the subject\u2019s adaptive performance and tolerance to random latencies while performing the dynamic control task (i.e. position control) and simultaneously receiving either visual feedback or electrotactile feedback for communicating the momentary tracking error. Moreover, it reported the minimum time delay needed for an abrupt impairment of users\u2019 performance. The experimental results have shown that electrotactile feedback performance is less prone to changes with longer delays. However, visual feedback drops faster than electrotactile with increased time delays. This is a good indication for the effectiveness of electrotactile feedback in enabling close- loop control in prosthetics, since some delays are inevitable. The second study describes the development of a novel non-invasive compact multichannel interface for electrotactile feedback, containing 24 pads electrode matrix, with fully programmable stimulation unit, that investigates the ability of able-bodied human subjects to localize the electrotactile stimulus delivered through the electrode matrix. Furthermore, it designed a novel dual parameter -modulation (interleaved frequency and intensity) and compared it to conventional stimulation (same frequency for all pads). In addition and for the first time, it compared the electrotactile stimulation to mechanical stimulation. More, it exposes the integration of virtual prosthesis with the developed system in order to achieve better user experience and object manipulation through mapping the acquired real-time collected tactile data and feedback it simultaneously to the user. The experimental results demonstrated that the proposed interleaved coding substantially improved the spatial localization compared to same-frequency stimulation. Furthermore, it showed that same-frequency stimulation was equivalent to mechanical stimulation, whereas the performance with dual-parameter modulation was significantly better. The third study presents the realization of a novel, flexible, screen- printed e-skin based on P(VDF-TrFE) piezoelectric polymers, that would cover the fingertips and the palm of the prosthetic hand (particularly the Michelangelo hand by Ottobock) and an assistive sensorized glove for stroke patients. Moreover, it developed a new validation methodology to examine the sensors behavior while being solicited. The characterization results showed compatibility between the expected (modeled) behavior of the electrical response of each sensor to measured mechanical (normal) force at the skin surface, which in turn proved the combination of both fabrication and assembly processes was successful. This paves the way to define a practical, simplified and reproducible characterization protocol for e-skin patches In conclusion, by adopting innovative methodologies in sensing and stimulation systems, this thesis advances the overall development of close-loop sensory feedback human-machine interface used for restoration of sense of touch in prosthetics. Moreover, this research could lead to high-bandwidth high-fidelity transmission of tactile information for modern dexterous prostheses that could ameliorate the end user experience and facilitate it acceptance in the daily life

Touching on elements for a non-invasive sensory feedback system for use in a prosthetic hand

Hand amputation results in the loss of motor and sensory functions, impacting activities of daily life and quality of life. Commercially available prosthetic hands restore the motor function but lack sensory feedback, which is crucial to receive information about the prosthesis state in real-time when interacting with the external environment. As a supplement to the missing sensory feedback, the amputee needs to rely on visual and audio cues to operate the prosthetic hand, which can be mentally demanding. This thesis revolves around finding potential solutions to contribute to an intuitive non-invasive sensory feedback system that could be cognitively less burdensome and enhance the sense of embodiment (the feeling that an artificial limb belongs to one’s own body), increasing acceptance of wearing a prosthesis.A sensory feedback system contains sensors to detect signals applied to the prosthetics. The signals are encoded via signal processing to resemble the detected sensation delivered by actuators on the skin. There is a challenge in implementing commercial sensors in a prosthetic finger. Due to the prosthetic finger’s curvature and the fact that some prosthetic hands use a covering rubber glove, the sensor response would be inaccurate. This thesis shows that a pneumatic touch sensor integrated into a rubber glove eliminates these errors. This sensor provides a consistent reading independent of the incident angle of stimulus, has a sensitivity of 0.82 kPa/N, a hysteresis error of 2.39±0.17%, and a linearity error of 2.95±0.40%.For intuitive tactile stimulation, it has been suggested that the feedback stimulus should be modality-matched with the intention to provide a sensation that can be easily associated with the real touch on the prosthetic hand, e.g., pressure on the prosthetic finger should provide pressure on the residual limb. A stimulus should also be spatially matched (e.g., position, size, and shape). Electrotactile stimulation has the ability to provide various sensations due to it having several adjustable parameters. Therefore, this type of stimulus is a good candidate for discrimination of textures. A microphone can detect texture-elicited vibrations to be processed, and by varying, e.g., the median frequency of the electrical stimulation, the signal can be presented on the skin. Participants in a study using electrotactile feedback showed a median accuracy of 85% in differentiating between four textures.During active exploration, electrotactile and vibrotactile feedback provide spatially matched modality stimulations, providing continuous feedback and providing a displaced sensation or a sensation dispatched on a larger area. Evaluating commonly used stimulation modalities using the Rubber Hand Illusion, modalities which resemble the intended sensation provide a more vivid illusion of ownership for the rubber hand.For a potentially more intuitive sensory feedback, the stimulation can be somatotopically matched, where the stimulus is experienced as being applied on a site corresponding to their missing hand. This is possible for amputees who experience referred sensation on their residual stump. However, not all amputees experience referred sensations. Nonetheless, after a structured training period, it is possible to learn to associate touch with specific fingers, and the effect persisted after two weeks. This effect was evaluated on participants with intact limbs, so it remains to evaluate this effect for amputees.In conclusion, this thesis proposes suggestions on sensory feedback systems that could be helpful in future prosthetic hands to (1) reduce their complexity and (2) enhance the sense of body ownership to enhance the overall sense of embodiment as an addition to an intuitive control system

Electronic systems for the restoration of the sense of touch in upper limb prosthetics

In the last few years, research on active prosthetics for upper limbs focused on improving the human functionalities and the control. New methods have been proposed for measuring the user muscle activity and translating it into the prosthesis control commands. Developing the feed-forward interface so that the prosthesis better follows the intention of the user is an important step towards improving the quality of life of people with limb amputation. However, prosthesis users can neither feel if something or someone is touching them over the prosthesis and nor perceive the temperature or roughness of objects. Prosthesis users are helped by looking at an object, but they cannot detect anything otherwise. Their sight gives them most information. Therefore, to foster the prosthesis embodiment and utility, it is necessary to have a prosthetic system that not only responds to the control signals provided by the user, but also transmits back to the user the information about the current state of the prosthesis. This thesis presents an electronic skin system to close the loop in prostheses towards the restoration of the sense of touch in prosthesis users. The proposed electronic skin system inlcudes an advanced distributed sensing (electronic skin), a system for (i) signal conditioning, (ii) data acquisition, and (iii) data processing, and a stimulation system. The idea is to integrate all these components into a myoelectric prosthesis. Embedding the electronic system and the sensing materials is a critical issue on the way of development of new prostheses. In particular, processing the data, originated from the electronic skin, into low- or high-level information is the key issue to be addressed by the embedded electronic system. Recently, it has been proved that the Machine Learning is a promising approach in processing tactile sensors information. Many studies have been shown the Machine Learning eectiveness in the classication of input touch modalities.More specically, this thesis is focused on the stimulation system, allowing the communication of a mechanical interaction from the electronic skin to prosthesis users, and the dedicated implementation of algorithms for processing tactile data originating from the electronic skin. On system level, the thesis provides design of the experimental setup, experimental protocol, and of algorithms to process tactile data. On architectural level, the thesis proposes a design ow for the implementation of digital circuits for both FPGA and integrated circuits, and techniques for the power management of embedded systems for Machine Learning algorithms

Prosthetic Control and Sensory Feedback for Upper Limb Amputees

Hand amputation could dramatically degrade the life quality of amputees. Many amputees use prostheses to restore part of the hand functions. Myoelectric prosthesis provides the most dexterous control. However, they are facing high rejection rate. One of the reasons is the lack of sensory feedback. There is a need for providing sensory feedback for myoelectric prosthesis users. It can improve object manipulation abilities, enhance the perceptual embodiment of myoelectric prostheses and help reduce phantom limb pain. This PhD work focuses on building bi-directional prostheses for upper limb amputees. In the introduction chapter, first, an overview of upper limb amputee demographics and upper limb prosthesis is given. Then the human somatosensory system is briefly introduced. The next part reviews invasive and non-invasive sensory feedback methods reported in the literature. The rest of the chapter describes the motivation of the project and the thesis organization. The first step to build a bi-directional prostheses is to investigate natural and robust multifunctional prosthetic control. Most of the commerical prostheses apply non-pattern recognition based myoelectric control methods, which offers only limited functionalities. In this thesis work, pattern recognition based prosthetic control employing three commonly used and representative machine learning algorithms is investigated. Three datasets involving different levels of upper arm movements are used for testing the algorithm effectiveness. The influence of time-domain features, window and increment sizes, algorithms, and post-processing techniques are analyzed and discussed. The next three chapters address different aspects of providing sensory feedback. The first focus of sensory feedback process is the automatic phantom map detection. Many amputees have referred sensation from their missing hand on their residual limbs (phantom maps). This skin area can serve as a target for providing amputees with non-invasive tactile sensory feedback. One of the challenges of providing sensory feedback on the phantom map is to define the accurate boundary of each phantom digit because the phantom map distribution varies from person to person. Automatic phantom map detection methods based on four decomposition support vector machine algorithms and three sampling methods are proposed. The accuracy and training/ classification time of each algorithm using a dense stimulation array and two coarse stimulation arrays are presented and compared. The next focus of the thesis is to develop non-invasive tactile display. The design and psychophysical testing results of three types of non-invasive tactile feedback arrays are presented: two with vibrotactile modality and one with multi modality. For vibrotactile, two types of miniaturized vibrators: eccentric rotating masses (ERMs) and linear resonant actuators (LRAs) were first tested on healthy subjects and their effectiveness was compared. Then the ERMs are integrated into a vibrotactile glove to assess the feasibility of providing sensory feedback for unilateral upper limb amputees on the contralateral hand. For multimodal stimulation, miniature multimodal actuators integrating servomotors and vibrators were designed. The actuator can be used to deliver both high-frequency vibration and low-frequency pressures simultaneously. By utilizing two modalities at the same time, the actuator stimulates different types of mechanoreceptors and thus h

Mechanisms for enabling closed-loop upper limb sensorimotor prosthetic control

Myoelectric upper limb prostheses are limited in their ability to provide sensory feedback to a user. The lack of sensory feedback forces prosthesis users to rely on visual feedback alone in manipulating objects, and often leads to abandonment of the prosthesis in favor of the user's unimpaired arm. Consequently, there is a critical need to develop mechanisms that enable people with upper limb amputations to be able to receive sensory feedback from the environment. The goal of this dissertation is to describe the development and evaluation of various mechanisms that enable simultaneous myoelectric control of hand prostheses with proprioceptive and touch/pressure feedback. Sensory feedback is enabled through the use of a passive skin stretch mechanism for proprioception (Chapter 2), an epidermal electronic device that can provide electrotactile stimulation (Chapter 3), and a custom-built prosthetic hand that relays contact and pressure information from the fingertips (Chapter 4). In each of these chapters, motor control is simultaneously enabled through the use of electromyographic sensors. The remainder of the dissertation focuses on a method of enabling long-term wear of electrotactile stimulation electrodes by modeling (Chapter 5) and controlling (Chapter 6) sensation intensity in response to changes in the impedance of the electrode-skin interface. The techniques described in this dissertation have the potential to improve prosthesis embodiment for a person with an upper limb amputation, with the ultimate goal of reducing prosthesis abandonment and improving quality of life