Prosthetic Control and Sensory Feedback for Upper Limb Amputees

Hand amputation could dramatically degrade the life quality of amputees. Many amputees use prostheses to restore part of the hand functions. Myoelectric prosthesis provides the most dexterous control. However, they are facing high rejection rate. One of the reasons is the lack of sensory feedback. There is a need for providing sensory feedback for myoelectric prosthesis users. It can improve object manipulation abilities, enhance the perceptual embodiment of myoelectric prostheses and help reduce phantom limb pain. This PhD work focuses on building bi-directional prostheses for upper limb amputees. In the introduction chapter, first, an overview of upper limb amputee demographics and upper limb prosthesis is given. Then the human somatosensory system is briefly introduced. The next part reviews invasive and non-invasive sensory feedback methods reported in the literature. The rest of the chapter describes the motivation of the project and the thesis organization. The first step to build a bi-directional prostheses is to investigate natural and robust multifunctional prosthetic control. Most of the commerical prostheses apply non-pattern recognition based myoelectric control methods, which offers only limited functionalities. In this thesis work, pattern recognition based prosthetic control employing three commonly used and representative machine learning algorithms is investigated. Three datasets involving different levels of upper arm movements are used for testing the algorithm effectiveness. The influence of time-domain features, window and increment sizes, algorithms, and post-processing techniques are analyzed and discussed. The next three chapters address different aspects of providing sensory feedback. The first focus of sensory feedback process is the automatic phantom map detection. Many amputees have referred sensation from their missing hand on their residual limbs (phantom maps). This skin area can serve as a target for providing amputees with non-invasive tactile sensory feedback. One of the challenges of providing sensory feedback on the phantom map is to define the accurate boundary of each phantom digit because the phantom map distribution varies from person to person. Automatic phantom map detection methods based on four decomposition support vector machine algorithms and three sampling methods are proposed. The accuracy and training/ classification time of each algorithm using a dense stimulation array and two coarse stimulation arrays are presented and compared. The next focus of the thesis is to develop non-invasive tactile display. The design and psychophysical testing results of three types of non-invasive tactile feedback arrays are presented: two with vibrotactile modality and one with multi modality. For vibrotactile, two types of miniaturized vibrators: eccentric rotating masses (ERMs) and linear resonant actuators (LRAs) were first tested on healthy subjects and their effectiveness was compared. Then the ERMs are integrated into a vibrotactile glove to assess the feasibility of providing sensory feedback for unilateral upper limb amputees on the contralateral hand. For multimodal stimulation, miniature multimodal actuators integrating servomotors and vibrators were designed. The actuator can be used to deliver both high-frequency vibration and low-frequency pressures simultaneously. By utilizing two modalities at the same time, the actuator stimulates different types of mechanoreceptors and thus h

Multi-modality Sensory Feedback: a pilot study

The aim of this study is to test the hypotheses that a multi-modality sensory feedback device, incorporating mechanotactile and vibrotactile feedback can increase the subjects’ performance in localization and intensity discrimination.To the best of our knowledge, this is the first attempt combining mechanotactile and vibrotactile into a single sebsory feedback device. For persons without maps of referred sensation, the localization of the stimulation has to be memorized and predicted from previous stimulation. The outcome of this study shows that the hybrid stimulation relieves the mental load, but in future work activating both modalities simultaneously could increase haptic vocabulary. Hybrid stimulation also improves the performance for subjects with maps of referred sensation,

Automatic hand phantom map generation and detection using decomposition support vector machines

Background: There is a need for providing sensory feedback for myoelectric prosthesis users. Providing tactile feedback can improve object manipulation abilities, enhance the perceptual embodiment of myoelectric prostheses and help reduce phantom limb pain. Many amputees have referred sensation from their missing hand on their residual limbs (phantom maps). This skin area can serve as a target for providing amputees with non-invasive tactile sensory feedback. One of the challenges of providing sensory feedback on the phantom map is to define the accurate boundary of each phantom digit because the phantom map distribution varies from person to person. Methods: In this paper, automatic phantom map detection methods based on four decomposition support vector machine algorithms and three sampling methods are proposed, complemented by fuzzy logic and active learning strategies. The algorithms and methods are tested on two databases: the first one includes 400 generated phantom maps, whereby the phantom map generation algorithm was based on our observation of the phantom maps to ensure smooth phantom digit edges, variety, and representativeness. The second database includes five reported phantom map images and transformations thereof. The accuracy and training/ classification time of each algorithm using a dense stimulation array (with 100 $$\times$$ × 100 actuators) and two coarse stimulation arrays (with 3 $$\times$$ × 5 and 4 $$\times$$ × 6 actuators) are presented and compared. Results: Both generated and reported phantom map images share the same trends. Majority-pooling sampling effectively increases the training size, albeit introducing some noise, and thus produces the smallest error rates among the three proposed sampling methods. For different decomposition architectures, one-vs-one reduces unclassified regions and in general has higher classification accuracy than the other architectures. By introducing fuzzy logic to bias the penalty parameter, the influence of pooling-induced noise is reduced. Moreover, active learning with different strategies was also tested and shown to improve the accuracy by introducing more representative training samples. Overall, dense arrays employing one-vs-one fuzzy support vector machines with majority-pooling sampling have the smallest average absolute error rate (8.78% for generated phantom maps and 11.5% for reported and transformed phantom map images). The detection accuracy of coarse arrays was found to be significantly lower than for dense array. Conclusions: The results demonstrate the effectiveness of support vector machines using a dense array in detecting refined phantom map shapes, whereas coarse arrays are unsuitable for this task. We therefore propose a two-step approach, using first a non-wearable dense array to detect an accurate phantom map shape, then to apply a wearable coarse stimulation array customized according to the detection results. The proposed methodology can be used as a tool for helping haptic feedback designers and for tracking the evolvement of phantom maps

A Miniature Multimodal Actuator for Effective Tactile Feedback: Design and Characterization

We designed a miniature multimodal actuator by combining servomotors and vibrators. The actuator is mainly designed to non-invasively deliver tactile information to human skin by high-frequency vibration and low-frequency pressures simultaneously. By utilizing two modalities at the same time, the actuator stimulates different types of mechanoreceptors and thus has the potential to increase the band width to transfer tactile information. The designed actuator possess the advantage of light weight, compact size, low power consumption, short response time, large ranges of pressure and vibration, and non-back-drivable mechanism. Multimodal actuators provide new tools and possibilities to display tactile information on human skin efficiently and intuitively. (C) 2016 The Authors. Published by Elsevier Ltd

PCA and deep learning based myoelectric grasping control of a prosthetic hand

Background For the functional control of prosthetic hand, it is insufficient to obtain only the motion pattern information. As far as practicality is concerned, the control of the prosthetic hand force is indispensable. The application value of prosthetic hand will be greatly improved if the stable grip of prosthetic hand can be achieved. To address this problem, in this study, a bio-signal control method for grasping control of a prosthetic hand is proposed to improve patient’s sense of using prosthetic hand and the thus improving the quality of life. Methods A MYO gesture control armband is used to collect the surface electromyographic (sEMG) signals from the upper limb. The overlapping sliding window scheme are applied for data segmentation and the correlated features are extracted from each segmented data. Principal component analysis (PCA) methods are then deployed for dimension reduction. Deep neural network is used to generate sEMG-force regression model for force prediction at different levels. The predicted force values are input to a fuzzy controller for the grasping control of a prosthetic hand. A vibration feedback device is used to feed grasping force value back to patient’s arm to improve patient’s sense of using prosthetic hand and realize accurate grasping. To test the effectiveness of the scheme, 15 able-bodied subjects participated in the experiments. Results The classification results indicated that 8-channel sEMG applying all four time-domain features, with PCA reduction from 32 to 8 dimensions results in the highest classification accuracy. Based on the experimental results from 15 participants, the average recognition rate is over 95%. On the other hand, from the statistical results of standard deviation, the between-subject variations ranges from 3.58 to 1.25%, proving that the robustness and stability of the proposed approach. Conclusions The method proposed hereto control grasping power through the patient’s own sEMG signal, which achieves a high recognition rate to improve the success rate of grip and increases the sense of operation and also brings the gospel for upper extremity amputation patients

Novel Yersinia enterocolitica Prophages and a Comparative Analysis of Genomic Diversity

Yersinia enterocolitica is a major agent of foodborne diseases worldwide. Prophage plays an important role in the genetic evolution of the bacterial genome. Little is known about the genetic information about prophages in the genome of Y. enterocolitica, and no pathogenic Y. enterocolitica prophages have been described. In this study, we induced and described the genomes of six prophages from pathogenic Y. enterocolitica for the first time. Phylogenetic analysis based on whole genome sequencing revealed that these novel Yersinia phages are genetically distinct from the previously reported phages, showing considerable genetic diversity. Interestingly, the prophages induced from O:3 and O:9 Y. enterocolitica showed different genomic sequences and morphology but highly conserved among the same serotype strains, which classified into two diverse clusters. The three long-tailed Myoviridae prophages induced from serotype O:3 Y. enterocolitica were highly conserved, shared ≥99.99% identity and forming genotypic cluster A; the three Podoviridae prophages induced from the serotype O:9 strains formed cluster B, also shared more than 99.90% identity with one another. Cluster A was most closely related to O:5 non-pathogenic Y. enterocolitica prophage PY54 (61.72% identity). The genetic polymorphism of these two kinds of prophages and highly conserved among the same serotype strains, suggested a possible shared evolutionary past for these phages: originated from distinct ancestors, and entered pathogenic Y. enterocolitica as extrachromosomal genetic components during evolution when facing selective pressure. These results are critically important for further understanding of phage roles in host physiology and the pathology of disease

Human Streptococcus suis Outbreak, Sichuan, China

Streptococcus suis outbreak was associated with exposure to sick or dead pigs

Multi-modality Sensory Feedback: a pilot study

The aim of this study is to test the hypotheses that a multi-modality sensory feedback device, incorporating mechanotactile and vibrotactile feedback can increase the subjects’ performance in localization and intensity discrimination.To the best of our knowledge, this is the first attempt combining mechanotactile and vibrotactile into a single sebsory feedback device. For persons without maps of referred sensation, the localization of the stimulation has to be memorized and predicted from previous stimulation. The outcome of this study shows that the hybrid stimulation relieves the mental load, but in future work activating both modalities simultaneously could increase haptic vocabulary. Hybrid stimulation also improves the performance for subjects with maps of referred sensation

Vibrotactile Sensory Feedback System for Upper Limb Amputees

Although during the last decades, the dexterity of active myoelectric-controlled prosthesis has made significant progress, there is still no or very limited sensory feedback in the commercial prosthesis. Sensory feedback is important for active prosthesis users because it can not only increase grasping performance but also introduce an embodiment feeling to the amputee user. There has been some research focused on providing non-invasive sensory feedback to amputees because non-invasive feedback has higher user acceptance, compared to invasive ones. For non-invasive sensory feedback, vibrotactile was widely used for its relative small size, light weight, and low power consumption. A sensory feedback system were designed, incorporating pressure sensors, wireless communication modules, and a non-invasive haptic display. The flexible skin made of TangoBlack was attached to the robotic hand. Five miniaturized pressure sensors were embedded in the skin, one on each finger. The sensed pressure data were transmitted by custom-designed wireless communication modules to the haptic display control module. The sensory feedback was delivered by five pancake-shaped eccentric rotating mass (ERM) embedded in the socket. The distribution of ERMs corresponds to the shape of the phantom fingers. The vibrational amplitude was proportional to the sensed pressure. This system was tested on one amputee with phantom map. The finger identification test and handling fragile objects task were conducted. For the first one, the amputee was blind folded and wearing a headphone to eliminate visual and audio cues. The experimenter pressed the robotic finger and the subject answered which finger he felt being touched. The amputee could answer all the fingers correctly for all the trials. Then the subject was asked to grasp eggs and move them to a small basket. The amputee could move all the eggs without dropping or breaking any. There were no detectable interference between the feedback and EMG sensors

Tactile display on the remaining hand for unilateral hand amputees

Human rely profoundly on tactile feedback from fingertips to interact with the environment, whereas most hand prostheses used in clinics provide no tactile feedback. In this study we demonstrate the feasibility to use a tactile display glove that can be worn by a unilateral hand amputee on the remaining healthy hand to display tactile feedback from a hand prosthesis. The main benefit is that users could easily distinguish the feedback for each finger, even without training. The claimed advantage is supported by preliminary tests with healthy subjects. This approach may lead to the development of effective and affordable tactile display devices that provide tactile feedback for individual fingertip of hand prostheses