Prototyping a Capacitive Sensing Device for Gesture Recognition

Capacitive sensing is a technology that can detect proximity and touch. It can also be utilized to measure position and acceleration of gesture motions. This technology has many applications, such as replacing mechanical buttons in a gaming device interface, detecting respiration rate without direct contact with the skin, and providing gesture sensing capability for rehabilitation devices. In this thesis, an approach to prototype a capacitive gesture sensing device using the Eagle PCB design software is demonstrated. In addition, this paper tested and evaluated the resulting prototype device, validating the effectiveness of the approach

Stay-At-Home Motor Rehabilitation: Optimizing Spatiotemporal Learning on Low-Cost Capacitive Sensor Arrays

Repeated, consistent, and precise gesture performance is a key part of recovery for stroke and other motor-impaired patients. Close professional supervision to these exercises is also essential to ensure proper neuromotor repair, which consumes a large amount of medical resources. Gesture recognition systems are emerging as stay-at-home solutions to this problem, but the best solutions are expensive, and the inexpensive solutions are not universal enough to tackle patient-to-patient variability. While many methods have been studied and implemented, the gesture recognition system designer does not have a strategy to effectively predict the right method to fit the needs of a patient. This thesis establishes such a strategy by outlining the strengths and weaknesses of several spatiotemporal learning architectures combined with deep learning, specifically when low-cost, low-resolution capacitive sensor arrays are used. This is done by testing the immunity and robustness of those architectures to the type of variability that is common among stroke patients, investigating select hyperparameters and their impact on the architectures’ training progressions, and comparing test performance in different applications and scenarios. The models analyzed here are trained on a mixture of high-quality, healthy gestures and personalized, imperfectly performed gestures using a low-cost recognition system

Development of Gait Rehabilitation System Capable of Assisting Pelvic Movement of Normal Walking

Gait rehabilitation training with robotic exoskeleton is drawing attention as a method for more advanced gait rehabilitation training. However, most of the rehabilitation robots are mainly focused on locomotion training in the sagittal plane. This study introduces a novel gait rehabilitation system with actuated pelvic motion to generate natural gait motion. The rehabilitation robot developed in this study, COWALK, is a lower-body exoskeleton system with 15 degrees of freedom (DoFs). The COWALK can generate multi-DoF pelvic movement along with leg movements. To produce natural gait patterns, the actuation of pelvic movement is essential. In the COWALK, the pelvic movement mechanism is designed to help hemiplegic patients regain gait balance during gait training. To verify the effectiveness of the developed system, the gait patterns with and without pelvic movement were compared to the normal gait on a treadmill. The experimental results show that the active control of pelvic movement combined with the active control of leg movement can make the gait pattern much more natural

Distributed Sensing and Stimulation Systems Towards Sense of Touch Restoration in Prosthetics

Modern prostheses aim at restoring the functional and aesthetic characteristics of the lost limb. To foster prosthesis embodiment and functionality, it is necessary to restitute both volitional control and sensory feedback. Contemporary feedback interfaces presented in research use few sensors and stimulation units to feedback at most two discrete feedback variables (e.g. grasping force and aperture), whereas the human sense of touch relies on a distributed network of mechanoreceptors providing high-fidelity spatial information. To provide this type of feedback in prosthetics, it is necessary to sense tactile information from artificial skin placed on the prosthesis and transmit tactile feedback above the amputation in order to map the interaction between the prosthesis and the environment. This thesis proposes the integration of distributed sensing systems (e-skin) to acquire tactile sensation, and non-invasive multichannel electrotactile feedback and virtual reality to deliver high-bandwidth information to the user. Its core focus addresses the development and testing of close-loop sensory feedback human-machine interface, based on the latest distributed sensing and stimulation techniques for restoring the sense of touch in prosthetics. To this end, the thesis is comprised of two introductory chapters that describe the state of art in the field, the objectives and the used methodology and contributions; as well as three studies distributed over stimulation system level and sensing system level. The first study presents the development of close-loop compensatory tracking system to evaluate the usability and effectiveness of electrotactile sensory feedback in enabling real-time close-loop control in prosthetics. It examines and compares the subject\u2019s adaptive performance and tolerance to random latencies while performing the dynamic control task (i.e. position control) and simultaneously receiving either visual feedback or electrotactile feedback for communicating the momentary tracking error. Moreover, it reported the minimum time delay needed for an abrupt impairment of users\u2019 performance. The experimental results have shown that electrotactile feedback performance is less prone to changes with longer delays. However, visual feedback drops faster than electrotactile with increased time delays. This is a good indication for the effectiveness of electrotactile feedback in enabling close- loop control in prosthetics, since some delays are inevitable. The second study describes the development of a novel non-invasive compact multichannel interface for electrotactile feedback, containing 24 pads electrode matrix, with fully programmable stimulation unit, that investigates the ability of able-bodied human subjects to localize the electrotactile stimulus delivered through the electrode matrix. Furthermore, it designed a novel dual parameter -modulation (interleaved frequency and intensity) and compared it to conventional stimulation (same frequency for all pads). In addition and for the first time, it compared the electrotactile stimulation to mechanical stimulation. More, it exposes the integration of virtual prosthesis with the developed system in order to achieve better user experience and object manipulation through mapping the acquired real-time collected tactile data and feedback it simultaneously to the user. The experimental results demonstrated that the proposed interleaved coding substantially improved the spatial localization compared to same-frequency stimulation. Furthermore, it showed that same-frequency stimulation was equivalent to mechanical stimulation, whereas the performance with dual-parameter modulation was significantly better. The third study presents the realization of a novel, flexible, screen- printed e-skin based on P(VDF-TrFE) piezoelectric polymers, that would cover the fingertips and the palm of the prosthetic hand (particularly the Michelangelo hand by Ottobock) and an assistive sensorized glove for stroke patients. Moreover, it developed a new validation methodology to examine the sensors behavior while being solicited. The characterization results showed compatibility between the expected (modeled) behavior of the electrical response of each sensor to measured mechanical (normal) force at the skin surface, which in turn proved the combination of both fabrication and assembly processes was successful. This paves the way to define a practical, simplified and reproducible characterization protocol for e-skin patches In conclusion, by adopting innovative methodologies in sensing and stimulation systems, this thesis advances the overall development of close-loop sensory feedback human-machine interface used for restoration of sense of touch in prosthetics. Moreover, this research could lead to high-bandwidth high-fidelity transmission of tactile information for modern dexterous prostheses that could ameliorate the end user experience and facilitate it acceptance in the daily life

Touching on elements for a non-invasive sensory feedback system for use in a prosthetic hand

Hand amputation results in the loss of motor and sensory functions, impacting activities of daily life and quality of life. Commercially available prosthetic hands restore the motor function but lack sensory feedback, which is crucial to receive information about the prosthesis state in real-time when interacting with the external environment. As a supplement to the missing sensory feedback, the amputee needs to rely on visual and audio cues to operate the prosthetic hand, which can be mentally demanding. This thesis revolves around finding potential solutions to contribute to an intuitive non-invasive sensory feedback system that could be cognitively less burdensome and enhance the sense of embodiment (the feeling that an artificial limb belongs to one’s own body), increasing acceptance of wearing a prosthesis.A sensory feedback system contains sensors to detect signals applied to the prosthetics. The signals are encoded via signal processing to resemble the detected sensation delivered by actuators on the skin. There is a challenge in implementing commercial sensors in a prosthetic finger. Due to the prosthetic finger’s curvature and the fact that some prosthetic hands use a covering rubber glove, the sensor response would be inaccurate. This thesis shows that a pneumatic touch sensor integrated into a rubber glove eliminates these errors. This sensor provides a consistent reading independent of the incident angle of stimulus, has a sensitivity of 0.82 kPa/N, a hysteresis error of 2.39±0.17%, and a linearity error of 2.95±0.40%.For intuitive tactile stimulation, it has been suggested that the feedback stimulus should be modality-matched with the intention to provide a sensation that can be easily associated with the real touch on the prosthetic hand, e.g., pressure on the prosthetic finger should provide pressure on the residual limb. A stimulus should also be spatially matched (e.g., position, size, and shape). Electrotactile stimulation has the ability to provide various sensations due to it having several adjustable parameters. Therefore, this type of stimulus is a good candidate for discrimination of textures. A microphone can detect texture-elicited vibrations to be processed, and by varying, e.g., the median frequency of the electrical stimulation, the signal can be presented on the skin. Participants in a study using electrotactile feedback showed a median accuracy of 85% in differentiating between four textures.During active exploration, electrotactile and vibrotactile feedback provide spatially matched modality stimulations, providing continuous feedback and providing a displaced sensation or a sensation dispatched on a larger area. Evaluating commonly used stimulation modalities using the Rubber Hand Illusion, modalities which resemble the intended sensation provide a more vivid illusion of ownership for the rubber hand.For a potentially more intuitive sensory feedback, the stimulation can be somatotopically matched, where the stimulus is experienced as being applied on a site corresponding to their missing hand. This is possible for amputees who experience referred sensation on their residual stump. However, not all amputees experience referred sensations. Nonetheless, after a structured training period, it is possible to learn to associate touch with specific fingers, and the effect persisted after two weeks. This effect was evaluated on participants with intact limbs, so it remains to evaluate this effect for amputees.In conclusion, this thesis proposes suggestions on sensory feedback systems that could be helpful in future prosthetic hands to (1) reduce their complexity and (2) enhance the sense of body ownership to enhance the overall sense of embodiment as an addition to an intuitive control system

Soft Gloves: A Review on Recent Developments in Actuation, Sensing, Control and Applications

Interest in soft gloves, both robotic and haptic, has enormously grown over the past decade, due to their inherent compliance, which makes them particularly suitable for direct interaction with the human hand. Robotic soft gloves have been developed for hand rehabilitation, for ADLs assistance, or sometimes for both. Haptic soft gloves may be applied in virtual reality (VR) applications or to give sensory feedback in combination with prostheses or to control robots. This paper presents an updated review of the state of the art of soft gloves, with a particular focus on actuation, sensing, and control, combined with a detailed analysis of the devices according to their application field. The review is organized on two levels: a prospective review allows the highlighting of the main trends in soft gloves development and applications, and an analytical review performs an in-depth analysis of the technical solutions developed and implemented in the revised scientific research. Additional minor evaluations integrate the analysis, such as a synthetic investigation of the main results in the clinical studies and trials referred in literature which involve soft gloves

Evaluation of Individual Finger Forces During Activities of Daily Living In Healthy Individuals and Those with Hand Arthritis

Hand-Osteoarthritis (H-OA) leads to pain, loss of grip strength, and decreased hand function. Current treatment for H-OA involves joint protection programs (JPP) which seek to reduce joint loading during activity. The use of wearable technology to measure hand forces during activity has the potential to determine the effectiveness of JPP. The objective of this thesis was to develop and validate a method of directly measuring finger forces during the performance of activities of daily living, and then use that system to measure the envelope of hand forces during activity in healthy individuals and in patients with H-OA. A commercially-available capacitive sensor system was validated for use in this application and found an envelope of applied forces consistent with previous literature. Using the measurement system and protocols presented in this thesis, the effectiveness of JPP at reducing hand forces can, for the first time, be objectively quantified

A Biomechanical and Physiological Signal Monitoring System for Four Degrees of Upper Limb Movement

A lack of adherence to prescribed physical therapy regimens in improper healing results in poor outcomes for those affected by musculoskeletal disorders (MSDs) of the upper limb. Societal and psychological barriers to proper adherence can be addressed through the system presented in this work consisting of the following components: an ambulatory biosignal acquisition sleeve, an electromyography (EMG) based motion repetition detection algorithm, and the design of a compatible capacitive EMG acquisition module. The biosignal acquisition sleeve was untethered, unobtrusive to motion, contained only modular components, and collected biomechanical and physiological sensor data to form full motion profiles of the following four degrees of freedom: elbow flexion—extension, forearm pronation—supination, wrist flexion—extension, and ulnar--radial deviation. The piloted sleeve simultaneously collected data from four inertial sensors, two electromyography (EMG) sensors and a flex-bend sensor. A visualization application was developed to present the information in a manner meaningful to the user. As well, an EMG based motion repetition detector was developed for use within the system. It was validated using an existing database of 23 subjects with varying musculoskeletal health, achieving a success rate of 95.43%. This algorithm was modified for use with the sleeve, resulting in a 95% success rate. An electrode and analog front end module was proposed, relying on unique material structures and low-noise, precision sensing techniques. The system prototype presented a resource-conscious tool for multi-modality tracking of elbow, forearm, and wrist motion, which could eventually be integrated into upper limb MSD rehabilitation

Investigation of novel control strategies for promoting motor learning in the upper limb with a haptic computer exercise system in able-bodied adults and those with motor impairments

Motor impairments caused by stroke and cerebral palsy (CP) are common and often affect the function of the upper limb, which to be restored requires rehabilitation. As positive outcome is correlated to how early and intensive therapy is and since the resources of the healthcare providers are limited, robotic devices have been introduced to provide adjunctive therapy. The algorithms that control the manner those devices apply forces to the impaired limb are called haptic control algorithms (HCA) and to this date there has not been conclusive evidence as to what the behaviour of these algorithms should be. One type of HCAs is error augmentation (EA) which is a rather understudied but promising approach. This work presents to the literature two novel control strategies of the EA type that incorporate adaptive features namely Error Augmenting Adaptive(EA) and Error Augmenting Proportional (EA). Those two algorithms were implemented for and deployed to a single point of attachment robotic rehabilitation system. The effectiveness in inducing motor learning of the developed algorithms was evaluated in a trial with able-bodied participants and compared against a third more established assistive HCA namely Assistance As Needed (AAN) and a control condition (no forces). Four groups (one per condition) practised reaching movements with a speed and accuracy requirement using their non-dominant arm to interact with the robot under a visual rotation of a 100o. To assess learning kinematic measures were collected to measure their performance on reaching and circle-drawing movements. Also, bilateral transfer to the arm that did not receive practice was assessed. Changes in the participants’ valence, arousal and dominance were assessed with a Self-Assessment Manikin questionnaire. All groups learned to move their non-dominant arm under a visual perturbation showing comparable improvements in all key measures (p<0.05). Passive movements and EAP led to greater improvement in movement smoothness (p<0.05) and resulted in more retention of the improvements after a washout block (p<0.05) was introduced. Conversely, EAA showed a better effect on improving mean velocity (p<0.05). All groups performed similarly in terms of improving movement error and duration but EAA and AAN achieved peak performance faster (p<0.05). Similar improvements were measured on the arm that did not receive any training which were fully retained post-washout indicating that bilateral transfer occurred and led to better retention (p<0.05). The findings of this work indicate that different attributes can be exploited from the developed HCAs to induce motor learning and improve different aspects of the movement suggesting that multimodal training protocols tailored to the needs of the patient are the way forward. Also, this work showed that bilateral transfer training has great potential in upper limb rehabilitation and the positive effects of the different HCAs on the arm that received practice transfer to the one that did not receive training. It is recommended that the findings of this work to be further investigated in experimental therapy protocols for those who suffer from neurological impairments such stroke and CP