Novel soft bending actuator based power augmentation hand exoskeleton controlled by human intention

This article presents the development of a soft material power augmentation wearable robot using novel bending soft artificial muscles. This soft exoskeleton was developed as a human hand power augmentation system for healthy or partially hand disabled individuals. The proposed prototype serves healthy manual workers by decreasing the muscular effort needed for grasping objects. Furthermore, it is a power augmentation wearable robot for partially hand disabled or post-stroke patients, supporting and augmenting the fingers’ grasping force with minimum muscular effort in most everyday activities. This wearable robot can fit any adult hand size without the need for any mechanical system changes or calibration. Novel bending soft actuators are developed to actuate this power augmentation device. The performance of these actuators has been experimentally assessed. A geometrical kinematic analysis and mathematical output force model have been developed for the novel actuators. The performance of this mathematical model has been proven experimentally with promising results. The control system of this exoskeleton is created by hybridization between cascaded position and force closed loop intelligent controllers. The cascaded position controller is designed for the bending actuators to follow the fingers in their bending movements. The force controller is developed to control the grasping force augmentation. The operation of the control system with the exoskeleton has been experimentally validated. EMG signals were monitored during the experiments to determine that the proposed exoskeleton system decreased the muscular efforts of the wearer

Estimation and Early Prediction of Grip Force Based on sEMG Signals and Deep Recurrent Neural Networks

Hands are used for communicating with the surrounding environment and have a complex structure that enables them to perform various tasks with their multiple degrees of freedom. Hand amputation can prevent a person from performing their daily activities. In that event, finding a suitable, fast, and reliable alternative for the missing limb can affect the lives of people who suffer from such conditions. As the most important use of the hands is to grasp objects, the purpose of this study is to accurately predict gripping force from surface electromyography (sEMG) signals during a pinch-type grip. In that regard, gripping force and sEMG signals are derived from 10 healthy subjects. Results show that for this task, recurrent networks outperform nonrecurrent ones, such as a fully connected multilayer perceptron (MLP) network. Gated recurrent unit (GRU) and long short-term memory (LSTM) networks can predict the gripping force with R-squared values of 0.994 and 0.992, respectively, and a prediction rate of over 1300 predictions per second. The predominant advantage of using such frameworks is that the gripping force can be predicted straight from preprocessed sEMG signals without any form of feature extraction, not to mention the ability to predict future force values using larger prediction horizons adequately. The methods presented in this study can be used in the myoelectric control of prosthetic hands or robotic grippers.Comment: 9 pages, accepted for publication in journal of the Brazilian Society of Mechanical Sciences and Engineerin

Relating forearm muscle electrical activity to finger forces

The electromyogram (EMG) signal is desired to be used as a control signal for applications such as multifunction prostheses, wheelchair navigation, gait generation, grasping control, virtual keyboards, and gesture-based interfaces [25]. Several research studies have attempted to relate the electromyogram (EMG) activity of the forearm muscles to the mechanical activity of the wrist, hand and/or fingers [41], [42], [43]. A primary interest is for EMG control of powered upper-limb prostheses and rehabilitation orthotics. Existing commercial EMG-controlled devices are limited to rudimentary control capabilities of either discrete states (e.g. hand close/open), or one degree of freedom proportional control [4], [36]. Classification schemes for discriminating between hand/wrist functions and individual finger movements have demonstrated accuracy up to 95% [38], [39], [29]. These methods may provide for increased amputee function, though continuous control of movement is not generally achieved. This thesis considered proportional control via EMG-based estimation of finger forces with the goal of identifying whether multiple degrees of freedom of proportional control information are available from the surface EMG of the forearm. Electromyogram (EMG) activity from the extensor and flexor muscles of the forearm was sensed with bipolar surface electrodes and related to the force produced at the four fingertips during constant-posture, slowly force-varying contractions from 20 healthy subjects. The contractions ranged between 30% maximum voluntary contractions (MVC) extension and 30% MVC flexion. EMG amplitude sampling rate, least squares regularization, linear vs. nonlinear models and number of electrodes used in the system identification were studied. Results are supportive that multiple degrees of freedom of proportional control information are available from the surface EMG of the forearm, at least in healthy subjects. An EMG amplitude sampling frequency of 4.096 Hz was found to produce models which allowed for good EMG amplitude estimates. Least squares regularization with a pseudo-inverse tolerance of 0.055 resulted in significant improvement in modeling results, with an average error of 4.69% MVC-6.59% MVC (maximum voluntary contraction). Increasing polynomial order did not significantly improve modeling results. Results from smaller electrode arrays remained fairly good with as few as six electrodes, with the average %MVC error ranging from 5.13%-7.01% across the four fingers. This study also identified challenges in the current experimental study design and subsequent system identification when EMG-force modeling is performed with four fingers simultaneously. Methods to compensate for these issues have been proposed in this thesis

Subject-Independent Frameworks for Robotic Devices: Applying Robot Learning to EMG Signals

The capability of having human and robots cooperating together has increased the interest in the control of robotic devices by means of physiological human signals. In order to achieve this goal it is crucial to be able to catch the human intention of movement and to translate it in a coherent robot action. Up to now, the classical approach when considering physiological signals, and in particular EMG signals, is to focus on the specific subject performing the task since the great complexity of these signals. This thesis aims to expand the state of the art by proposing a general subject-independent framework, able to extract the common constraints of human movement by looking at several demonstration by many different subjects. The variability introduced in the system by multiple demonstrations from many different subjects allows the construction of a robust model of human movement, able to face small variations and signal deterioration. Furthermore, the obtained framework could be used by any subject with no need for long training sessions. The signals undergo to an accurate preprocessing phase, in order to remove noise and artefacts. Following this procedure, we are able to extract significant information to be used in online processes. The human movement can be estimated by using well-established statistical methods in Robot Programming by Demonstration applications, in particular the input can be modelled by using a Gaussian Mixture Model (GMM). The performed movement can be continuously estimated with a Gaussian Mixture Regression (GMR) technique, or it can be identified among a set of possible movements with a Gaussian Mixture Classification (GMC) approach. We improved the results by incorporating some previous information in the model, in order to enriching the knowledge of the system. In particular we considered the hierarchical information provided by a quantitative taxonomy of hand grasps. Thus, we developed the first quantitative taxonomy of hand grasps considering both muscular and kinematic information from 40 subjects. The results proved the feasibility of a subject-independent framework, even by considering physiological signals, like EMG, from a wide number of participants. The proposed solution has been used in two different kinds of applications: (I) for the control of prosthesis devices, and (II) in an Industry 4.0 facility, in order to allow human and robot to work alongside or to cooperate. Indeed, a crucial aspect for making human and robots working together is their mutual knowledge and anticipation of other’s task, and physiological signals are capable to provide a signal even before the movement is started. In this thesis we proposed also an application of Robot Programming by Demonstration in a real industrial facility, in order to optimize the production of electric motor coils. The task was part of the European Robotic Challenge (EuRoC), and the goal was divided in phases of increasing complexity. This solution exploits Machine Learning algorithms, like GMM, and the robustness was assured by considering demonstration of the task from many subjects. We have been able to apply an advanced research topic to a real factory, achieving promising results

Towards electrodeless EMG linear envelope signal recording for myo-activated prostheses control

After amputation, the residual muscles of the limb may function in a normal way, enabling the electromyogram (EMG) signals recorded from them to be used to drive a replacement limb. These replacement limbs are called myoelectric prosthesis. The prostheses that use EMG have always been the first choice for both clinicians and engineers. Unfortunately, due to the many drawbacks of EMG (e.g. skin preparation, electromagnetic interferences, high sample rate, etc.); researchers have aspired to find suitable alternatives. One proposes the dry-contact, low-cost sensor based on a force-sensitive resistor (FSR) as a valid alternative which instead of detecting electrical events, detects mechanical events of muscle. FSR sensor is placed on the skin through a hard, circular base to sense the muscle contraction and to acquire the signal. Similarly, to reduce the output drift (resistance) caused by FSR edges (creep) and to maintain the FSR sensitivity over a wide input force range, signal conditioning (Voltage output proportional to force) is implemented. This FSR signal acquired using FSR sensor can be used directly to replace the EMG linear envelope (an important control signal in prosthetics applications). To find the best FSR position(s) to replace a single EMG lead, the simultaneous recording of EMG and FSR output is performed. Three FSRs are placed directly over the EMG electrodes, in the middle of the targeted muscle and then the individual (FSR1, FSR2 and FSR3) and combination of FSR (e.g. FSR1+FSR2, FSR2-FSR3) is evaluated. The experiment is performed on a small sample of five volunteer subjects. The result shows a high correlation (up to 0.94) between FSR output and EMG linear envelope. Consequently, the usage of the best FSR sensor position shows the ability of electrode less FSR-LE to proportionally control the prosthesis (3-D claw). Furthermore, FSR can be used to develop a universal programmable muscle signal sensor that can be suitable to control the myo-activated prosthesis

Force-Aware Interface via Electromyography for Natural VR/AR Interaction

While tremendous advances in visual and auditory realism have been made for virtual and augmented reality (VR/AR), introducing a plausible sense of physicality into the virtual world remains challenging. Closing the gap between real-world physicality and immersive virtual experience requires a closed interaction loop: applying user-exerted physical forces to the virtual environment and generating haptic sensations back to the users. However, existing VR/AR solutions either completely ignore the force inputs from the users or rely on obtrusive sensing devices that compromise user experience. By identifying users' muscle activation patterns while engaging in VR/AR, we design a learning-based neural interface for natural and intuitive force inputs. Specifically, we show that lightweight electromyography sensors, resting non-invasively on users' forearm skin, inform and establish a robust understanding of their complex hand activities. Fuelled by a neural-network-based model, our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. Through an interactive psychophysical study, we show that human perception of virtual objects' physical properties, such as stiffness, can be significantly enhanced by our interface. We further demonstrate that our interface enables ubiquitous control via finger tapping. Ultimately, we envision our findings to push forward research towards more realistic physicality in future VR/AR.Comment: ACM Transactions on Graphics (SIGGRAPH Asia 2022

Assistive Device For Elderly Rehabilitation: Signal Processing Techniques.

Ph.DDOCTOR OF PHILOSOPH

Real-time EMG based pattern recognition control for hand prostheses : a review on existing methods, challenges and future implementation

Upper limb amputation is a condition that significantly restricts the amputees from performing their daily activities. The myoelectric prosthesis, using signals from residual stump muscles, is aimed at restoring the function of such lost limbs seamlessly. Unfortunately, the acquisition and use of such myosignals are cumbersome and complicated. Furthermore, once acquired, it usually requires heavy computational power to turn it into a user control signal. Its transition to a practical prosthesis solution is still being challenged by various factors particularly those related to the fact that each amputee has different mobility, muscle contraction forces, limb positional variations and electrode placements. Thus, a solution that can adapt or otherwise tailor itself to each individual is required for maximum utility across amputees. Modified machine learning schemes for pattern recognition have the potential to significantly reduce the factors (movement of users and contraction of the muscle) affecting the traditional electromyography (EMG)-pattern recognition methods. Although recent developments of intelligent pattern recognition techniques could discriminate multiple degrees of freedom with high-level accuracy, their efficiency level was less accessible and revealed in real-world (amputee) applications. This review paper examined the suitability of upper limb prosthesis (ULP) inventions in the healthcare sector from their technical control perspective. More focus was given to the review of real-world applications and the use of pattern recognition control on amputees. We first reviewed the overall structure of pattern recognition schemes for myo-control prosthetic systems and then discussed their real-time use on amputee upper limbs. Finally, we concluded the paper with a discussion of the existing challenges and future research recommendations