123 research outputs found
Recommended from our members
Thumbs up, thumbs down:non-verbal human-robot interaction through real-time EMG classification via inductive and supervised transductive transfer learning
In this study, we present a transfer learning method for gesture classification via an inductive and supervised transductive approach with an electromyographic dataset gathered via the Myo armband. A ternary gesture classification problem is presented by states of ’thumbs up’, ’thumbs down’, and ’relax’ in order to communicate in the affirmative or negative in a non-verbal fashion to a machine. Of the nine statistical learning paradigms benchmarked over 10-fold cross validation (with three methods of feature selection), an ensemble of Random Forest and Support Vector Machine through voting achieves the best score of 91.74% with a rule-based feature selection method. When new subjects are considered, this machine learning approach fails to generalise new data, and thus the processes of Inductive and Supervised Transductive Transfer Learning are introduced with a short calibration exercise (15 s). Failure of generalisation shows that 5 s of data per-class is the strongest for classification (versus one through seven seconds) with only an accuracy of 55%, but when a short 5 s per class calibration task is introduced via the suggested transfer method, a Random Forest can then classify unseen data from the calibrated subject at an accuracy of around 97%, outperforming the 83% accuracy boasted by the proprietary Myo system. Finally, a preliminary application is presented through social interaction with a humanoid Pepper robot, where the use of our approach and a most-common-class metaclassifier achieves 100% accuracy for all trials of a ‘20 Questions’ game
A Transferable Adaptive Domain Adversarial Neural Network for Virtual Reality Augmented EMG-Based Gesture Recognition
Within the field of electromyography-based (EMG) gesture recognition,
disparities exist between the offline accuracy reported in the literature and
the real-time usability of a classifier. This gap mainly stems from two
factors: 1) The absence of a controller, making the data collected dissimilar
to actual control. 2) The difficulty of including the four main dynamic factors
(gesture intensity, limb position, electrode shift, and transient changes in
the signal), as including their permutations drastically increases the amount
of data to be recorded. Contrarily, online datasets are limited to the exact
EMG-based controller used to record them, necessitating the recording of a new
dataset for each control method or variant to be tested. Consequently, this
paper proposes a new type of dataset to serve as an intermediate between
offline and online datasets, by recording the data using a real-time
experimental protocol. The protocol, performed in virtual reality, includes the
four main dynamic factors and uses an EMG-independent controller to guide
movements. This EMG-independent feedback ensures that the user is in-the-loop
during recording, while enabling the resulting dynamic dataset to be used as an
EMG-based benchmark. The dataset is comprised of 20 able-bodied participants
completing three to four sessions over a period of 14 to 21 days. The ability
of the dynamic dataset to serve as a benchmark is leveraged to evaluate the
impact of different recalibration techniques for long-term (across-day) gesture
recognition, including a novel algorithm, named TADANN. TADANN consistently and
significantly (p<0.05) outperforms using fine-tuning as the recalibration
technique.Comment: 10 Pages. The last three authors shared senior authorshi
A transferable adaptive domain adversarial neural network for virtual reality augmented EMG-Based gesture recognition
Within the field of electromyography-based (EMG) gesture recognition, disparities exist between the off line accuracy reported in the literature and the real-time usability of a classifier. This gap mainly stems from two factors: 1) The absence of a controller, making the data collected dissimilar to actual control. 2) The difficulty of including the four main dynamic factors (gesture intensity, limb position, electrode shift, and transient changes in the signal), as including their permutations drastically increases the amount of data to be recorded. Contrarily, online datasets are limited to the exact EMG-based controller used to record them, necessitating the recording of a new dataset for each control method or variant to be tested. Consequently, this paper proposes a new type of dataset to serve as an intermediate between off line and online datasets, by recording the data using a real-time experimental protocol. The protocol, performed in virtual reality, includes the four main dynamic factors and uses an EMG-independent controller to guide movements. This EMG-independent feedback ensures that the user is in-the-loop during recording, while enabling the resulting dynamic dataset to be used as an EMG-based benchmark. The dataset is comprised of 20 able-bodied participants completing three to four sessions over a period of 14 to 21 days. The ability of the dynamic dataset to serve as a benchmark is leveraged to evaluate the impact of different-recalibration techniques for long-term (across-day) gesture recognition, including a novel algorithm, named TADANN. TADANN consistently and significantly (p <; 0.05) outperforms using fine-tuning as the recalibration technique
Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning
In recent years, deep learning algorithms have become increasingly more
prominent for their unparalleled ability to automatically learn discriminant
features from large amounts of data. However, within the field of
electromyography-based gesture recognition, deep learning algorithms are seldom
employed as they require an unreasonable amount of effort from a single person,
to generate tens of thousands of examples.
This work's hypothesis is that general, informative features can be learned
from the large amounts of data generated by aggregating the signals of multiple
users, thus reducing the recording burden while enhancing gesture recognition.
Consequently, this paper proposes applying transfer learning on aggregated data
from multiple users, while leveraging the capacity of deep learning algorithms
to learn discriminant features from large datasets. Two datasets comprised of
19 and 17 able-bodied participants respectively (the first one is employed for
pre-training) were recorded for this work, using the Myo Armband. A third Myo
Armband dataset was taken from the NinaPro database and is comprised of 10
able-bodied participants. Three different deep learning networks employing
three different modalities as input (raw EMG, Spectrograms and Continuous
Wavelet Transform (CWT)) are tested on the second and third dataset. The
proposed transfer learning scheme is shown to systematically and significantly
enhance the performance for all three networks on the two datasets, achieving
an offline accuracy of 98.31% for 7 gestures over 17 participants for the
CWT-based ConvNet and 68.98% for 18 gestures over 10 participants for the raw
EMG-based ConvNet. Finally, a use-case study employing eight able-bodied
participants suggests that real-time feedback allows users to adapt their
muscle activation strategy which reduces the degradation in accuracy normally
experienced over time.Comment: Source code and datasets available:
https://github.com/Giguelingueling/MyoArmbandDatase
Surface EMG-Based Inter-Session/Inter-Subject Gesture Recognition by Leveraging Lightweight All-ConvNet and Transfer Learning
Gesture recognition using low-resolution instantaneous HD-sEMG images opens
up new avenues for the development of more fluid and natural muscle-computer
interfaces. However, the data variability between inter-session and
inter-subject scenarios presents a great challenge. The existing approaches
employed very large and complex deep ConvNet or 2SRNN-based domain adaptation
methods to approximate the distribution shift caused by these inter-session and
inter-subject data variability. Hence, these methods also require learning over
millions of training parameters and a large pre-trained and target domain
dataset in both the pre-training and adaptation stages. As a result, it makes
high-end resource-bounded and computationally very expensive for deployment in
real-time applications. To overcome this problem, we propose a lightweight
All-ConvNet+TL model that leverages lightweight All-ConvNet and transfer
learning (TL) for the enhancement of inter-session and inter-subject gesture
recognition performance. The All-ConvNet+TL model consists solely of
convolutional layers, a simple yet efficient framework for learning invariant
and discriminative representations to address the distribution shifts caused by
inter-session and inter-subject data variability. Experiments on four datasets
demonstrate that our proposed methods outperform the most complex existing
approaches by a large margin and achieve state-of-the-art results on
inter-session and inter-subject scenarios and perform on par or competitively
on intra-session gesture recognition. These performance gaps increase even more
when a tiny amount (e.g., a single trial) of data is available on the target
domain for adaptation. These outstanding experimental results provide evidence
that the current state-of-the-art models may be overparameterized for
sEMG-based inter-session and inter-subject gesture recognition tasks
Guidage non-intrusif d'un bras robotique à l'aide d'un bracelet myoélectrique à électrode sèche
Depuis plusieurs années la robotique est vue comme une solution clef pour améliorer la qualité de vie des personnes ayant subi une amputation. Pour créer de nouvelles prothèses intelligentes qui peuvent être facilement intégrées à la vie quotidienne et acceptée par ces personnes, celles-ci doivent être non-intrusives, fiables et peu coûteuses. L’électromyographie de surface fournit une interface intuitive et non intrusive basée sur l’activité musculaire de l’utilisateur permettant d’interagir avec des robots. Cependant, malgré des recherches approfondies dans le domaine de la classification des signaux sEMG, les classificateurs actuels manquent toujours de fiabilité, car ils ne sont pas robustes face au bruit à court terme (par exemple, petit déplacement des électrodes, fatigue musculaire) ou à long terme (par exemple, changement de la masse musculaire et des tissus adipeux) et requiert donc de recalibrer le classifieur de façon périodique. L’objectif de mon projet de recherche est de proposer une interface myoélectrique humain-robot basé sur des algorithmes d’apprentissage par transfert et d’adaptation de domaine afin d’augmenter la fiabilité du système à long-terme, tout en minimisant l’intrusivité (au niveau du temps de préparation) de ce genre de système. L’aspect non intrusif est obtenu en utilisant un bracelet à électrode sèche possédant dix canaux. Ce bracelet (3DC Armband) est de notre (Docteur Gabriel Gagnon-Turcotte, mes co-directeurs et moi-même) conception et a été réalisé durant mon doctorat. À l’heure d’écrire ces lignes, le 3DC Armband est le bracelet sans fil pour l’enregistrement de signaux sEMG le plus performant disponible. Contrairement aux dispositifs utilisant des électrodes à base de gel qui nécessitent un rasage de l’avant-bras, un nettoyage de la zone de placement et l’application d’un gel conducteur avant l’utilisation, le brassard du 3DC peut simplement être placé sur l’avant-bras sans aucune préparation. Cependant, cette facilité d’utilisation entraîne une diminution de la qualité de l’information du signal. Cette diminution provient du fait que les électrodes sèches obtiennent un signal plus bruité que celle à base de gel. En outre, des méthodes invasives peuvent réduire les déplacements d’électrodes lors de l’utilisation, contrairement au brassard. Pour remédier à cette dégradation de l’information, le projet de recherche s’appuiera sur l’apprentissage profond, et plus précisément sur les réseaux convolutionels. Le projet de recherche a été divisé en trois phases. La première porte sur la conception d’un classifieur permettant la reconnaissance de gestes de la main en temps réel. La deuxième porte sur l’implémentation d’un algorithme d’apprentissage par transfert afin de pouvoir profiter des données provenant d’autres personnes, permettant ainsi d’améliorer la classification des mouvements de la main pour un nouvel individu tout en diminuant le temps de préparation nécessaire pour utiliser le système. La troisième phase consiste en l’élaboration et l’implémentation des algorithmes d’adaptation de domaine et d’apprentissage faiblement supervisé afin de créer un classifieur qui soit robuste au changement à long terme.For several years, robotics has been seen as a key solution to improve the quality of life of people living with upper-limb disabilities. To create new, smart prostheses that can easily be integrated into everyday life, they must be non-intrusive, reliable and inexpensive. Surface electromyography provides an intuitive interface based on a user’s muscle activity to interact with robots. However, despite extensive research in the field of sEMG signal classification, current classifiers still lack reliability due to their lack of robustness to short-term (e.g. small electrode displacement, muscle fatigue) or long-term (e.g. change in muscle mass and adipose tissue) noise. In practice, this mean that to be useful, classifier needs to be periodically re-calibrated, a time consuming process. The goal of my research project is to proposes a human-robot myoelectric interface based on transfer learning and domain adaptation algorithms to increase the reliability of the system in the long term, while at the same time reducing the intrusiveness (in terms of hardware and preparation time) of this kind of systems. The non-intrusive aspect is achieved from a dry-electrode armband featuring ten channels. This armband, named the 3DC Armband is from our (Dr. Gabriel Gagnon-Turcotte, my co-directors and myself) conception and was realized during my doctorate. At the time of writing, the 3DC Armband offers the best performance for currently available dry-electrodes, surface electromyographic armbands. Unlike gel-based electrodes which require intrusive skin preparation (i.e. shaving, cleaning the skin and applying conductive gel), the 3DC Armband can simply be placed on the forearm without any preparation. However, this ease of use results in a decrease in the quality of information. This decrease is due to the fact that the signal recorded by dry electrodes is inherently noisier than gel-based ones. In addition, other systems use invasive methods (intramuscular electromyography) to capture a cleaner signal and reduce the source of noises (e.g. electrode shift). To remedy this degradation of information resulting from the non-intrusiveness of the armband, this research project will rely on deep learning, and more specifically on convolutional networks. The research project was divided into three phases. The first is the design of a classifier allowing the recognition of hand gestures in real-time. The second is the implementation of a transfer learning algorithm to take advantage of the data recorded across multiple users, thereby improving the system’s accuracy, while decreasing the time required to use the system. The third phase is the development and implementation of a domain adaptation and self-supervised learning to enhance the classifier’s robustness to long-term changes
sEMG-based hand gesture recognition with deep learning
Hand gesture recognition based on surface electromyographic (sEMG) signals is a promising approach for the development of Human-Machine Interfaces (HMIs) with a natural control, such as intuitive robot interfaces or poly-articulated prostheses. However, real-world applications are limited by reliability problems due to motion artifacts, postural and temporal variability, and sensor re-positioning.
This master thesis is the first application of deep learning on the Unibo-INAIL dataset, the first public sEMG dataset exploring the variability between subjects, sessions and arm postures, by collecting data over 8 sessions of each of 7 able-bodied subjects executing 6 hand gestures in 4 arm postures. In the most recent studies, the variability is addressed with training strategies based on training set composition, which improve inter-posture and inter-day generalization of classical (i.e. non-deep) machine learning classifiers, among which the RBF-kernel SVM yields the highest accuracy.
The deep architecture realized in this work is a 1d-CNN implemented in Pytorch, inspired by a 2d-CNN reported to perform well on other public benchmark databases. On this 1d-CNN, various training strategies based on training set composition were implemented and tested.
Multi-session training proves to yield higher inter-session validation accuracies than single-session training. Two-posture training proves to be the best postural training (proving the benefit of training on more than one posture), and yields 81.2% inter-posture test accuracy. Five-day training proves to be the best multi-day training, and yields 75.9% inter-day test accuracy. All results are close to the baseline. Moreover, the results of multi-day trainings highlight the phenomenon of user adaptation, indicating that training should also prioritize recent data.
Though not better than the baseline, the achieved classification accuracies rightfully place the 1d-CNN among the candidates for further research
Interpreting Deep Learning Features for Myoelectric Control: A Comparison with Handcrafted Features
The research in myoelectric control systems primarily focuses on extracting
discriminative representations from the electromyographic (EMG) signal by
designing handcrafted features. Recently, deep learning techniques have been
applied to the challenging task of EMG-based gesture recognition. The adoption
of these techniques slowly shifts the focus from feature engineering to feature
learning. However, the black-box nature of deep learning makes it hard to
understand the type of information learned by the network and how it relates to
handcrafted features. Additionally, due to the high variability in EMG
recordings between participants, deep features tend to generalize poorly across
subjects using standard training methods. Consequently, this work introduces a
new multi-domain learning algorithm, named ADANN, which significantly enhances
(p=0.00004) inter-subject classification accuracy by an average of 19.40%
compared to standard training. Using ADANN-generated features, the main
contribution of this work is to provide the first topological data analysis of
EMG-based gesture recognition for the characterisation of the information
encoded within a deep network, using handcrafted features as landmarks. This
analysis reveals that handcrafted features and the learned features (in the
earlier layers) both try to discriminate between all gestures, but do not
encode the same information to do so. Furthermore, using convolutional network
visualization techniques reveal that learned features tend to ignore the most
activated channel during gesture contraction, which is in stark contrast with
the prevalence of handcrafted features designed to capture amplitude
information. Overall, this work paves the way for hybrid feature sets by
providing a clear guideline of complementary information encoded within learned
and handcrafted features.Comment: The first two authors shared first authorship. The last three authors
shared senior authorship. 32 page
A Deep Learning Sequential Decoder for Transient High-Density Electromyography in Hand Gesture Recognition Using Subject-Embedded Transfer Learning
Hand gesture recognition (HGR) has gained significant attention due to the
increasing use of AI-powered human-computer interfaces that can interpret the
deep spatiotemporal dynamics of biosignals from the peripheral nervous system,
such as surface electromyography (sEMG). These interfaces have a range of
applications, including the control of extended reality, agile prosthetics, and
exoskeletons. However, the natural variability of sEMG among individuals has
led researchers to focus on subject-specific solutions. Deep learning methods,
which often have complex structures, are particularly data-hungry and can be
time-consuming to train, making them less practical for subject-specific
applications. In this paper, we propose and develop a generalizable, sequential
decoder of transient high-density sEMG (HD-sEMG) that achieves 73% average
accuracy on 65 gestures for partially-observed subjects through
subject-embedded transfer learning, leveraging pre-knowledge of HGR acquired
during pre-training. The use of transient HD-sEMG before gesture stabilization
allows us to predict gestures with the ultimate goal of counterbalancing system
control delays. The results show that the proposed generalized models
significantly outperform subject-specific approaches, especially when the
training data is limited, and there is a significant number of gesture classes.
By building on pre-knowledge and incorporating a multiplicative
subject-embedded structure, our method comparatively achieves more than 13%
average accuracy across partially observed subjects with minimal data
availability. This work highlights the potential of HD-sEMG and demonstrates
the benefits of modeling common patterns across users to reduce the need for
large amounts of data for new users, enhancing practicality
- …