661 research outputs found
From Unimodal to Multimodal: improving the sEMG-Based Pattern Recognition via deep generative models
Multimodal hand gesture recognition (HGR) systems can achieve higher
recognition accuracy. However, acquiring multimodal gesture recognition data
typically requires users to wear additional sensors, thereby increasing
hardware costs. This paper proposes a novel generative approach to improve
Surface Electromyography (sEMG)-based HGR accuracy via virtual Inertial
Measurement Unit (IMU) signals. Specifically, we trained a deep generative
model based on the intrinsic correlation between forearm sEMG signals and
forearm IMU signals to generate virtual forearm IMU signals from the input
forearm sEMG signals at first. Subsequently, the sEMG signals and virtual IMU
signals were fed into a multimodal Convolutional Neural Network (CNN) model for
gesture recognition. To evaluate the performance of the proposed approach, we
conducted experiments on 6 databases, including 5 publicly available databases
and our collected database comprising 28 subjects performing 38 gestures,
containing both sEMG and IMU data. The results show that our proposed approach
outperforms the sEMG-based unimodal HGR method (with increases of
2.15%-13.10%). It demonstrates that incorporating virtual IMU signals,
generated by deep generative models, can significantly enhance the accuracy of
sEMG-based HGR. The proposed approach represents a successful attempt to
transition from unimodal HGR to multimodal HGR without additional sensor
hardware
Multikernel convolutional neural network for sEMG based hand gesture classification
openIl riconoscimento dei gesti della mano è un argomento ampiamente discusso in letteratura, dove vengono analizzate diverse tecniche sia in termini di tipi di segnale in ingresso che di algoritmi. Tra i più utilizzati ci sono i segnali elettromiografici (sEMG), già ampiamente sfruttati nelle applicazioni di interazione uomo-macchina (HMI). Determinare come decodificare le informazioni contenute nei segnali EMG in modo robusto e accurato è un problema chiave per il quale è urgente trovare una soluzione.
Recentemente, molti incarichi di riconoscimento dei pattern EMG sono stati affrontati utilizzando metodi di deep learning. Nonostante le elevate prestazioni di questi ultimi, le loro capacitĂ di generalizzazione sono spesso limitate dall'elevata eterogeneitĂ tra i soggetti, l'impedenza cutanea, il posizionamento dei sensori, ecc.
Inoltre, poiché questo progetto è focalizzato sull'applicazione in tempo reale di protesi, ci sono maggiori vincoli sui tempi di risposta del sistema che riducono la complessità dei modelli. In questa tesi è stata testata una rete neurale convoluzionale multi-kernel su diversi dataset pubblici per verificare la sua generalizzabilità . Inoltre, è stata analizzata la capacità del modello di superare i limiti inter-soggetto e inter-sessione in giorni diversi, preservando i vincoli legati a un sistema embedded. I risultati confermano le difficoltà incontrate nell'estrazione di informazioni dai segnali emg; tuttavia, dimostrano la possibilità di ottenere buone prestazioni per un uso robusto di mani prostetiche. Inoltre, è possibile ottenere prestazioni migliori personalizzando il modello con tecniche di transfer learning e di adattamento al dominio.Hand gesture recognition is a widely discussed topic in the literature, where different techniques are analyzed in terms of both input signal types and algorithms. Among the most widely used are electromyographic signals (sEMG), which are already widely exploited in human-computer interaction (HMI) applications. Determining how to decode the information contained in EMG signals robustly and accurately is a key problem for which a solution is urgently needed.
Recently, many EMG pattern recognition tasks have been addressed using deep learning methods. Despite their high performance, their generalization capabilities are often limited by high heterogeneity among subjects, skin impedance, sensor placement, etc.
In addition, because this project is focused on the real-time application of prostheses, there are greater constraints on the system response times that reduce the complexity of the models. In this thesis, a multi-kernel convolutional neural network was tested on several public datasets to verify its generalizability. In addition, the model's ability to overcome inter-subject and inter-session constraints on different days while preserving the constraints associated with an embedded system was analyzed. The results confirm the difficulties encountered in extracting information from emg signals; however, they demonstrate the possibility of achieving good performance for robust use of prosthetic hands. In addition, better performance can be achieved by customizing the model with transfer learning and domain-adaptationtechniques
Surface EMG-Based Inter-Session/Inter-Subject Gesture Recognition by Leveraging Lightweight All-ConvNet and Transfer Learning
Gesture recognition using low-resolution instantaneous HD-sEMG images opens
up new avenues for the development of more fluid and natural muscle-computer
interfaces. However, the data variability between inter-session and
inter-subject scenarios presents a great challenge. The existing approaches
employed very large and complex deep ConvNet or 2SRNN-based domain adaptation
methods to approximate the distribution shift caused by these inter-session and
inter-subject data variability. Hence, these methods also require learning over
millions of training parameters and a large pre-trained and target domain
dataset in both the pre-training and adaptation stages. As a result, it makes
high-end resource-bounded and computationally very expensive for deployment in
real-time applications. To overcome this problem, we propose a lightweight
All-ConvNet+TL model that leverages lightweight All-ConvNet and transfer
learning (TL) for the enhancement of inter-session and inter-subject gesture
recognition performance. The All-ConvNet+TL model consists solely of
convolutional layers, a simple yet efficient framework for learning invariant
and discriminative representations to address the distribution shifts caused by
inter-session and inter-subject data variability. Experiments on four datasets
demonstrate that our proposed methods outperform the most complex existing
approaches by a large margin and achieve state-of-the-art results on
inter-session and inter-subject scenarios and perform on par or competitively
on intra-session gesture recognition. These performance gaps increase even more
when a tiny amount (e.g., a single trial) of data is available on the target
domain for adaptation. These outstanding experimental results provide evidence
that the current state-of-the-art models may be overparameterized for
sEMG-based inter-session and inter-subject gesture recognition tasks
Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning
In recent years, deep learning algorithms have become increasingly more
prominent for their unparalleled ability to automatically learn discriminant
features from large amounts of data. However, within the field of
electromyography-based gesture recognition, deep learning algorithms are seldom
employed as they require an unreasonable amount of effort from a single person,
to generate tens of thousands of examples.
This work's hypothesis is that general, informative features can be learned
from the large amounts of data generated by aggregating the signals of multiple
users, thus reducing the recording burden while enhancing gesture recognition.
Consequently, this paper proposes applying transfer learning on aggregated data
from multiple users, while leveraging the capacity of deep learning algorithms
to learn discriminant features from large datasets. Two datasets comprised of
19 and 17 able-bodied participants respectively (the first one is employed for
pre-training) were recorded for this work, using the Myo Armband. A third Myo
Armband dataset was taken from the NinaPro database and is comprised of 10
able-bodied participants. Three different deep learning networks employing
three different modalities as input (raw EMG, Spectrograms and Continuous
Wavelet Transform (CWT)) are tested on the second and third dataset. The
proposed transfer learning scheme is shown to systematically and significantly
enhance the performance for all three networks on the two datasets, achieving
an offline accuracy of 98.31% for 7 gestures over 17 participants for the
CWT-based ConvNet and 68.98% for 18 gestures over 10 participants for the raw
EMG-based ConvNet. Finally, a use-case study employing eight able-bodied
participants suggests that real-time feedback allows users to adapt their
muscle activation strategy which reduces the degradation in accuracy normally
experienced over time.Comment: Source code and datasets available:
https://github.com/Giguelingueling/MyoArmbandDatase
A Transferable Adaptive Domain Adversarial Neural Network for Virtual Reality Augmented EMG-Based Gesture Recognition
Within the field of electromyography-based (EMG) gesture recognition,
disparities exist between the offline accuracy reported in the literature and
the real-time usability of a classifier. This gap mainly stems from two
factors: 1) The absence of a controller, making the data collected dissimilar
to actual control. 2) The difficulty of including the four main dynamic factors
(gesture intensity, limb position, electrode shift, and transient changes in
the signal), as including their permutations drastically increases the amount
of data to be recorded. Contrarily, online datasets are limited to the exact
EMG-based controller used to record them, necessitating the recording of a new
dataset for each control method or variant to be tested. Consequently, this
paper proposes a new type of dataset to serve as an intermediate between
offline and online datasets, by recording the data using a real-time
experimental protocol. The protocol, performed in virtual reality, includes the
four main dynamic factors and uses an EMG-independent controller to guide
movements. This EMG-independent feedback ensures that the user is in-the-loop
during recording, while enabling the resulting dynamic dataset to be used as an
EMG-based benchmark. The dataset is comprised of 20 able-bodied participants
completing three to four sessions over a period of 14 to 21 days. The ability
of the dynamic dataset to serve as a benchmark is leveraged to evaluate the
impact of different recalibration techniques for long-term (across-day) gesture
recognition, including a novel algorithm, named TADANN. TADANN consistently and
significantly (p<0.05) outperforms using fine-tuning as the recalibration
technique.Comment: 10 Pages. The last three authors shared senior authorshi
Deep convolutional spiking neural network based hand gesture recognition
Novel technologies for EMG (Electromyogram) based hand gesture recognition have been investigated for many industrial applications. In this paper, a novel approach which is based on a specific designed spiking convolution neural network which is fed by a novel EMG signal energy density map is presented. The experimental results indicate that the new approach not only rapidly decreases the required processing time but also increases the average recognition accuracy to 98.76% based on the Strathclyde dataset and to 98.21% based on the CapgMyo open source dataset. A relative comparison of experimental results between the proposed novel EMG based hand gesture recognition methodology and other similar approaches indicates the superior effectiveness of the new design
sEMG-based hand gesture recognition with deep learning
Hand gesture recognition based on surface electromyographic (sEMG) signals is a promising approach for the development of Human-Machine Interfaces (HMIs) with a natural control, such as intuitive robot interfaces or poly-articulated prostheses. However, real-world applications are limited by reliability problems due to motion artifacts, postural and temporal variability, and sensor re-positioning.
This master thesis is the first application of deep learning on the Unibo-INAIL dataset, the first public sEMG dataset exploring the variability between subjects, sessions and arm postures, by collecting data over 8 sessions of each of 7 able-bodied subjects executing 6 hand gestures in 4 arm postures. In the most recent studies, the variability is addressed with training strategies based on training set composition, which improve inter-posture and inter-day generalization of classical (i.e. non-deep) machine learning classifiers, among which the RBF-kernel SVM yields the highest accuracy.
The deep architecture realized in this work is a 1d-CNN implemented in Pytorch, inspired by a 2d-CNN reported to perform well on other public benchmark databases. On this 1d-CNN, various training strategies based on training set composition were implemented and tested.
Multi-session training proves to yield higher inter-session validation accuracies than single-session training. Two-posture training proves to be the best postural training (proving the benefit of training on more than one posture), and yields 81.2% inter-posture test accuracy. Five-day training proves to be the best multi-day training, and yields 75.9% inter-day test accuracy. All results are close to the baseline. Moreover, the results of multi-day trainings highlight the phenomenon of user adaptation, indicating that training should also prioritize recent data.
Though not better than the baseline, the achieved classification accuracies rightfully place the 1d-CNN among the candidates for further research
- …