29 research outputs found
Upper-limb Geometric MyoPassivity Map for Physical Human-Robot Interaction
The intrinsic biomechanical characteristic of the human upper limb plays a
central role in absorbing the interactive energy during physical human-robot
interaction (pHRI). We have recently shown that based on the concept of
``Excess of Passivity (EoP)," from nonlinear control theory, it is possible to
decode such energetic behavior for both upper and lower limbs. The extracted
knowledge can be used in the design of controllers for optimizing the
transparency and fidelity of force fields in human-robot interaction and in
haptic systems. In this paper, for the first time, we investigate the frequency
behavior of the passivity map for the upper limb when the muscle co-activation
was controlled in real-time through visual electromyographic feedback. Five
healthy subjects (age: 27 +/- 5) were included in this study. The energetic
behavior was evaluated at two stimulation frequencies at eight interaction
directions over two controlled muscle co-activation levels. Electromyography
(EMG) was captured using the Delsys Wireless Trigno system. Results showed a
correlation between EMG and EoP, which was further altered by increasing the
frequency. The proposed energetic behavior is named the Geometric MyoPassivity
(GMP) map. The findings indicate that the GMP map has the potential to be used
in real-time to quantify the absorbable energy, thus passivity margin of
stability for upper limb interaction during pHRI
From Unstable Contacts to Stable Control: A Deep Learning Paradigm for HD-sEMG in Neurorobotics
In the past decade, there has been significant advancement in designing
wearable neural interfaces for controlling neurorobotic systems, particularly
bionic limbs. These interfaces function by decoding signals captured
non-invasively from the skin's surface. Portable high-density surface
electromyography (HD-sEMG) modules combined with deep learning decoding have
attracted interest by achieving excellent gesture prediction and myoelectric
control of prosthetic systems and neurorobots. However, factors like
pixel-shape electrode size and unstable skin contact make HD-sEMG susceptible
to pixel electrode drops. The sparse electrode-skin disconnections rooted in
issues such as low adhesion, sweating, hair blockage, and skin stretch
challenge the reliability and scalability of these modules as the perception
unit for neurorobotic systems. This paper proposes a novel deep-learning model
providing resiliency for HD-sEMG modules, which can be used in the wearable
interfaces of neurorobots. The proposed 3D Dilated Efficient CapsNet model
trains on an augmented input space to computationally `force' the network to
learn channel dropout variations and thus learn robustness to channel dropout.
The proposed framework maintained high performance under a sensor dropout
reliability study conducted. Results show conventional models' performance
significantly degrades with dropout and is recovered using the proposed
architecture and the training paradigm
ViT-MDHGR: Cross-day Reliability and Agility in Dynamic Hand Gesture Prediction via HD-sEMG Signal Decoding
Surface electromyography (sEMG) and high-density sEMG (HD-sEMG) biosignals
have been extensively investigated for myoelectric control of prosthetic
devices, neurorobotics, and more recently human-computer interfaces because of
their capability for hand gesture recognition/prediction in a wearable and
non-invasive manner. High intraday (same-day) performance has been reported.
However, the interday performance (separating training and testing days) is
substantially degraded due to the poor generalizability of conventional
approaches over time, hindering the application of such techniques in real-life
practices. There are limited recent studies on the feasibility of multi-day
hand gesture recognition. The existing studies face a major challenge: the need
for long sEMG epochs makes the corresponding neural interfaces impractical due
to the induced delay in myoelectric control. This paper proposes a compact
ViT-based network for multi-day dynamic hand gesture prediction. We tackle the
main challenge as the proposed model only relies on very short HD-sEMG signal
windows (i.e., 50 ms, accounting for only one-sixth of the convention for
real-time myoelectric implementation), boosting agility and responsiveness. Our
proposed model can predict 11 dynamic gestures for 20 subjects with an average
accuracy of over 71% on the testing day, 3-25 days after training. Moreover,
when calibrated on just a small portion of data from the testing day, the
proposed model can achieve over 92% accuracy by retraining less than 10% of the
parameters for computational efficiency
From pulse width modulated TENS to cortical modulation:based on EEG functional connectivity analysis
HYDRA-HGR: A Hybrid Transformer-based Architecture for Fusion of Macroscopic and Microscopic Neural Drive Information
Development of advance surface Electromyogram (sEMG)-based Human-Machine
Interface (HMI) systems is of paramount importance to pave the way towards
emergence of futuristic Cyber-Physical-Human (CPH) worlds. In this context, the
main focus of recent literature was on development of different Deep Neural
Network (DNN)-based architectures that perform Hand Gesture Recognition (HGR)
at a macroscopic level (i.e., directly from sEMG signals). At the same time,
advancements in acquisition of High-Density sEMG signals (HD-sEMG) have
resulted in a surge of significant interest on sEMG decomposition techniques to
extract microscopic neural drive information. However, due to complexities of
sEMG decomposition and added computational overhead, HGR at microscopic level
is less explored than its aforementioned DNN-based counterparts. In this
regard, we propose the HYDRA-HGR framework, which is a hybrid model that
simultaneously extracts a set of temporal and spatial features through its two
independent Vision Transformer (ViT)-based parallel architectures (the so
called Macro and Micro paths). The Macro Path is trained directly on the
pre-processed HD-sEMG signals, while the Micro path is fed with the p-to-p
values of the extracted Motor Unit Action Potentials (MUAPs) of each source.
Extracted features at macroscopic and microscopic levels are then coupled via a
Fully Connected (FC) fusion layer. We evaluate the proposed hybrid HYDRA-HGR
framework through a recently released HD-sEMG dataset, and show that it
significantly outperforms its stand-alone counterparts. The proposed HYDRA-HGR
framework achieves average accuracy of 94.86% for the 250 ms window size, which
is 5.52% and 8.22% higher than that of the Macro and Micro paths, respectively
Therapist-in-the-Loop robotics-assisted mirror rehabilitation therapy: An Assist-as-Needed framework
framework for robotics-assisted mirror rehabilitation therapy integrated with adaptive Assist-as-Needed (ANN) training, to be adjusted based on the impairment and disability level of the patient’s affected limb. Closed-loop system stability has been investigated using a combination of the Circle Criterion and the Small-Gain Theorem to account both for time-delay and the time-varying adaptive ANN training. Experiments to investigate the performance of the proposed framework are reported. I
Force-Aware Interface via Electromyography for Natural VR/AR Interaction
While tremendous advances in visual and auditory realism have been made for
virtual and augmented reality (VR/AR), introducing a plausible sense of
physicality into the virtual world remains challenging. Closing the gap between
real-world physicality and immersive virtual experience requires a closed
interaction loop: applying user-exerted physical forces to the virtual
environment and generating haptic sensations back to the users. However,
existing VR/AR solutions either completely ignore the force inputs from the
users or rely on obtrusive sensing devices that compromise user experience.
By identifying users' muscle activation patterns while engaging in VR/AR, we
design a learning-based neural interface for natural and intuitive force
inputs. Specifically, we show that lightweight electromyography sensors,
resting non-invasively on users' forearm skin, inform and establish a robust
understanding of their complex hand activities. Fuelled by a
neural-network-based model, our interface can decode finger-wise forces in
real-time with 3.3% mean error, and generalize to new users with little
calibration. Through an interactive psychophysical study, we show that human
perception of virtual objects' physical properties, such as stiffness, can be
significantly enhanced by our interface. We further demonstrate that our
interface enables ubiquitous control via finger tapping. Ultimately, we
envision our findings to push forward research towards more realistic
physicality in future VR/AR.Comment: ACM Transactions on Graphics (SIGGRAPH Asia 2022
A Deep Learning Sequential Decoder for Transient High-Density Electromyography in Hand Gesture Recognition Using Subject-Embedded Transfer Learning
Hand gesture recognition (HGR) has gained significant attention due to the
increasing use of AI-powered human-computer interfaces that can interpret the
deep spatiotemporal dynamics of biosignals from the peripheral nervous system,
such as surface electromyography (sEMG). These interfaces have a range of
applications, including the control of extended reality, agile prosthetics, and
exoskeletons. However, the natural variability of sEMG among individuals has
led researchers to focus on subject-specific solutions. Deep learning methods,
which often have complex structures, are particularly data-hungry and can be
time-consuming to train, making them less practical for subject-specific
applications. In this paper, we propose and develop a generalizable, sequential
decoder of transient high-density sEMG (HD-sEMG) that achieves 73% average
accuracy on 65 gestures for partially-observed subjects through
subject-embedded transfer learning, leveraging pre-knowledge of HGR acquired
during pre-training. The use of transient HD-sEMG before gesture stabilization
allows us to predict gestures with the ultimate goal of counterbalancing system
control delays. The results show that the proposed generalized models
significantly outperform subject-specific approaches, especially when the
training data is limited, and there is a significant number of gesture classes.
By building on pre-knowledge and incorporating a multiplicative
subject-embedded structure, our method comparatively achieves more than 13%
average accuracy across partially observed subjects with minimal data
availability. This work highlights the potential of HD-sEMG and demonstrates
the benefits of modeling common patterns across users to reduce the need for
large amounts of data for new users, enhancing practicality