40 research outputs found
An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recognition
The focus of this paper is dynamic gesture recognition in the context of the
interaction between humans and machines. We propose a model consisting of two
sub-networks, a transformer and an ordered-neuron long-short-term-memory
(ON-LSTM) based recurrent neural network (RNN). Each sub-network is trained to
perform the task of gesture recognition using only skeleton joints. Since each
sub-network extracts different types of features due to the difference in
architecture, the knowledge can be shared between the sub-networks. Through
knowledge distillation, the features and predictions from each sub-network are
fused together into a new fusion classifier. In addition, a cyclical learning
rate can be used to generate a series of models that are combined in an
ensemble, in order to yield a more generalizable prediction. The proposed
ensemble of knowledge-sharing models exhibits an overall accuracy of 86.11%
using only skeleton information, as tested using the Dynamic Hand Gesture-14/28
datasetComment: Accepted at International Joint Conference on Neural Networ
LE-HGR: A Lightweight and Efficient RGB-based Online Gesture Recognition Network for Embedded AR Devices
Online hand gesture recognition (HGR) techniques are essential in augmented
reality (AR) applications for enabling natural human-to-computer interaction
and communication. In recent years, the consumer market for low-cost AR devices
has been rapidly growing, while the technology maturity in this domain is still
limited. Those devices are typical of low prices, limited memory, and
resource-constrained computational units, which makes online HGR a challenging
problem. To tackle this problem, we propose a lightweight and computationally
efficient HGR framework, namely LE-HGR, to enable real-time gesture recognition
on embedded devices with low computing power. We also show that the proposed
method is of high accuracy and robustness, which is able to reach high-end
performance in a variety of complicated interaction environments. To achieve
our goal, we first propose a cascaded multi-task convolutional neural network
(CNN) to simultaneously predict probabilities of hand detection and regress
hand keypoint locations online. We show that, with the proposed cascaded
architecture design, false-positive estimates can be largely eliminated.
Additionally, an associated mapping approach is introduced to track the hand
trace via the predicted locations, which addresses the interference of
multi-handedness. Subsequently, we propose a trace sequence neural network
(TraceSeqNN) to recognize the hand gesture by exploiting the motion features of
the tracked trace. Finally, we provide a variety of experimental results to
show that the proposed framework is able to achieve state-of-the-art accuracy
with significantly reduced computational cost, which are the key properties for
enabling real-time applications in low-cost commercial devices such as mobile
devices and AR/VR headsets.Comment: Published in: 2019 IEEE International Symposium on Mixed and
Augmented Reality Adjunct (ISMAR-Adjunct
SHREC'17 Track: 3D Hand Gesture Recognition Using a Depth and Skeletal Dataset
International audienceHand gesture recognition is recently becoming one of the most attractive field of research in pattern recognition. The objective of this track is to evaluate the performance of recent recognition approaches using a challenging hand gesture dataset containing 14 gestures, performed by 28 participants executing the same gesture with two different numbers of fingers. Two research groups have participated to this track, the accuracy of their recognition algorithms have been evaluated and compared to three other state-of-the-art approaches