Hand Gesture Recognition Using a Radar Echo I–Q Plot and a Convolutional Neural Network

We propose a hand gesture recognition technique using a convolutional neural network applied to radar echo inphase/quadrature (I/Q) plot trajectories. The proposed technique is demonstrated to accurately recognize six types of hand gestures for ten participants. The system consists of a low-cost 2.4-GHz continuous-wave monostatic radar with a single antenna. The radar echo trajectories are converted to low-resolution images and are used for the training and evaluation of the proposed technique. Results indicate that the proposed technique can recognize hand gestures with average accuracy exceeding 90%

Duodepth: Static Gesture Recognition Via Dual Depth Sensors

Static gesture recognition is an effective non-verbal communication channel between a user and their devices; however many modern methods are sensitive to the relative pose of the user's hands with respect to the capture device, as parts of the gesture can become occluded. We present two methodologies for gesture recognition via synchronized recording from two depth cameras to alleviate this occlusion problem. One is a more classic approach using iterative closest point registration to accurately fuse point clouds and a single PointNet architecture for classification, and the other is a dual Point-Net architecture for classification without registration. On a manually collected data-set of 20,100 point clouds we show a 39.2% reduction in misclassification for the fused point cloud method, and 53.4% for the dual PointNet, when compared to a standard single camera pipeline.Comment: 26th International Conference on Image Processin

An approach to sign language translation using the Intel Realsense camera

An Intel RealSense camera is used for translating static manual American Sign Language gestures into text. The system uses palm orientation and finger joint data as inputs for either a support vector machine or a neural network whose architecture has been optimized by a genetic algorithm. A data set consisting of 100 samples of 26 gestures (the letters of the alphabet) is extracted from 10 participants. When comparing the different learners in combination with different standard preprocessing techniques, the highest accuracy of 95% is achieved by a support vector machine with a scaling method, as well as principal component analysis, used for preprocessing. The highest performing neural network system reaches 92.1% but produces predictions much faster. We also present a simple software solution that uses the trained classifiers to enable user-friendly sign language translation

A robust method for VR-based hand gesture recognition using density-based CNN

Many VR-based medical purposes applications have been developed to help patients with mobility decrease caused by accidents, diseases, or other injuries to do physical treatment efficiently. VR-based applications were considered more effective helper for individual physical treatment because of their low-cost equipment and flexibility in time and space, less assistance of a physical therapist. A challenge in developing a VR-based physical treatment was understanding the body part movement accurately and quickly. We proposed a robust pipeline to understanding hand motion accurately. We retrieved our data from movement sensors such as HTC vive and leap motion. Given a sequence position of palm, we represent our data as binary 2D images of gesture shape. Our dataset consisted of 14 kinds of hand gestures recommended by a physiotherapist. Given 33 3D points that were mapped into binary images as input, we trained our proposed density-based CNN. Our CNN model concerned with our input characteristics, having many 'blank block pixels', 'single-pixel thickness' shape and generated as a binary image. Pyramid kernel size applied on the feature extraction part and classification layer using softmax as loss function, have given 97.7% accuracy

Privacy-Constrained Biometric System for Non-cooperative Users

With the consolidation of the new data protection regulation paradigm for each individual within the European Union (EU), major biometric technologies are now confronted with many concerns related to user privacy in biometric deployments. When individual biometrics are disclosed, the sensitive information about his/her personal data such as financial or health are at high risk of being misused or compromised. This issue can be escalated considerably over scenarios of non-cooperative users, such as elderly people residing in care homes, with their inability to interact conveniently and securely with the biometric system. The primary goal of this study is to design a novel database to investigate the problem of automatic people recognition under privacy constraints. To do so, the collected data-set contains the subject's hand and foot traits and excludes the face biometrics of individuals in order to protect their privacy. We carried out extensive simulations using different baseline methods, including deep learning. Simulation results show that, with the spatial features extracted from the subject sequence in both individual hand or foot videos, state-of-the-art deep models provide promising recognition performance

Improving hand gestures recognition capabilities by ensembling convolutional networks

Hand gestures provide humans a convenient way to interact with computers and many applications. However, factors such as the complexity of hand gesture models, differences in hand size and position, and other factors can affect the performance of the recognition and classification algorithms. Some developments of deep learning such as Convolutional Neural Networks (CNN) and Capsule Networks (CapsNets) have been proposed to improve the performance of image recognition systems in this particular field. While CNNs are undoubtedly the most widely used networks for object detection and image classification, CapsNets emerged to solve part of the limitations of the former. For this reason, in this work a particular ensemble of both networks is proposed to solve the American Sign Language recognition problem very effectively. The method is based on increasing diversity in both the model and the dataset. The results obtained show that the proposed ensemble model together with a simple data augmentation process produces a very competitive accuracy performance with the all considered datasets.This work has been partially supported by Instituto de Salud Carlos III (Project PI17/00771, Ministerio de Economía y de Competitividad, Government of Spain) and Fundación Séneca (Project 20901/PI/18, Agencia de Ciencia y Tecnología, Murcia)

Gesture Recognition by Using Depth Data: Comparison of Different Methodologies

In this chapter, the problem of gesture recognition in the context of human computer interaction is considered. Several classifiers based on different approaches such as neural network (NN), support vector machine (SVM), hidden Markov model (HMM), deep neural network (DNN), and dynamic time warping (DTW) are used to build the gesture models. The performance of each methodology is evaluated considering different users performing the gestures. This performance analysis is required as the users perform gestures in a personalized way and with different velocity. So the problems concerning the different lengths of the gesture in terms of number of frames, the variability in its representation, and the generalization ability of the classifiers have been analyzed