A robust method for VR-based hand gesture recognition using density-based CNN

Many VR-based medical purposes applications have been developed to help patients with mobility decrease caused by accidents, diseases, or other injuries to do physical treatment efficiently. VR-based applications were considered more effective helper for individual physical treatment because of their low-cost equipment and flexibility in time and space, less assistance of a physical therapist. A challenge in developing a VR-based physical treatment was understanding the body part movement accurately and quickly. We proposed a robust pipeline to understanding hand motion accurately. We retrieved our data from movement sensors such as HTC vive and leap motion. Given a sequence position of palm, we represent our data as binary 2D images of gesture shape. Our dataset consisted of 14 kinds of hand gestures recommended by a physiotherapist. Given 33 3D points that were mapped into binary images as input, we trained our proposed density-based CNN. Our CNN model concerned with our input characteristics, having many 'blank block pixels', 'single-pixel thickness' shape and generated as a binary image. Pyramid kernel size applied on the feature extraction part and classification layer using softmax as loss function, have given 97.7% accuracy

Japanese sign language classification based on gathered images and neural networks

This paper proposes a method to classify words in Japanese Sign Language (JSL). This approach employs a combined gathered image generation technique and a neural network with convolutional and pooling layers (CNNs). The gathered image generation generates images based on mean images. Herein, the maximum difference value is between blocks of mean and JSL motions images. The gathered images comprise blocks that having the calculated maximum difference value. CNNs extract the features of the gathered images, while a support vector machine for multi-class classification, and a multilayer perceptron are employed to classify 20 JSL words. The experimental results had 94.1% for the mean recognition accuracy of the proposed method. These results suggest that the proposed method can obtain information to classify the sample words

Japanese sign language classification based on gathered images and neural networks

This paper proposes a method to classify words in Japanese Sign Language (JSL). This approach employs a combined gathered image generation technique and a neural network with convolutional and pooling layers (CNNs). The gathered image generation generates images based on mean images. Herein, the maximum difference value is between blocks of mean and JSL motions images. The gathered images comprise blocks that having the calculated maximum difference value. CNNs extract the features of the gathered images, while a support vector machine for multi-class classification, and a multilayer perceptron are employed to classify 20 JSL words. The experimental results had 94.1% for the mean recognition accuracy of the proposed method. These results suggest that the proposed method can obtain information to classify the sample words

GESTURE RECOGNITION FOR PENCAK SILAT TAPAK SUCI REAL-TIME ANIMATION

The main target in this research is a design of a virtual martial arts training system in real-time and as a tool in learning martial arts independently using genetic algorithm methods and dynamic time warping. In this paper, it is still in the initial stages, which is focused on taking data sets of martial arts warriors using 3D animation and the Kinect sensor cameras, there are 2 warriors x 8 moves x 596 cases/gesture = 9,536 cases. Gesture Recognition Studies are usually distinguished: body gesture and hand and arm gesture, head and face gesture, and, all three can be studied simultaneously in martial arts pencak silat, using martial arts stance detection with scoring methods. Silat movement data is recorded in the form of oni files using the OpenNI ™ (OFW) framework and BVH (Bio Vision Hierarchical) files as well as plug-in support software on Mocap devices. Responsiveness is a measure of time responding to interruptions, and is critical because the system must be able to meet the demand

AMERICAN SIGN LANGUAGE FINGERSPELLING USING HYBRID DISCRETE WAVELET TRANSFORM-GABOR FILTER AND CONVOLUTIONAL NEURAL NETWORK

American Sign Language (ASL) is widely used for communication by deaf and mute people. In fingerspelling, the letters of the writing system are represented using only hands. Generally, hearing people do not understand sign language and this creates a communication gap between the signer and speaker community. A real-time ASL fingerspelling recognizer can be developed to solve this problem. Sign language recognizer can also be trained for other applications such as human-computer interaction. In this paper, a hybrid Discrete Wavelet TransformGabor filter is used on the colour images to extract features. Classifiers are evaluated on signer dependent and independent datasets. For evaluation, it is very important to consider signer dependency. Random Forest, Support Vector Machine and K-Nearest Neighbors classifiers are evaluated on the extracted set of features to classify the 24 classes of ASL alphabets with 95.8%, 94.3% and 96.7% accuracy respectively on signer dependent dataset and 49.16%, 48.75% and 50.83% accuracy respectively on signer independent dataset. Lastly, Convolutional Neural Network was also trained and evaluated on both, which produced 97.01% accuracy on signer dependent and 76.25% accuracy on signer independent dataset

Hand Gesture to Control Virtual Keyboard using Neural Network

Disability is one of a person's physical and mental conditions that can inhibit normal daily activities. One of the disabilities that can be found in disability is speech without fingers. Persons with disabilities have obstacles in communicating with people around both verbally and in writing. Communication tools to help people with disabilities without finger fingers continue to be developed, one of them is by creating a virtual keyboard using a Leap Motion sensor. The hand gestures are captured using the Leap Motion sensor so that the direction of the hand gesture in the form of pitch, yaw, and roll is obtained. The direction values are grouped into normal, right, left, up, down, and rotating gestures to control the virtual keyboard. The amount of data used for gesture recognition in this study was 5400 data consisting of 3780 training data and 1620 test data. The results of data testing conducted using the Artificial Neural Network method obtained an accuracy value of 98.82%. This study also performed a virtual keyboard performance test directly by typing 20 types of characters conducted by 15 respondents three times. The average time needed by respondents in typing is 5.45 seconds per character

Metodología basada en entrenamiento automático para el reconocimiento del movimiento individual de los dedos de la mano usando análisis de señales electromiográficos de superficie

El reconocimiento delmovimiento de los dedos de la mano es un área de investigación activa en la aplicación de interfacesmúsculo Computador (muCI) en la que la persona realiza un gesto (combinación de movimientos de los dedos) y una máquina reconoce el movimiento real. Al reconocer los movimientos individuales de los dedos de la mano se puede simular la motricidad fina que proporcionen un control individual de los dedos. En esta tesis se presenta una metodología para el reconocimiento del movimiento individual de los dedos de la mano, basado en la estimación de características de las señales electromiográficas superficiales adquiridas en el antebrazo. Se adquirió un conjunto de datos con 54 sujetos y ocho señales (canales) por sujeto mediante un sensor inalámbrico, luego, se hizo una etiquetación automática de este conjunto para el posterior reconocimiento y se analizaron las características extraídas en tres tipos de dominios, tales como, el tiempo, la frecuencia y tiempofrecuencia, formando un conjunto de 720 características. Además, para la estimación de características en el dominio de tiempo-frecuencia se realizó un experimento con el fin de encontrar los parámetros más representativos en la descomposición con la transformada Wavelet discreta y así, generar un espacio de representación apropiado. Se seleccionó un subconjunto de características y se entrenó con los clasificadores Máquinas de Vectores de Soporte (SVM), análisis discriminante lineal (LDA) y vecinos más cercanos (k-NN) con una validación cruzada de 10 iteraciones para evitar el sobreajuste. Se logra una exactitud superior al 95% con el clasificador SVM y 98% con LDA, no obstante, el k-NN no obtuvo un buen rendimiento en términos de la media geométrica debido a que requiere de una sintonización de los k-vecinos, lo que implica que la metodología propuesta facilita el reconocimiento del movimiento individual de los dedos uilizando el clasificador LDA.Magister en Automatización y Contro