Hand gesture recognition system based in computer vision and machine learning

"Lecture notes in computational vision and biomechanics series, ISSN 2212-9391, vol. 19"Hand gesture recognition is a natural way of human computer interaction and an area of very active research in computer vision and machine learning. This is an area with many different possible applications, giving users a simpler and more natural way to communicate with robots/systems interfaces, without the need for extra devices. So, the primary goal of gesture recognition research applied to Human-Computer Interaction (HCI) is to create systems, which can identify specific human gestures and use them to convey information or controlling devices. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. This paper presents a solution, generic enough, with the help of machine learning algorithms, allowing its application in a wide range of human-computer interfaces, for real-time gesture recognition. Experiments carried out showed that the system was able to achieve an accuracy of 99.4% in terms of hand posture recognition and an average accuracy of 93.72% in terms of dynamic gesture recognition. To validate the proposed framework, two applications were implemented. The first one is a real-time system able to help a robotic soccer referee judge a game in real time. The prototype combines a vision-based hand gesture recognition system with a formal language definition, the Referee CommLang, into what is called the Referee Command Language Interface System (ReCLIS). The second one is a real-time system able to interpret the Portuguese Sign Language. Sign languages are not standard and universal and the grammars differ from country to country. Although the implemented prototype was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.(undefined

Indian Sign Language Recognition through Hybrid ConvNet-LSTM Networks

Dynamic hand gesture recognition is a challenging task of Human-Computer Interaction (HCI) and Computer Vision. The potential application areas of gesture recognition include sign language translation, video gaming, video surveillance, robotics, and gesture-controlled home appliances. In the proposed research, gesture recognition is applied to recognize sign language words from real-time videos. Classifying the actions from video sequences requires both spatial and temporal features. The proposed system handles the former by the Convolutional Neural Network (CNN), which is the core of several computer vision solutions and the latter by the Recurrent Neural Network (RNN), which is more efficient in handling the sequences of movements. Thus, the real-time Indian sign language (ISL) recognition system is developed using the hybrid CNN-RNN architecture. The system is trained with the proposed CasTalk-ISL dataset. The ultimate purpose of the presented research is to deploy a real-time sign language translator to break the hurdles present in the communication between hearing-impaired people and normal people. The developed system achieves 95.99% top-1 accuracy and 99.46% top-3 accuracy on the test dataset. The obtained results outperform the existing approaches using various deep models on different datasets

Implementation and Performance Analysis of Different Hand Gesture Recognition Methods

In recent few years, hand gesture recognition is one of the advanced grooming technologies in the era of human-computer interaction and computer vision due to a wide area of application in the real world. But it is a very complicated task to recognize hand gesture easily due to gesture orientation, light condition, complex background, translation and scaling of gesture images. To remove this limitation, several research works have developed which is successfully decrease this complexity. However, the intention of this paper is proposed and compared four different hand gesture recognition system and apply some optimization technique on it which ridiculously increased the existing model accuracy and model running time. After employed the optimization tricks, the adjusted gesture recognition model accuracy was 93.21% and the run time was 224 seconds which was 2.14% and 248 seconds faster than an existing similar hand gesture recognition model. The overall achievement of this paper could be applied for smart home control, camera control, robot control, medical system, natural talk, and many other fields in computer vision and human-computer interaction

Performance Improvement of Data Fusion Based Real-Time Hand Gesture Recognition by Using 3-D Convolution Neural Networks With Kinect V2

Hand gesture recognition is one of the most active areas of research in computer vision. It provides an easy way to interact with a machine without using any extra devices. Hand gestures are natural and intuitive communication way for the human being to interact with his environment. In this paper, we propose Data Fusion Based Real-Time Hand Gesture Recognition using 3-D Convolutional Neural Networks and Kinect V2. To achieve the accurate segmentation and tracking with Kinect V2. Convolution neural network to improve the validity and robustness of the system. Based on the experimental results, the proposed model is accurate, robust and performance with very low processor utilization. The performance of our proposed system in real life application, which is controlling various devices using Kinect V2. Keywords: Hand gesture recognition, Kinect V2, data fusion, Convolutional Neural Networks DOI: 10.7176/IKM/9-1-02

Piano Pedaller: A Measurement System for Classification and Visualisation of Piano Pedalling Techniques

date-added: 2017-12-22 18:53:42 +0000 date-modified: 2017-12-22 19:03:05 +0000 keywords: piano gesture recognition, optical sensor, real-time data acquisition, bela, music informatics local-url: https://pdfs.semanticscholar.org/fd00/fcfba2f41a3f182d2000ca4c05fb2b01c475.pdf publisher-url: http://homes.create.aau.dk/dano/nime17/ bdsk-url-1: http://www.nime.org/proceedings/2017/nime2017_paper0062.pdfdate-added: 2017-12-22 18:53:42 +0000 date-modified: 2017-12-22 19:03:05 +0000 keywords: piano gesture recognition, optical sensor, real-time data acquisition, bela, music informatics local-url: https://pdfs.semanticscholar.org/fd00/fcfba2f41a3f182d2000ca4c05fb2b01c475.pdf publisher-url: http://homes.create.aau.dk/dano/nime17/ bdsk-url-1: http://www.nime.org/proceedings/2017/nime2017_paper0062.pdfdate-added: 2017-12-22 18:53:42 +0000 date-modified: 2017-12-22 19:03:05 +0000 keywords: piano gesture recognition, optical sensor, real-time data acquisition, bela, music informatics local-url: https://pdfs.semanticscholar.org/fd00/fcfba2f41a3f182d2000ca4c05fb2b01c475.pdf publisher-url: http://homes.create.aau.dk/dano/nime17/ bdsk-url-1: http://www.nime.org/proceedings/2017/nime2017_paper0062.pdfdate-added: 2017-12-22 18:53:42 +0000 date-modified: 2017-12-22 19:03:05 +0000 keywords: piano gesture recognition, optical sensor, real-time data acquisition, bela, music informatics local-url: https://pdfs.semanticscholar.org/fd00/fcfba2f41a3f182d2000ca4c05fb2b01c475.pdf publisher-url: http://homes.create.aau.dk/dano/nime17/ bdsk-url-1: http://www.nime.org/proceedings/2017/nime2017_paper0062.pdfdate-added: 2017-12-22 18:53:42 +0000 date-modified: 2017-12-22 19:03:05 +0000 keywords: piano gesture recognition, optical sensor, real-time data acquisition, bela, music informatics local-url: https://pdfs.semanticscholar.org/fd00/fcfba2f41a3f182d2000ca4c05fb2b01c475.pdf publisher-url: http://homes.create.aau.dk/dano/nime17/ bdsk-url-1: http://www.nime.org/proceedings/2017/nime2017_paper0062.pdfThis paper presents the results of a study of piano pedalling techniques on the sustain pedal using a newly designed measurement system named Piano Pedaller. The system is comprised of an optical sensor mounted in the piano pedal bearing block and an embedded platform for recording audio and sensor data. This enables recording the pedalling gesture of real players and the piano sound under normal playing conditions. Using the gesture data collected from the system, the task of classifying these data by pedalling technique was undertaken using a Support Vector Machine (SVM). Results can be visualised in an audio based score following application to show pedalling together with the player’s position in the score

Dynamic gesture recognition using transformation invariant hand shape recognition

In this thesis a detailed framework is presented for accurate real time gesture recognition. Our approach to develop a hand-shape classifier, trained using computer animation, along with its application in dynamic gesture recognition is described. The system developed operates in real time and provides accurate gesture recognition. It operates using a single low resolution camera and operates in Matlab on a conventional PC running Windows XP. The hand shape classifier outlined in this thesis uses transformation invariant subspaces created using Principal Component Analysis (PCA). These subspaces are created from a large vocabulary created in a systematic maimer using computer animation. In recognising dynamic gestures we utilise both hand shape and hand position information; these are two o f the main features used by humans in distinguishing gestures. Hidden Markov Models (HMMs) are trained and employed to recognise this combination of hand shape and hand position features. During the course o f this thesis we have described in detail the inspiration and motivation behind our research and its possible applications. In this work our emphasis is on achieving a high speed system that works in real time with high accuracy

Hand Gesture Recognition for Performing General Operations on Computer Machine

In this paper we introduce a prototype of hand gesture recognition system to interpret single handed static human hand gestures for the purpose of performing operations on the computer machine. This system makes use of an external webcam to capture the single handed static hand gesture of the user, identify the hand gesture and perform the desired associated action after matching the input gesture with the ones present in the database. Our real time hand recognition system is three fold: 1) feature extraction, 2) enhancement and 3) recognition. Extraction of the feature is achieved with the help of background subtraction. For the purpose of feature enhancement and image processing OpenCV libraries are imported and used. For recognition, various features with their own objectives are constructed from hand postures and compared according to the similarity measures and the best- matched posture is used for performing the desired action after matching. We can also build application interfaces for devices running on Operating systems such as air conditioners, smart TVs and computers by using this system. DOI: 10.17762/ijritcc2321-8169.150312

End-to-End Multiview Gesture Recognition for Autonomous Car Parking System

The use of hand gestures can be the most intuitive human-machine interaction medium. The early approaches for hand gesture recognition used device-based methods. These methods use mechanical or optical sensors attached to a glove or markers, which hinders the natural human-machine communication. On the other hand, vision-based methods are not restrictive and allow for a more spontaneous communication without the need of an intermediary between human and machine. Therefore, vision gesture recognition has been a popular area of research for the past thirty years. Hand gesture recognition finds its application in many areas, particularly the automotive industry where advanced automotive human-machine interface (HMI) designers are using gesture recognition to improve driver and vehicle safety. However, technology advances go beyond active/passive safety and into convenience and comfort. In this context, one of America’s big three automakers has partnered with the Centre of Pattern Analysis and Machine Intelligence (CPAMI) at the University of Waterloo to investigate expanding their product segment through machine learning to provide an increased driver convenience and comfort with the particular application of hand gesture recognition for autonomous car parking. In this thesis, we leverage the state-of-the-art deep learning and optimization techniques to develop a vision-based multiview dynamic hand gesture recognizer for self-parking system. We propose a 3DCNN gesture model architecture that we train on a publicly available hand gesture database. We apply transfer learning methods to fine-tune the pre-trained gesture model on a custom-made data, which significantly improved the proposed system performance in real world environment. We adapt the architecture of the end-to-end solution to expand the state of the art video classifier from a single image as input (fed by monocular camera) to a multiview 360 feed, offered by a six cameras module. Finally, we optimize the proposed solution to work on a limited resources embedded platform (Nvidia Jetson TX2) that is used by automakers for vehicle-based features, without sacrificing the accuracy robustness and real time functionality of the system