Hand Tracking and Gesture Recognition for Human-Computer Interaction

The proposed work is part of a project that aims for the control of a videogame based on hand gesture recognition. This goal implies the restriction of real-time response and unconstrained environments. In this paper we present a real-time algorithm to track and recognise hand gestures for interacting with the videogame. This algorithm is based on three main steps: hand segmentation, hand tracking and gesture recognition from hand features. For the hand segmentation step we use the colour cue due to the characteristic colour values of human skin, its invariant properties and its computational simplicity. To prevent errors from hand segmentation we add a second step, hand tracking. Tracking is performed assuming a constant velocity model and using a pixel labeling approach. From the tracking process we extract several hand features that are fed to a finite state classifier which identifies the hand configuration. The hand can be classified into one of the four gesture classes or one of the four different movement directions. Finally, using the system's performance evaluation results we show the usability of the algorithm in a videogame environment

Vision-based hand gesture interaction using particle filter, principle component analysis and transition network

Vision-based human-computer interaction is becoming important nowadays. It offers natural interaction with computers and frees users from mechanical interaction devices, which is favourable especially for wearable computers. This paper presents a human-computer interaction system based on a conventional webcam and hand gesture recognition. This interaction system works in real time and enables users to control a computer cursor with hand motions and gestures instead of a mouse. Five hand gestures are designed on behalf of five mouse operations: moving, left click, left-double click, right click and no-action. An algorithm based on Particle Filter is used for tracking the hand position. PCA-based feature selection is used for recognizing the hand gestures. A transition network is also employed for improving the accuracy and reliability of the interaction system. This interaction system shows good performance in the recognition and interaction test

A Novel Approach for Operating Electrical Appliances Using Hand Gesture Recognition

Vision-based automatic hand gesture acknowledgement has been a very active research theme in recent years with inspiring applications such as human computer interaction (HCI), electronics device command, and signal language understanding. Hand sign recognition is presented through a curvature space procedure in which finding the boundary contours of the hand are engaged. This is a robust approach that is scale, translation and rotation invariant on the hand poses yet it is computationally demanding. A method for signal acknowledgement for signal language understanding has been proposed in computer vision. Human interaction involves various hand processing task like hand detection, recognition and hand tracking. This technology mainly focuses on the needs of physically challenged group of people and helps them to operate just by showing hand gestures. Thus, our project is aimed at making a system that could recognized human gesture through computer vision

Hand gesture recognition in uncontrolled environments

Human Computer Interaction has been relying on mechanical devices to feed information into computers with low efficiency for a long time. With the recent developments in image processing and machine learning methods, the computer vision community is ready to develop the next generation of Human Computer Interaction methods, including Hand Gesture Recognition methods. A comprehensive Hand Gesture Recognition based semantic level Human Computer Interaction framework for uncontrolled environments is proposed in this thesis. The framework contains novel methods for Hand Posture Recognition, Hand Gesture Recognition and Hand Gesture Spotting. The Hand Posture Recognition method in the proposed framework is capable of recognising predefined still hand postures from cluttered backgrounds. Texture features are used in conjunction with Adaptive Boosting to form a novel feature selection scheme, which can effectively detect and select discriminative texture features from the training samples of the posture classes. A novel Hand Tracking method called Adaptive SURF Tracking is proposed in this thesis. Texture key points are used to track multiple hand candidates in the scene. This tracking method matches texture key points of hand candidates within adjacent frames to calculate the movement directions of hand candidates. With the gesture trajectories provided by the Adaptive SURF Tracking method, a novel classi�er called Partition Matrix is introduced to perform gesture classification for uncontrolled environments with multiple hand candidates. The trajectories of all hand candidates extracted from the original video under different frame rates are used to analyse the movements of hand candidates. An alternative gesture classifier based on Convolutional Neural Network is also proposed. The input images of the Neural Network are approximate trajectory images reconstructed from the tracking results of the Adaptive SURF Tracking method. For Hand Gesture Spotting, a forward spotting scheme is introduced to detect the starting and ending points of the prede�ned gestures in the continuously signed gesture videos. A Non-Sign Model is also proposed to simulate meaningless hand movements between the meaningful gestures. The proposed framework can perform well with unconstrained scene settings, including frontal occlusions, background distractions and changing lighting conditions. Moreover, it is invariant to changing scales, speed and locations of the gesture trajectories

Hand Detection using HSV Model

Natural Human Computer Interaction (HCI) is the demand of today’s technology oriented world. Detecting and tracking of face and hands are important for gesture recognition. Skin detection is a very popular and useful technique for detecting and tracking human-body parts. It has been much attention mainly because of its vast range of applications such as, face detection and tracking, naked people detection, hand detection and tracking, people retrieval in databases and Internet, etc. Many models and algorithms are being used for detection of face, hand and its gesture. Hand detection using model or classification is to build a decision rule that will discriminate between skin and non-skin pixels. Identifying skin color pixels involves finding the range of values for which most skin pixels would fall in a given color space. All external factors will be eliminated to detect the hand and its color in the image in complex background. Keywords: image segmentation, hand detection, hci, computer vision, RGB, HS

Gesture Recognition for Enhancing Human Computer Interaction

Gesture recognition is critical in human-computer communication. As observed, a plethora of current technological developments are in the works, including biometric authentication, which we see all the time in our smartphones. Hand gesture focus, a frequent human-computer interface in which we manage our devices by presenting our hands in front of a webcam, can benefit people of different backgrounds. Some of the efforts in human-computer interface include voice assistance and virtual mouse implementation with voice commands, fingertip recognition and hand motion tracking based on an image in a live video. Human Computer Interaction (HCI), particularly vision-based gesture and object recognition, is becoming increasingly important. Hence, we focused to design and develop a system for monitoring fingers using extreme learning-based hand gesture recognition techniques. Extreme learning helps in quickly interpreting the hand gestures with improved accuracy which would be a highly useful in the domains like healthcare, financial transactions and global busines

Human Computer Interaction Based HEMD Using Hand Gesture

Hand gesture based Human-Computer-Interaction (HCI) is one of the most normal and spontaneous ways to communicate between people and apparatus to present a hand gesture recognition system with Webcam, Operates robustly in unrestrained environment and is insensible to hand variations and distortions. This classification consists of two major modules, that is, hand detection and gesture recognition. Diverse from conventional vision-based hand gesture recognition methods that use color-markers for hand detection, this system uses both the depth and color information from Webcam to detect the hand shape, which ensures the sturdiness in disorderly environments. Assurance its heftiness to input variations or the distortions caused by the low resolution of webcam, to apply a novel shape distance metric called Handle Earth Mover\u27s Distance (HEMD) for hand gesture recognition. Consequently, in this paper concept operates accurately and efficiently. The intend of this paper is to expand robust and resourceful hand segmentation algorithm where three algorithms for hand segmentation using different color spaces with required thresholds have were utilized. Hand tracking and segmentation algorithm is found to be most resourceful to handle the challenge of apparition based organization such as skin dye detection. Noise may hold, for a moment, in the segmented image due to lively background. Tracking algorithm was developed and applied on the segmented hand contour for elimination of unnecessary background nois

Vision-based gesture recognition system for human-computer interaction

Hand gesture recognition, being a natural way of human computer interaction, is an area of active research in computer vision and machine learning. This is an area with many different possible applications, giving users a simpler and more natural way to communicate with robots/systems interfaces, without the need for extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them to convey information or for device control. This work intends to study and implement a solution, generic enough, able to interpret user commands, composed of a set of dynamic and static gestures, and use those solutions to build an application able to work in a realtime human-computer interaction systems. The proposed solution is composed of two modules controlled by a FSM (Finite State Machine): a real time hand tracking and feature extraction system, supported by a SVM (Support Vector Machine) model for static hand posture classification and a set of HMMs (Hidden Markov Models) for dynamic single stroke hand gesture recognition. The experimental results showed that the system works very reliably, being able to recognize the set of defined commands in real-time. The SVM model for hand posture classification, trained with the selected hand features, achieved an accuracy of 99,2%. The proposed solution as the advantage of being computationally simple to train and use, and at the same time generic enough, allowing its application in any robot/system command interface

Hand Tracking based on Hierarchical Clustering of Range Data

Fast and robust hand segmentation and tracking is an essential basis for gesture recognition and thus an important component for contact-less human-computer interaction (HCI). Hand gesture recognition based on 2D video data has been intensively investigated. However, in practical scenarios purely intensity based approaches suffer from uncontrollable environmental conditions like cluttered background colors. In this paper we present a real-time hand segmentation and tracking algorithm using Time-of-Flight (ToF) range cameras and intensity data. The intensity and range information is fused into one pixel value, representing its combined intensity-depth homogeneity. The scene is hierarchically clustered using a GPU based parallel merging algorithm, allowing a robust identification of both hands even for inhomogeneous backgrounds. After the detection, both hands are tracked on the CPU. Our tracking algorithm can cope with the situation that one hand is temporarily covered by the other hand.Comment: Technical Repor