Bayesian fusion of hidden Markov models for understanding bimanual movements

Understanding hand and body gestures is a part of a wide spectrum of current research in computer vision and human-computer interaction. A part of this can be the recognition of movements in which the two hands move simultaneously to do something or imply a meaning. We present a Bayesian network for fusing hidden Markov models in order to recognise a bimanual movement. A bimanual movement is tracked and segmented by a tracking algorithm. Hidden Markov models are assigned to the segments in order to learn and recognize the partial movement within each segment. A Bayesian network fuses the HMMs in order to perceive the movement of the two hands as a single entity

Comparison of EEG Pattern Recognition of Motor Imagery for Finger Movement Classification

The detection of a hand movement beforehand can be a beneficent tool to control a prosthetic hand for upper extremity rehabilitation. To be able to achieve smooth control, the intention detection is acquired from the human body, especially from brain signal or electroencephalogram (EEG) signal. However, many constraints hamper the development of this brain-computer interface (BCI, especially for finger movement detection). Most of the researchers have focused on the detection of the left and right-hand movement. This article presents the comparison of various pattern recognition method for recognizing five individual finger movements, i.e., the thumb, index, middle, ring, and pinky finger movements. The EEG pattern recognition utilized common spatial pattern (CSP) for feature extraction. As for the classifier, four classifiers, i.e., random forest (RF), support vector machine (SVM), k-nearest neighborhood (kNN), and linear discriminant analysis (LDA) were tested and compared to each other. The experimental results indicated that the EEG pattern recognition with RF achieved the best accuracy of about 54%. Other published publication reported that the classification of the individual finger movement is still challenging and need more efforts to make the best performance

Towards Developing an Effective Hand Gesture Recognition System for Human Computer Interaction: A Literature Survey

Gesture recognition is a mathematical analysis of movement of body parts (hand / face) done with the help of computing device. It helps computers to understand human body language and build a more powerful link between humans and machines. Many research works are developed in the field of hand gesture recognition. Each works have achieved different recognition accuracies with different hand gesture datasets, however most of the firms are having insufficient insight to develop necessary achievements to meet their development in real time datasets. Under such circumstances, it is very essential to have a complete knowledge of recognition methods of hand gesture recognition, its strength and weakness and the development criteria as well. Lots of reports declare its work to be better but a complete relative analysis is lacking in these works. In this paper, we provide a study of representative techniques for hand gesture recognition, recognition methods and also presented a brief introduction about hand gesture recognition. The main objective of this work is to highlight the position of various recognition techniqueswhich can indirectly help in developing new techniques for solving the issues in the hand gesture recognition systems. Moreover we present a concise description about the hand gesture recognition systems recognition methods and the instructions for future research

Hand tracking and bimanual movement understanding

Bimanual movements are a subset ot human movements in which the two hands move together in order to do a task or imply a meaning A bimanual movement appearing in a sequence of images must be understood in order to enable computers to interact with humans in a natural way This problem includes two main phases, hand tracking and movement recognition. We approach the problem of hand tracking from a neuroscience point ot view First the hands are extracted and labelled by colour detection and blob analysis algorithms In the presence of the two hands one hand may occlude the other occasionally Therefore, hand occlusions must be detected in an image sequence A dynamic model is proposed to model the movement of each hand separately Using this model in a Kalman filtering proccss the exact starting and end points of hand occlusions are detected We exploit neuroscience phenomena to understand the beha\ tour of the hands during occlusion periods Based on this, we propose a general hand tracking algorithm to track and reacquire the hands over a movement including hand occlusion The advantages of the algorithm and its generality are demonstrated in the experiments. In order to recognise the movements first we recognise the movement of a hand Using statistical pattern recognition methods (such as Principal Component Analysis and Nearest Neighbour) the static shape of each hand appearing in an image is recognised A Graph- Matching algorithm and Discrete Midden Markov Models (DHMM) as two spatio-temporal pattern recognition techniques are imestigated tor recognising a dynamic hand gesture For recognising bimanual movements we consider two general forms ot these movements, single and concatenated periodic We introduce three Bayesian networks for recognising die movements The networks are designed to recognise and combinc the gestures of the hands in order to understand the whole movement Experiments on different types ot movement demonstrate the advantages and disadvantages of each network

RGBD Human-Hand recognition for the Interaction with Robot-Hand

New low cost sensors and the new open free libraries for 3D image processing are permitting to achieve important advances for robot vision applications such as tridimensional object recognition, semantic mapping, navigation and localization of robots, human detection and/or gesture recognition for human-machine interaction. In this paper, a method to recognize the human hand and to track the fingers is proposed. This new method is based on point clouds from range images, RGBD. It does not require visual marks, camera calibration, environment knowledge and complex expensive acquisition systems. Furthermore, this method has been implemented to create a human interface in order to move a robot hand. The human hand is recognized and the movement of the fingers is analyzed. Afterwards, it is imitated from a Barret hand, using communication events programmed from ROS.This work was supported in part by the Valencia Regional Government and the Research and Innovation Vice-president Office of the University of Alicante for their financial support through the projects GV2012/102 and GRE10-16, respectively

Dynamic Hand Gesture-Featured Human Motor Adaptation in Tool Delivery using Voice Recognition

Human-robot collaboration has benefited users with higher efficiency towards interactive tasks. Nevertheless, most collaborative schemes rely on complicated human-machine interfaces, which might lack the requisite intuitiveness compared with natural limb control. We also expect to understand human intent with low training data requirements. In response to these challenges, this paper introduces an innovative human-robot collaborative framework that seamlessly integrates hand gesture and dynamic movement recognition, voice recognition, and a switchable control adaptation strategy. These modules provide a user-friendly approach that enables the robot to deliver the tools as per user need, especially when the user is working with both hands. Therefore, users can focus on their task execution without additional training in the use of human-machine interfaces, while the robot interprets their intuitive gestures. The proposed multimodal interaction framework is executed in the UR5e robot platform equipped with a RealSense D435i camera, and the effectiveness is assessed through a soldering circuit board task. The experiment results have demonstrated superior performance in hand gesture recognition, where the static hand gesture recognition module achieves an accuracy of 94.3\%, while the dynamic motion recognition module reaches 97.6\% accuracy. Compared with human solo manipulation, the proposed approach facilitates higher efficiency tool delivery, without significantly distracting from human intents.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Research on Hand Action Pattern Recognition of Bionic Limb Based on Surface Electromyography

Hands are important parts of a human body. It is not only the main tool for people to engage in productive labor, but also an important communication tool. When the hand moves, the human body produces a kind of signal named surface electromyography (sEMG), which is a kind of electrophysiological signal that accompanies muscle activity. It contains a lot of information about human movement consciousness. The bionic limb is driven by multi-degree-freedom control, which is got by converting the recognition result and this can meet the urgent need of people with disabilities for autonomous operation. A profound study of hand action pattern technology based on sEMG signals can achieve the ability of the bionic limb to distinguish the hand action fast and accurately. From the perspective of the pattern recognition of the bionic limb, this paper discussed the human hand action pattern recognition technology of sEMG. By analyzing and summarizing the current development of human hand movement recognition, the author proposed a bionic limb schema based on artificial neural network and the improved DT-SVM hand action recognition system. According to the research results, it is necessary to expand the type and total amount of hand movements and gesture recognition, in order to adapt to the objective requirements of the diversity of hand action patterns in the application of the bionic limb