Gait speeds classifications by supervised modulation based machine-learning using Kinect camera

Early indication of some diseases such as Parkinson and Multiple Sclerosis often manifests with walking difficulties. Gait analysis provides vital information for assessing the walking patterns during the locomotion, especially when the outcomes are quantitative measures. This paper explores methods that can respond to the changes in the gait features during the swing stage using Kinect Camera, a low cost, marker-free, and portable device offered by Microsoft. Kinect has been exploited for tracking the skeletal positional data of body joints to assess and evaluate the gait performance. Linear kinematic gait features are extracted to discriminate between walking speeds by using five supervised modulation based machine-learning classifiers as follow: Decision Trees (DT), linear/nonlinear Support Vector Machines (SVMs), subspace discriminant and k-Nearest Neighbour (k-NN). The role of modulation techniques such as Frequency Modulation (FM) for increasing the efficiency of classifiers have been explored. The experimental results show that all five classifiers can successfully distinguish gait futures signal associated with walking patterns with high accuracy (average expected value of 86.19% with maximum of 92.9%). This validates the capability of the presented methodology in detecting key “indicators” of health events. Keywords: Gait Analysis, Kinematic Gait Features, Amplitude and Frequency Modulations, Baseband Signal, Passband Mapping, Machine-Learning, Classification Techniqu

An original framework for understanding human actions and body language by using deep neural networks

The evolution of both fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has allowed the development of efficient automatic systems for the analysis of people's behaviour. By studying hand movements it is possible to recognize gestures, often used by people to communicate information in a non-verbal way. These gestures can also be used to control or interact with devices without physically touching them. In particular, sign language and semaphoric hand gestures are the two foremost areas of interest due to their importance in Human-Human Communication (HHC) and Human-Computer Interaction (HCI), respectively. While the processing of body movements play a key role in the action recognition and affective computing fields. The former is essential to understand how people act in an environment, while the latter tries to interpret people's emotions based on their poses and movements; both are essential tasks in many computer vision applications, including event recognition, and video surveillance. In this Ph.D. thesis, an original framework for understanding Actions and body language is presented. The framework is composed of three main modules: in the first one, a Long Short Term Memory Recurrent Neural Networks (LSTM-RNNs) based method for the Recognition of Sign Language and Semaphoric Hand Gestures is proposed; the second module presents a solution based on 2D skeleton and two-branch stacked LSTM-RNNs for action recognition in video sequences; finally, in the last module, a solution for basic non-acted emotion recognition by using 3D skeleton and Deep Neural Networks (DNNs) is provided. The performances of RNN-LSTMs are explored in depth, due to their ability to model the long term contextual information of temporal sequences, making them suitable for analysing body movements. All the modules were tested by using challenging datasets, well known in the state of the art, showing remarkable results compared to the current literature methods

DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks

3D scene understanding is important for robots to interact with the 3D world in a meaningful way. Most previous works on 3D scene understanding focus on recognizing geometrical or semantic properties of the scene independently. In this work, we introduce Data Associated Recurrent Neural Networks (DA-RNNs), a novel framework for joint 3D scene mapping and semantic labeling. DA-RNNs use a new recurrent neural network architecture for semantic labeling on RGB-D videos. The output of the network is integrated with mapping techniques such as KinectFusion in order to inject semantic information into the reconstructed 3D scene. Experiments conducted on a real world dataset and a synthetic dataset with RGB-D videos demonstrate the ability of our method in semantic 3D scene mapping.Comment: Published in RSS 201