International audienceMost of the developed societies are experiencing an aging trend of theirpopulation. Aging is correlated with cognitive impairment such as dementia and its mostcommon type: Alzheimer's disease. So, there is an urgent need to develop technological toolsto help doctors to do early and precise diagnoses of cognitive decline.Inability to correctly perform purposeful skilled movements with hands and other forelimbsmost commonly is associated with Alzheimer’s disease [1]. These patients have difficulty tocorrectly imitate hand gestures and mime tool use, e.g. pretend to brush one’s hair. Thesepatients make spatial and temporal error

Bourgeois, Jérémy

Bremond, Francois

Chapoulie, Emmanuelle

Negin, Farhood

Robert, Philippe

HAL-UNICE

Praxis and Gesture Recognition

INRIA a CCSD electronic archive server

HAL Id: hal-01416372https://hal.inria.fr/hal-01416372Submitted on 14 Dec 2016HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Praxis and Gesture RecognitionFarhood Negin, Jérémy Bourgeois, Emmanuelle Chapoulie, Philippe Robert,Francois BremondTo cite this version:Farhood Negin, Jérémy Bourgeois, Emmanuelle Chapoulie, Philippe Robert, Francois Bremond.Praxis and Gesture Recognition. The 10th World Conference of Gerontechnology (ISG 2016), Sep2016, Nice, France. ￿hal-01416372￿Praxis and Gesture Recognition  Farhood Negin, Jérémy Bourgeois, Emmanuelle Chapoulie, Philippe Robert, Francois. Bremond  Keywords: Gesture Recognition, Static and Dynamic Gestures, Alzheimer Disease, Reflection Time, Motion Descriptors   Address: INRIA Méditerranée – Sophia Antipolis, France; EA CoBTeK, Université Cote d’Azur, Nice, France; Centre Mémoire de Ressources et de Recherche, CHU de Nice, Nice, France. e-mail: farhood.negin@inria.fr   Introduction: Most of the developed societies are experiencing an aging trend of their population. Aging is correlated with cognitive impairment such as dementia and its most common type: Alzheimer's disease. So, there is an urgent need to develop technological tools to help doctors to do early and precise diagnoses of cognitive decline.  Inability to correctly perform purposeful skilled movements with hands and other forelimbs most commonly is associated with Alzheimer’s disease [1]. These patients have difficulty to correctly imitate hand gestures and mime tool use, e.g. pretend to brush one’s hair. These patients make spatial and temporal errors. Goal: We propose a gesture recognition and evaluation framework as a complementary tool to help doctors to spot symptoms of cognitive impairment at its early stages. It is also useful to assess one’s cognitive status. First, the algorithm classifies the defined gestures in the gestures set and then it evaluates gestures of the same category to see how well they perform compared to correct gesture templates. Methods Shape and motion descriptors such as HOG (histogram of oriented gradient) [2] and HOF (histogram of optical flow) [3] are an efficient clue to characterize different gestures (Figure 1a). Extracted descriptors are utilized as input to train the classifiers. We use bag-of-visual-words approach to characterize gestures with descriptors. The classification happens in two steps: first we train a classifier to distinguish different gestures and after, we train another classifier with correct and incorrect samples of the same class. This way, we could recognize which gesture is performed and whether it is performed accurately or not.              Experiments and Results: the framework is fed by input data which comes from a depth sensor (Kinect®, Microsoft©). At first phase, the correct samples of gestures which are Figure 1. a) Extracted motion descriptors while performing a gesture b) virtual avatar guides patients in a virtual reality environment a b     performed by clinicians, are recorded. We train the framework using correct instances of each gesture class. In the second phase, participants were asked to perform the gestures. We use virtual reality as modality to interact with subjects to make the experiments more immersive and realistic experience. First an avatar performs a specific gesture and then she asks the subject to repeat the same gesture (Figure 1b).  In this work, we analyze two categories of gestures. First category is dynamic gestures where the whole motion of the hands is considered as a complete gesture. Second category of gestures is static gestures where only a static pose of hands is the desired gesture. For static gestures, we also need to detect this key frame. Moreover, reaction time which starts after avatar asked the subject to do the gesture, until subject really starts to perform the gesture, could be an important diagnostic factor. Our algorithm uses motion descriptors to detect key frames and reaction time. In the preliminary tests, our framework successfully recognized more than 80% of the dynamic gestures. It also detects key frames and reaction time with a high precision. Conclusion: the proposed gesture recognition framework to help doctors by providing a complete assessment of gestures performed by subject.  Future Work: we will do extensive evaluation on large dataset collected from cognitively impaired patients with different levels of impairment. We will investigate more feature, like deep features on static gestures to achieve better classification accuracy. Detecting head and its relative position to hands could be a helpful clue to characterize different gestures.  References 1. Mozaz, M., Garaigordobil, M., Rothi, L.J.G., Anderson, J., Crucian, G.P.,Heilman, K.M., 2006. Posture recognition in alzheimers disease. Brain and cognition 62, 241–245. 2. Dalal, Navneet, and Bill Triggs. "Histograms of oriented gradients for human detection." Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on. Vol. 1. IEEE, 2005. 3. Dalal, Navneet, Bill Triggs, and Cordelia Schmid. "Human detection using oriented histograms of flow and appearance." Computer Vision–ECCV 2006. Springer Berlin Heidelberg, 2006. 428-441.   

Archive Ouverte en Sciences de l'Information et de la Communication

https://hal.inria.fr/hal-01416372/file/Farhood_ISG2016%20%281%29.pdf

Praxis and Gesture Recognition

Abstract

Similar works

Full text

Available Versions

HAL-UNICE

INRIA a CCSD electronic archive server

Archive Ouverte en Sciences de l'Information et de la Communication