Heterogeneous hand gesture recognition using 3D dynamic skeletal data

International audienceHand gestures are the most natural and intuitive non-verbal communication medium while interacting with a computer, and related research efforts have recently boosted interest. Additionally, the identifiable features of the hand pose provided by current commercial inexpensive depth cameras can be exploited in various gesture recognition based systems, especially for Human-Computer Interaction. In this paper, we focus our attention on 3D dynamic gesture recognition systems using the hand pose information. Specifically, we use the natural structure of the hand topology-called later hand skeletal data-to extract effective hand kinematic descriptors from the gesture sequence. Descriptors are then encoded in a statistical and temporal representation using respectively a Fisher kernel and a multi-level temporal pyramid. A linear SVM classifier can be applied directly on the feature vector computed over the whole presegmented gesture to perform the recognition. Furthermore, for early recognition from continuous stream, we introduced a prior gesture detection phase achieved using a binary classifier before the final gesture recognition. The proposed approach is evaluated on three hand gesture datasets containing respectively 10, 14 and 25 gestures with specific challenging tasks. Also, we conduct an experiment to assess the influence of depth-based hand pose estimation on our approach. Experimental results demonstrate the potential of the proposed solution in terms of hand gesture recognition and also for a low-latency gesture recognition. Comparative results with state-of-the-art methods are reported

A cost analysis of transcription systems

We compare different approaches to transcribing natural history data and summarise the advantages and disadvantages of each approach using six case studies from four different natural history collections. We summarise the main cost considerations when planning a transcription project and discuss the limitations we current have in understanding the costs behind transcription and data quality.Non peer reviewe

SHREC'17 Track: 3D Hand Gesture Recognition Using a Depth and Skeletal Dataset

International audienceHand gesture recognition is recently becoming one of the most attractive field of research in pattern recognition. The objective of this track is to evaluate the performance of recent recognition approaches using a challenging hand gesture dataset containing 14 gestures, performed by 28 participants executing the same gesture with two different numbers of fingers. Two research groups have participated to this track, the accuracy of their recognition algorithms have been evaluated and compared to three other state-of-the-art approaches

Designing an herbarium digitisation workflow with built-in image quality management

Digitisation of natural history collections has evolved from creating databases for the recording of specimens’ catalogue and label data to include digital images of specimens. This has been driven by several important factors, such as a need to increase global accessibility to specimens and to preserve the original specimens by limiting their manual handling. The size of the collections pointed to the need of high throughput digitisation workflows. However, digital imaging of large numbers of fragile specimens is an expensive and time-consuming process that should be performed only once. To achieve this, the digital images produced need to be useful for the largest set of applications possible and have a potentially unlimited shelf life. The constraints on digitisation speed need to be balanced against the applicability and longevity of the images, which, in turn, depend directly on the quality of those images. As a result, the quality criteria that specimen images need to fulfil influence the design, implementation and execution of digitisation workflows. Different standards and guidelines for producing quality research images from specimens have been proposed; however, their actual adaptation to suit the needs of different types of specimens requires further analysis. This paper presents the digitisation workflow implemented by Meise Botanic Garden (MBG). This workflow is relevant because of its modular design, its strong focus on image quality assessment, its flexibility that allows combining in-house and outsourced digitisation, processing, preservation and publishing facilities and its capacity to evolve for integrating alternative components from different sources. The design and operation of the digitisation workflow is provided to showcase how it was derived, with particular attention to the built-in audit trail within the workflow, which ensures the scalable production of high-quality specimen images and how this audit trail ensures that new modules do not affect either the speed of imaging or the quality of the images produced

Essential omega-3 fatty acids tune microglial phagocytosis of synaptic elements in the mouse developing brain

AbstractOmega-3 fatty acids (n-3 PUFAs) are essential for the functional maturation of the brain. Westernization of dietary habits in both developed and developing countries is accompanied by a progressive reduction in dietary intake of n-3 PUFAs. Low maternal intake of n-3 PUFAs has been linked to neurodevelopmental diseases in Humans. However, the n-3 PUFAs deficiency-mediated mechanisms affecting the development of the central nervous system are poorly understood. Active microglial engulfment of synapses regulates brain development. Impaired synaptic pruning is associated with several neurodevelopmental disorders. Here, we identify a molecular mechanism for detrimental effects of low maternal n-3 PUFA intake on hippocampal development in mice. Our results show that maternal dietary n-3 PUFA deficiency increases microglia-mediated phagocytosis of synaptic elements in the rodent developing hippocampus, partly through the activation of 12/15-lipoxygenase (LOX)/12-HETE signaling, altering neuronal morphology and affecting cognitive performance of the offspring. These findings provide a mechanistic insight into neurodevelopmental defects caused by maternal n-3 PUFAs dietary deficiency.Infrastructure de Recherche Translationnelle pour les Biothérapies en NeurosciencesProgram Initiative d’Excellenc

Reconnaisance de gestes dynamiques de la main - De la création de descripteurs aux récentes méthodes d’apprentissage profond

Hand gestures are the most natural and intuitive non-verbal communica- tion medium while using a computer, and related research efforts have recently boosted interest. Additionally, data provided by current com- mercial inexpensive depth cameras can be exploited in various gesture recognition based systems. The area of hand gesture analysis covers hand pose estimation and gesture recognition. Hand pose estimation is con- sidered to be more challenging than other human part estimation due to the small size of the hand, its greater complexity and its important self occlusions. Beside, the development of a precise hand gesture recognition system is also challenging due to high dissimilarities between gestures derived from ad-hoc, cultural and/or individual factors of users. First, we propose an original framework to represent hand gestures by using hand shape and motion descriptors computed on 3D hand skeletal fea- tures. We use a temporal pyramid to model the dynamic of gestures and a linear SVM to perform the classification. Additionally, we create the Dynamic Hand Gesture dataset containing 2800 sequences of 14 gesture types. Evaluation results show the promising way of using hand skele- tal data to perform hand gesture recognition. Experiments are carried out on three hand gesture datasets, containing a set of fine and coarse heterogeneous gestures. Furthermore, results of our approach in terms of latency demonstrated improvements for a low-latency hand gesture recog- nition systems, where an early classification is needed. Then, we extend the study of hand gesture analysis to online recognition. Using a deep learning approach, we employ a transfer learning strategy to learn hand posture and shape features from depth image dataset originally created for hand pose estimation. Second, we model the temporal variations of the hand poses and its shapes using a recurrent deep learning technology. Finally, both information are merged to perform accurate prior detection and recognition of hand gestures. Experiments on two datasets demon- strate that the proposed approach is capable to detect an occurring gesture and to recognize its type far before its end.Les gestes de la main sont le moyen de communication non verbal le plus naturel et le plus intuitif lorsqu’il est question d’interaction avec un ordinateur. Les efforts de recherche qui y sont liés ont récemment relancé son intérêt. De plus, les données fournies par des caméras de pro- fondeur actuellement commercialisées à des prix abordables peuvent être exploitées dans une large variété de systèmes de reconnaissance de gestes. L’analyse des gestes de la main s’appuie sur l’estimation de la pose de la main et la reconnaissance de gestes. L’estimation de la pose de la main est considérée comme étant un défi plus important que l’estimation de la pose de n’importe quelle autre partie du corps du fait de la petite taille d’une main, de sa plus grande complexité et de ses nombreuses occultations. Par ailleurs, le développement d’un système précis de reconnaissance des gestes de la main est également difficile du fait des grandes dissimilarités entre les gestes dérivant de facteurs ad-hoc, culturels et/ou individuels in- hérents aux acteurs. Dans un premier temps, nous proposons un système original pour représenter les gestes de la main en utilisant des descrip- teurs de forme de main et de mouvement calculés sur des caractéristiques de squelette de main 3D. Nous utilisons une pyramide temporelle pour modéliser la dynamique des gestes et une machine à vecteurs de support (SVM) pour effectuer la classification. De plus, nous proposons une base de données de gestes de mains dynamiques contenant 2800 séquences de 14 types de gestes. Les résultats montrent une utilisation prometteuse des données de squelette pour reconnaître des gestes de la main. Des ex- périmentations sont menées sur trois ensembles de données, contenant un ensemble de gestes hétérogènes fins et grossiers. En outre, les résultats de notre approche en termes de latence ont démontré que notre système peut reconnaître un geste avant sa fin. Dans un second temps, nous étendons l’étude de l’analyse des gestes de main à une reconnaissance en ligne. En utilisant une approche d’apprentissage profond, nous employons une stratégie de transfert d’apprentissage afin d’entrainer des caractéristiques de pose et de forme de la main à partir d’images de profondeur d’une base de données crée à l’origine pour un problème d’estimation de la pose de la main. Nous modélisons ensuite les variations temporelles des poses de la main et de ses formes grâce à une methode d’apprentissage profond récurrente. Enfin, les deux informations sont fusionnées pour effectuer une détection préalable et une reconnaissance précise des gestes de main. Des expériences menées sur deux ensembles de données ont démontré que l’approche proposée est capable de détecter un geste qui se produit et de reconnaître son type bien avant sa fin

Reconnaissance de gestes dynamiques de la main : des méthodes traditionnelles aux récentes avancées en apprentissage profond

Les gestes de la main sont le moyen de communication non verbal le plus naturel et le plus intuitif lorsqu'il est question d'interaction avec un ordinateur. L'analyse des gestes de la main s'appuie sur l'estimation de la pose de la main et la reconnaissance de gestes. L'estimation de la pose de la main est considérée comme un défi difficile du fait de la petite taille d'une main, de sa plus grande complexité et de ses nombreuses occultations. Par ailleurs, le développement d'un système de reconnaissance des gestes est également difficile du fait des grandes dissimilarités entre les gestes dérivant de facteurs ad-hoc, culturels et/ou individuels inhérents aux acteurs. Nous proposons un système pour représenter les gestes de la main en utilisant des descripteurs de forme et de mouvement calculés sur des squelettes de main 3D. De plus, nous proposons une base de données de gestes de mains dynamiques contenant 14 types de gestes. Les résultats montrent une utilisation prometteuse des données de squelette pour reconnaître des gestes de main. Dans un second temps, nous étendons l'étude de l'analyse des gestes de main à une reconnaissance en ligne. En utilisant une approche d'apprentissage profond, nous employons une stratégie de transfert d'apprentissage afin d’entraîner des caractéristiques de pose et de forme de la main à partir d'images de profondeur d'une base de données crée à l'origine pour un problème d'estimation de la pose de la main. Nous modélisons ensuite les variations temporelles des poses de la main et de ses formes grâce à une méthode d'apprentissage profond récurrente. Enfin, les deux informations sont fusionnées pour détecter et reconnaître des gestes de main.Hand gestures are the most natural and intuitive non-verbal communication medium while using a computer, and related research efforts have recently boosted interest. The area of hand gesture analysis covers hand pose estimation and gesture recognition. Hand pose estimation is considered to be more challenging than other human part estimation due to the small size of the hand, its greater complexity and its important self occlusions. Beside, the development of a precise hand gesture recognition system is also challenging due to high dissimilarities between gestures derived from ad-hoc, cultural and/or individual factors of users. First, we propose an original framework to represent hand gestures by using hand shape and motion descriptors computed on 3D hand skeletal features. Additionally, we create the Dynamic Hand Gesture dataset containing 14 gesture types. Evaluation results show the promising way of using hand skeletal data to perform hand gesture recognition. Then, we extend the study of hand gesture analysis to online recognition. Using a deep learning approach, we employ a transfer learning strategy to learn hand posture and shape features from depth image dataset originally created for hand pose estimation. Second, we model the temporal variations of the hand poses and its shapes using a recurrent deep learning technology. Finally, both information are merged to perform accurate prior detection and recognition of hand gestures. Experiments on two datasets demonstrate that the proposed approach is capable to detect an occurring gesture and to recognize its type far before its end

3D Hand Gesture Recognition by Analysing Set-of-Joints Trajectories

International audienceHand gesture recognition is recently becoming one of the most attractive field of research in Pattern Recognition. In this paper, a skeleton-based approach is proposed for 3D hand gesture recognition. Specifically, we consider the sequential data of hand geometric configuration to capture the hand shape variation, and explore the temporal character of hand motion. 3D Hand gesture are represented as a set of relevant spatiotemporal motion trajectories of hand-parts in an Euclidean space. Trajectories are then interpreted as elements lying on Riemannian manifold of shape space to capture their shape variations and achieve gesture recognition using a linear SVM classifier. The proposed approach is evaluated on a challenging hand gesture dataset containing 14 gestures, performed by 20 participants performing the same gesture with two di↵erent numbers of fingers. Experimental results show that our skeleton-based approach consistently achieves superior performance over a depth-based approach

3D Hand Gesture Recognition by Analysing Set-of-Joints Trajectories

International audienceHand gesture recognition is recently becoming one of the most attractive field of research in Pattern Recognition. In this paper, a skeleton-based approach is proposed for 3D hand gesture recognition. Specifically, we consider the sequential data of hand geometric configuration to capture the hand shape variation, and explore the temporal character of hand motion. 3D Hand gesture are represented as a set of relevant spatiotemporal motion trajectories of hand-parts in an Euclidean space. Trajectories are then interpreted as elements lying on Riemannian manifold of shape space to capture their shape variations and achieve gesture recognition using a linear SVM classifier. The proposed approach is evaluated on a challenging hand gesture dataset containing 14 gestures, performed by 20 participants performing the same gesture with two di↵erent numbers of fingers. Experimental results show that our skeleton-based approach consistently achieves superior performance over a depth-based approach

Skeleton-Based Dynamic Hand Gesture Recognition

International audienceIn this paper, a new skeleton-based approach is proposed for 3D hand gesture recognition. Specifically, we exploit the geometric shape of the hand to extract an effective de-scriptor from hand skeleton connected joints returned by the Intel RealSense depth camera. Each descriptor is then encoded by a Fisher Vector representation obtained using a Gaussian Mixture Model. A multi-level representation of Fisher Vectors and other skeleton-based geometric features is guaranteed by a temporal pyramid to obtain the final feature vector, used later to achieve the classification by a linear SVM classifier. The proposed approach is evaluated on a challenging hand gesture dataset containing 14 gestures, performed by 20 participants performing the same gesture with two different numbers of fingers. Experimental results show that our skeleton-based approach consistently achieves superior performance over a depth-based approach