Heterogeneous hand gesture recognition using 3D dynamic skeletal data

International audienceHand gestures are the most natural and intuitive non-verbal communication medium while interacting with a computer, and related research efforts have recently boosted interest. Additionally, the identifiable features of the hand pose provided by current commercial inexpensive depth cameras can be exploited in various gesture recognition based systems, especially for Human-Computer Interaction. In this paper, we focus our attention on 3D dynamic gesture recognition systems using the hand pose information. Specifically, we use the natural structure of the hand topology-called later hand skeletal data-to extract effective hand kinematic descriptors from the gesture sequence. Descriptors are then encoded in a statistical and temporal representation using respectively a Fisher kernel and a multi-level temporal pyramid. A linear SVM classifier can be applied directly on the feature vector computed over the whole presegmented gesture to perform the recognition. Furthermore, for early recognition from continuous stream, we introduced a prior gesture detection phase achieved using a binary classifier before the final gesture recognition. The proposed approach is evaluated on three hand gesture datasets containing respectively 10, 14 and 25 gestures with specific challenging tasks. Also, we conduct an experiment to assess the influence of depth-based hand pose estimation on our approach. Experimental results demonstrate the potential of the proposed solution in terms of hand gesture recognition and also for a low-latency gesture recognition. Comparative results with state-of-the-art methods are reported

Indexing and retrieval VRML models

International audienceIn this paper we present a three-dimensional model retrieval system. A three-dimensional model is described by two invariant descriptors: a shape index and a histogram of distances between meshes. This work focuses on extracting invariant descriptors that well represent a three-dimensional model, and on combining theses descriptors in order to get a better retrieval performance. An experimental evaluation demonstrates the good performance of the approach

3D Mesh Skeleton Extraction Using Topological and Geometrical Analyses

International audienceThis paper describes a novel and unified approach for Reeb graph construction and simplification as well as constriction approximation on 3D polygonal meshes. The key idea of our algorithm is that discrete contours - curves carried by the edges of the mesh and approximating the continuous contours of a mapping function - encode both topological and geometrical shape characteristics. Firstly, mesh feature points are computed. Then they are used as geodesic origins for the computation of an invariant mapping function that reveals the shape most significant features. Secondly, for each vertex in the mesh, its discrete contour is computed. As the set of discrete contours recovers the whole surface, each of them can be analyzed, both to detect topological changes or constrictions. Constriction approximations enable Reeb graphs refinement into more visually meaningful skeletons, that we refer as enhanced topological skeletons. Without pre-processing stages and without input parameters, our method provides nice-looking and affine- invariant skeletons, with satisfactory execution times. This makes enhanced topological skeletons good candidates for applications needing high level shape representations, such as mesh deformation (experimented in this paper), retrieval, compression, metamorphosis, etc

Topology driven 3D mesh hierarchical segmentation

International audienceIn this paper, we propose to address the semantic- oriented 3D mesh hierarchical segmentation problem, using enhanced topological skeletons. This high level information drives both the feature boundary computation as well as the feature hierarchy definition. Proposed hierarchical scheme is based on the key idea that the topology of a feature is a more important decomposition criterion than its geometry. First, the enhanced topological skeleton of the input triangulated surface is constructed. Then it is used to delimit the core of the object and to identify junction areas. This second step results in a fine segmentation of the object. Finally, a fine to coarse strategy enables a semantic-oriented hierarchical composition of features, subdividing human limbs into arms and hands for example. Method performance is evaluated according to seven criteria enumerated in latest segmentation surveys. Thanks to the high level description it uses as an input, presented approach results, with low computation times, in robust and meaningful compatible hierarchical decompositions

Learning Boundary Edges for 3D-Mesh Segmentation

International audienceThis paper presents a 3D-mesh segmentation algorithm based on a learning approach. A large database of manually segmented 3D-meshes is used to learn a boundary edge function. The function is learned using a classifier which automatically selects from a pool of geometric features the most relevant ones to detect candidate boundary edges. We propose a processing pipeline that produces smooth closed boundaries using this edge function. This pipeline successively selects a set of candidate boundary contours, closes them and optimizes them using a snake movement. Our algorithm was evaluated quantitatively using two different segmentation benchmarks and was shown to outperform most recent algorithms from the state-of-the-art

SHREC'17 Track: 3D Hand Gesture Recognition Using a Depth and Skeletal Dataset

International audienceHand gesture recognition is recently becoming one of the most attractive field of research in pattern recognition. The objective of this track is to evaluate the performance of recent recognition approaches using a challenging hand gesture dataset containing 14 gestures, performed by 28 participants executing the same gesture with two different numbers of fingers. Two research groups have participated to this track, the accuracy of their recognition algorithms have been evaluated and compared to three other state-of-the-art approaches

Contributions à la recherche et à l'analyse de modèles 3D

The use of three-dimensional models in the multimedia applications, is growing both in number and in size. The development of mode- ling tools, 3D-scanners, graphic accelerated hardware, Web3D and so on, offers access to three-dimensional materials of high quality. The constantly increasing needs concerning these kinds of data, are rapidly changing. While it becomes more and more easy to create new 3D-models, what about process and analysis after the creation of the 3D-models? Today, the 3D designer no longer asks : "How to create a new 3D-model?", but more probably "How to retrieve an existing 3D-model that is similar to those I already own in order to reuse it?" and "How to get the inner structure of a 3D-mesh model without any a priori knowledge on it?" This habilitation thesis aims to provide some answers to these two questions. In response to the first question, we developed a new Bayesian framework to retrieve 3D-models from a query made of one or more 2D- views, or of an entire 3D-model. The framework has been tested in an industrial application context and with an international benchmark. Each of these experiments has shown excellent results. The second question has been addressed in terms of topological analysis of the 3D-meshes with the help of Reeb graphs. This theoretical work has been applied to several practical domains, such as automatic 3D-mesh deformation, 3D-model retrieval, and 3D-mesh segmentation, and has always highlighted outstanding results. Finally, the segmentation of 3D-meshes, which is a frequent pre-processing step before any other analysis of the mesh, has drawn our attention. We proposed a reliable and robust metric to compare segmentations and evaluate the performances of the 3D-mesh segmentation methods, as well as a new learning-based segmentation approach that out- performs existing ones. To conclude, new perspectives of research on 3D- meshes are open.L'utilisation de modèles tridimensionnels dans les applications multimédia, prend de l'ampleur de jour en jour. Le développement des outils de modélisation, des scanners 3D, des cartes graphiques accélérées, du Web3D, etc. ouvre l'accès à des données tridimensionnelles de grande qualité. Les besoins, sans cesse croissants, concernant ce type de données, changent rapidement. S'il devient de plus en plus facile de créer de nouveaux modèles 3D, qu'en est-il du traitement et de l'analyse de ces modèles après leur création ? De nos jours, le concepteur d'objets 3D ne pose plus la question : " Comment créer un nouvel objet 3D ? ", mais plus vrai- semblablement " Comment retrouver un modèle 3D similaire à ceux en ma possession pour le réutiliser ? " et " Comment retrouver la structure d'un modèle 3D maillé sans connaissance a priori sur celui-ci ? " Cette habilitation a pour but d'apporter des éléments de réponse à ces deux questions. En réponse à la première question, nous avons développé un nouveau système bayésien pour retrouver des modèles 3D à partir d'une requête constituée d'une ou plusieurs vues 2D, ou d'un modèle 3D entier. Ce système a été testé dans un contexte applicatif industriel ainsi qu'avec un benchmark international. Chaque expérience a mis en évidence les excellents résultats de notre approche. La seconde question a été abordée sous l'angle de l'analyse topologique des maillages 3D grâce aux graphes de Reeb. Ce travail théorique a été appliqué à différents domaines comme la déformation automatique, l'indexation et la segmentation de maillages 3D. L'approche a toujours montré des résultats remarquables dans ces domaines. Finalement, la segmentation de maillages 3D, qui est une étape de pré-traitement fréquente avant d'autres analyses du maillage, a attiré notre attention. Nous avons proposé une métrique fiable et robuste pour la comparaison de segmentations et l'évaluation des performances des méthodes de segmentation de maillages 3D, ainsi qu'une approche de la segmentation par apprentissage qui surpasse les méthodes existantes. Pour terminer, de nouvelles pistes de recherche sur les maillages 3D sont ouvertes

Contributions à la recherche et à l'analyse de modèles 3D

The use of three-dimensional models in the multimedia applications, is growing both in number and in size. The development of mode- ling tools, 3D-scanners, graphic accelerated hardware, Web3D and so on, offers access to three-dimensional materials of high quality. The constantly increasing needs concerning these kinds of data, are rapidly changing. While it becomes more and more easy to create new 3D-models, what about process and analysis after the creation of the 3D-models? Today, the 3D designer no longer asks : "How to create a new 3D-model?", but more probably "How to retrieve an existing 3D-model that is similar to those I already own in order to reuse it?" and "How to get the inner structure of a 3D-mesh model without any a priori knowledge on it?" This habilitation thesis aims to provide some answers to these two questions. In response to the first question, we developed a new Bayesian framework to retrieve 3D-models from a query made of one or more 2D- views, or of an entire 3D-model. The framework has been tested in an industrial application context and with an international benchmark. Each of these experiments has shown excellent results. The second question has been addressed in terms of topological analysis of the 3D-meshes with the help of Reeb graphs. This theoretical work has been applied to several practical domains, such as automatic 3D-mesh deformation, 3D-model retrieval, and 3D-mesh segmentation, and has always highlighted outstanding results. Finally, the segmentation of 3D-meshes, which is a frequent pre-processing step before any other analysis of the mesh, has drawn our attention. We proposed a reliable and robust metric to compare segmentations and evaluate the performances of the 3D-mesh segmentation methods, as well as a new learning-based segmentation approach that out- performs existing ones. To conclude, new perspectives of research on 3D- meshes are open.L'utilisation de modèles tridimensionnels dans les applications multimédia, prend de l'ampleur de jour en jour. Le développement des outils de modélisation, des scanners 3D, des cartes graphiques accélérées, du Web3D, etc. ouvre l'accès à des données tridimensionnelles de grande qualité. Les besoins, sans cesse croissants, concernant ce type de données, changent rapidement. S'il devient de plus en plus facile de créer de nouveaux modèles 3D, qu'en est-il du traitement et de l'analyse de ces modèles après leur création ? De nos jours, le concepteur d'objets 3D ne pose plus la question : " Comment créer un nouvel objet 3D ? ", mais plus vrai- semblablement " Comment retrouver un modèle 3D similaire à ceux en ma possession pour le réutiliser ? " et " Comment retrouver la structure d'un modèle 3D maillé sans connaissance a priori sur celui-ci ? " Cette habilitation a pour but d'apporter des éléments de réponse à ces deux questions. En réponse à la première question, nous avons développé un nouveau système bayésien pour retrouver des modèles 3D à partir d'une requête constituée d'une ou plusieurs vues 2D, ou d'un modèle 3D entier. Ce système a été testé dans un contexte applicatif industriel ainsi qu'avec un benchmark international. Chaque expérience a mis en évidence les excellents résultats de notre approche. La seconde question a été abordée sous l'angle de l'analyse topologique des maillages 3D grâce aux graphes de Reeb. Ce travail théorique a été appliqué à différents domaines comme la déformation automatique, l'indexation et la segmentation de maillages 3D. L'approche a toujours montré des résultats remarquables dans ces domaines. Finalement, la segmentation de maillages 3D, qui est une étape de pré-traitement fréquente avant d'autres analyses du maillage, a attiré notre attention. Nous avons proposé une métrique fiable et robuste pour la comparaison de segmentations et l'évaluation des performances des méthodes de segmentation de maillages 3D, ainsi qu'une approche de la segmentation par apprentissage qui surpasse les méthodes existantes. Pour terminer, de nouvelles pistes de recherche sur les maillages 3D sont ouvertes

Fast and precise kinematic skeleton extraction of 3D dynamic meshes

International audienceShape skeleton extraction is a fundamental preprocessing task in shape-based pattern recognition. This paper presents a new algorithm for fast and precise extraction of kinematic skeletons of 3D dynamic surface meshes. Unlike previous approaches, surface motions are characterized by the mesh local length deviation induced by its transformation through time. Then a static skeleton extraction algorithm based on Reeb graphs exploits this latter information to extract the kinematic skeleton. This hybrid static and dynamic shape analysis enables the precise detection of objects' articulations as well as potentially-articulated immobile shape features. Experiments show that the proposed algorithm is faster than previous techniques and still achieves better accuracy

Reeb chart unfolding based 3D shape signatures

International audienceThis paper presents a novel surface parameterization based technique that addresses the pose insensitive shape signature problem for surface models of arbitrary genus. It is based on the key idea that two surface models are similar if the canonical mappings of their sub-parts introduce similar distortions. First, a Reeb graph of the shape is computed so as to segment it into charts of controlled topology, denoted as Reeb charts, that have either disk or annulus topology. Next, we define for each Reeb chart a straightforward mapping to the canonical planar domain. Then, we compute a stretching signature of the canonical mapping based on an area distortion evaluation. Finally, the input shape is represented by the set of the stretching signatures. An application to pose-insensitive shape similarity is proposed by comparing the signatures of the different Reeb charts. Promising experimental results are presented and compared to state-of-the-art techniques. The gain provided by this new signature as well as its interest for partial shape similarity are demonstrated