Numérisation 3D de visages par une approche de super-résolution spatio-temporelle non-rigide

La mesure de la forme 3D du visage est une problématique qui attire de plus en plus de chercheurs et qui trouve son application dans des domaines divers tels que la biométrie, l animation et la chirurgie faciale. Les solutions actuelles sont souvent basées sur des systèmes projecteur/caméra et utilisent de la lumière structurée pour compenser l insuffisance de la texture faciale. L information 3D est ensuite calculée en décodant la distorsion des patrons projetés sur le visage. Une des techniques les plus utilisées de la lumière structurée est la codification sinusoïdale par décalage de phase qui permet une numérisation 3D de résolution pixélique. Cette technique exige une étape de déroulement de phase, sensible à l éclairage ambiant surtout quand le nombre de patrons projetés est limité. En plus, la projection de plusieurs patrons impacte le délai de numérisation et peut générer des artefacts surtout pour la capture d un visage en mouvement. Une alternative aux approches projecteur-caméra consiste à estimer l information 3D par appariement stéréo suivi par une triangulation optique. Cependant, le modèle calculé par cette technique est généralement non-dense et manque de précision. Des travaux récents proposent la super-résolution pour densifier et débruiter les images de profondeur. La super-résolution a été particulièrement proposée pour les caméras 3D TOF (Time-Of-Flight) qui fournissent des scans 3D très bruités. Ce travail de thèse propose une solution de numérisation 3D à faible coût avec un schéma de super-résolution spatio-temporelle. Elle utilise un système multi-caméra étalonné assisté par une source de projection non-étalonnée. Elle est particulièrement adaptée à la reconstruction 3D de visages, i.e. rapide et mobile. La solution proposée est une approche hybride qui associe la stéréovision et la codification sinusoïdale par décalage de phase, et qui non seulement profite de leurs avantages mais qui surmonte leurs faiblesses. Le schéma de la super-résolution proposé permet de corriger l information 3D, de compléter la vue scannée du visage en traitant son aspect déformable.3D face measurement is increasingly demanded for many applications such as bio-metrics, animation and facial surgery. Current solutions often employ a structured light camera/projector device to overcome the relatively uniform appearance of skin. Depth in-formation is recovered by decoding patterns of the projected structured light. One of the most widely used structured-light coding is sinusoidal phase shifting which allows a 3Ddense resolution. Current solutions mostly utilize more than three phase-shifted sinusoidal patterns to recover the depth information, thus impacting the acquisition delay. They further require projector-camera calibration whose accuracy is crucial for phase to depth estimation step. Also, they need an unwrapping stage which is sensitive to ambient light, especially when the number of patterns decreases. An alternative to projector-camera systems consists of recovering depth information by stereovision using a multi-camera system. A stereo matching step finds correspondence between stereo images and the 3D information is obtained by optical triangulation. However, the model computed in this way generally is quite sparse. To up sample and denoise depth images, researchers looked into super-resolution techniques. Super-resolution was especially proposed for time-of-flight cameras which have very low data quality and a very high random noise. This thesis proposes a3D acquisition solution with a 3D space-time non-rigid super-resolution capability, using a calibrated multi-camera system coupled with a non calibrated projector device, which is particularly suited to 3D face scanning, i.e. rapid and easily movable. The proposed solution is a hybrid stereovision and phase-shifting approach, using two shifted patterns and a texture image, which not only takes advantage of the assets of stereovision and structured light but also overcomes their weaknesses. The super-resolution scheme involves a 3D non-rigid registration for 3D artifacts correction in the presence of small non-rigid deformations as facial expressions.LYON-Ecole Centrale (690812301) / SudocSudocFranceF

Fusion d'Experts pour une Biométrie Faciale 3D Robuste aux Déformations

Session "Posters"National audienceNous étudions dans cet article l'apport de la géométrie tridimensionnelle du visage dans la reconnaissance des individus. La principale contribution est d'associer plusieurs experts (matcheurs) de biométrie faciale 3D afin d'achever de meilleures performances comparées aux performances individuelles de chacun, notamment en présence d'expressions. Les experts utilisés sont : (E1) Courbes radiales élastiques, (E2) MS-eLBP, une version étendue multi-échelle de l'opérateur LBP, (E3) l'algorithme de recalage non-rigide TPS, en plus d'un expert de référence (Eref) l'algorithme de recalage rigide connu ICP. Profitant de la complémentarité de chacun des experts, la présente approche affiche un taux d'identification qui dépasse les 99% en présence d'expressions faciales sur la base FRGCv2. Une étude comparative avec l'état de l'art confirme le choix et l'intérêt de combiner plusieurs experts afin d'achever de meilleurs performance

Multi-size Pooling for Stereo Matching Cost

International audienc

Learning Confidence Measures by Multi-modal Convolutional Neural Networks

International audienc

Nose tip localization on 2.5D facial models using differential geometry based point signatures and SVM classifier

International audienceNose tip localization on 2.5D facial models using differential geometry based point signatures and SVM classifie

A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking

International audienceAutomatic 2.5D face landmarking aims at locatingfacial feature points on 2.5D face models, such as eye corners,nose tip, etc. and has many applications ranging from faceregistration to facial expression recognition. In this paper, wepropose a rotation invariant 2.5D face landmarking solutionbased on facial curvature analysis combined with a generic2.5D face model and make use of a coarse-to-fine strategy formore accurate facial feature points localization. Experimentedon more than 1600 face models randomly selected from theFRGC dataset, our technique displays, compared to a groundtruth from a manual 3D face landmarking, a 100% of good nosetip localization in 8 mm precision and 100% of good localizationfor the eye inner corner in 12 mm precision

A Novel Trilateral Filter based Adaptive Support Weight Method for Stereo Matching

International audienceAdaptive support weight (ASW) approach represents the state-of-the-art local stereo matching method. Recent extensive evaluation studies on ASW approaches show that the bilateral filter weight function enables outstanding performance on a large dataset in comparison with various weight functions. However, it does not resolve the ambiguity induced by nearby pixels at different disparities but with similar colors. In this paper, we propose a novel trilateral filter based ASW method which remedies such ambiguities by considering disparity discontinuities through color discontinuity boundaries, i.e., the strength of the boundary between two pixels. The experimental evaluation on the Middlebury benchmark shows that the proposed algorithm ranks 15th out of 150 submissions and is the current most accurate local stereo matching algorithm

A Fast Trilateral Filter based Adaptive Support Weight Method for Stereo Matching

International audienceAdaptive support weight (ASW) methods represent the state-of-the-art in local stereo matching and the bilateral filter based ASW method achieves outstanding performance. However, this method fails to resolve the ambiguity induced by nearby pixels at different disparities but with similar colors. In this paper, we introduce a novel trilateral filter based ASW method that remedies such ambiguities by considering the possible disparity discontinuities through color discontinuity boundaries, i.e., the boundary strength between two pixels, which is measured by a local energy model. We also present a recursive trilateral filter based ASW method whose computational complexity is O(N) excluding the boundary detection, where N denotes the input image size. This complexity is thus independent of the support window size. The recursive trilateral filter based method is a non-local cost aggregation strategy. The experimental evaluation on the Middlebury benchmark shows that the proposed method, whose average error rate is 4.95%, outperforms other local methods in terms of accuracy and also the average runtime of the proposed trilateral filter based cost aggregation is roughly 260 milliseconds on a 3.4 GHz Inter Core i7 CPU, which is comparable to the state-of-the-art efficiency

Fast Determination of Melanin based on Skin Hyperspectral Reflectance

International audienc

A Novel Trilateral Filter based Adaptive Support Weight Method for Stereo Matching

International audienceAdaptive support weight (ASW) approach represents the state-of-the-art local stereo matching method. Recent extensive evaluation studies on ASW approaches show that the bilateral filter weight function enables outstanding performance on a large dataset in comparison with various weight functions. However, it does not resolve the ambiguity induced by nearby pixels at different disparities but with similar colors. In this paper, we propose a novel trilateral filter based ASW method which remedies such ambiguities by considering disparity discontinuities through color discontinuity boundaries, i.e., the strength of the boundary between two pixels. The experimental evaluation on the Middlebury benchmark shows that the proposed algorithm ranks 15th out of 150 submissions and is the current most accurate local stereo matching algorithm