13 research outputs found
Multilinear Wavelets: A Statistical Shape Space for Human Faces
We present a statistical model for D human faces in varying expression,
which decomposes the surface of the face using a wavelet transform, and learns
many localized, decorrelated multilinear models on the resulting coefficients.
Using this model we are able to reconstruct faces from noisy and occluded D
face scans, and facial motion sequences. Accurate reconstruction of face shape
is important for applications such as tele-presence and gaming. The localized
and multi-scale nature of our model allows for recovery of fine-scale detail
while retaining robustness to severe noise and occlusion, and is
computationally efficient and scalable. We validate these properties
experimentally on challenging data in the form of static scans and motion
sequences. We show that in comparison to a global multilinear model, our model
better preserves fine detail and is computationally faster, while in comparison
to a localized PCA model, our model better handles variation in expression, is
faster, and allows us to fix identity parameters for a given subject.Comment: 10 pages, 7 figures; accepted to ECCV 201
Learning from Millions of 3D Scans for Large-scale 3D Face Recognition
Deep networks trained on millions of facial images are believed to be closely
approaching human-level performance in face recognition. However, open world
face recognition still remains a challenge. Although, 3D face recognition has
an inherent edge over its 2D counterpart, it has not benefited from the recent
developments in deep learning due to the unavailability of large training as
well as large test datasets. Recognition accuracies have already saturated on
existing 3D face datasets due to their small gallery sizes. Unlike 2D
photographs, 3D facial scans cannot be sourced from the web causing a
bottleneck in the development of deep 3D face recognition networks and
datasets. In this backdrop, we propose a method for generating a large corpus
of labeled 3D face identities and their multiple instances for training and a
protocol for merging the most challenging existing 3D datasets for testing. We
also propose the first deep CNN model designed specifically for 3D face
recognition and trained on 3.1 Million 3D facial scans of 100K identities. Our
test dataset comprises 1,853 identities with a single 3D scan in the gallery
and another 31K scans as probes, which is several orders of magnitude larger
than existing ones. Without fine tuning on this dataset, our network already
outperforms state of the art face recognition by over 10%. We fine tune our
network on the gallery set to perform end-to-end large scale 3D face
recognition which further improves accuracy. Finally, we show the efficacy of
our method for the open world face recognition problem.Comment: 11 page
Dense 3D Face Correspondence
We present an algorithm that automatically establishes dense correspondences
between a large number of 3D faces. Starting from automatically detected sparse
correspondences on the outer boundary of 3D faces, the algorithm triangulates
existing correspondences and expands them iteratively by matching points of
distinctive surface curvature along the triangle edges. After exhausting
keypoint matches, further correspondences are established by generating evenly
distributed points within triangles by evolving level set geodesic curves from
the centroids of large triangles. A deformable model (K3DM) is constructed from
the dense corresponded faces and an algorithm is proposed for morphing the K3DM
to fit unseen faces. This algorithm iterates between rigid alignment of an
unseen face followed by regularized morphing of the deformable model. We have
extensively evaluated the proposed algorithms on synthetic data and real 3D
faces from the FRGCv2, Bosphorus, BU3DFE and UND Ear databases using
quantitative and qualitative benchmarks. Our algorithm achieved dense
correspondences with a mean localisation error of 1.28mm on synthetic faces and
detected anthropometric landmarks on unseen real faces from the FRGCv2
database with 3mm precision. Furthermore, our deformable model fitting
algorithm achieved 98.5% face recognition accuracy on the FRGCv2 and 98.6% on
Bosphorus database. Our dense model is also able to generalize to unseen
datasets.Comment: 24 Pages, 12 Figures, 6 Tables and 3 Algorithm
On Intelligent Surveillance Systems and Face Recognition for Mass Transport Security
We describe a project to trial and develop enhanced surveillance technologies for public safety. A key technology is robust recognition of faces from low-resolution CCTV footage where there may be as few as 12 pixels between the eyes. Current commercial face recognition systems require 60-90 pixels between the eyes as well as tightly controlled image capture conditions. Our group has thus concentrated on fundamental face recognition issues such as robustness to low resolution and image capture conditions as required for uncontrolled CCTV surveillance. In this paper, we propose a fast multi-class pattern classification approach to enhance PCA and FLD methods for 2D face recognition under changes in pose, illumination, and expression. The method first finds the optimal weights of features pairwise and constructs a feature chain in order to determine the weights for all features. Computational load of the proposed approach is extremely low by design, in order to facilitate usage in automated surveillance. The method is evaluated on PIE, FERET, and Asian Face databases, with the results showing that the method performs remarkably well compared to several benchmark appearance-based methods. Moreover, the method can reliably recognise faces with large pose angles from just one gallery image
Advanced image processing techniques for detection and quantification of drusen
Dissertation presented to obtain the degree of Doctor of Philosophy in Electrical Engineering, speciality on Perceptional Systems, by the Universidade Nova de Lisboa,
Faculty of Sciences and TechnologyDrusen are common features in the ageing macula, caused by accumulation of extracellular materials beneath the retinal surface, visible in retinal fundus images as yellow spots.
In the ophthalmologistsâ opinion, the evaluation of the total drusen area, in a sequence of images taken during a treatment, will help to understand the disease progression and effectiveness. However, this evaluation is fastidious and difficult to reproduce when performed manually.
A literature review on automated drusen detection showed that the works already
published were limited to techniques of either adaptive or global thresholds which showed a tendency to produce a significant number of false positives. The purpose for this work was to propose an alternative method to automatically quantify drusen using advanced digital image processing techniques.
This methodology is based on a detection and modelling algorithm to automatically
quantify drusen. It includes an image pre-processing step to correct the uneven illumination by using smoothing splines fitting and to normalize the contrast. To quantify drusen a detection and modelling algorithm is adopted. The detection uses a new gradient based segmentation algorithm that isolates drusen and provides basic drusen characterization to the modelling stage. These are then fitted by Gaussian functions, to produce a model of the
image, which is used to compute the affected areas.
To validate the methodology, two software applications, one for semi-automated
(MD3RI) and other for automated detection of drusen (AD3RI), were implemented. The first
was developed for Ophthalmologists to manually analyse and mark drusen deposits, while the other implemented algorithms for automatic drusen quantification.Four studies to assess the methodology accuracy involving twelve specialists have
taken place. These compared the automated method to the specialists and evaluated its
repeatability. The studies were analysed regarding several indicators, which were based on the
total affected area and on a pixel-to-pixel analysis. Due to the high variability among the
graders involved in the first study, a new evaluation method, the Weighed Matching Analysis,
was developed to improve the pixel-to-pixel analysis by using the statistical significance of
the observations to differentiate positive and negative pixels.
From the results of these studies it was concluded that the methodology proposed is
capable to automatically measure drusen in an accurate and reproducible process. Also, the thesis proposes new image processing algorithms, for image pre-processing, image
segmentation,image modelling and images comparison, which are also applicable to other image processing fields
Fitting a Morphable Model to {3D} Scans of Faces
This paper presents a top-down approach to 3D data analysis by fitting a Morphable Model to scans of faces. In a unified framework, the algorithm optimizes shape, texture, pose and illumination simultaneously. The algorithm can be used as a core component in face recognition from scans. In an analysis-by-synthesis approach, raw scans are transformed into a PCA-based representation that is robust with respect to changes in pose and illumination. Illumination conditions are estimated in an explicit simulation that involves specular and diffuse components. The algorithm inverts the effect of shading in order to obtain the diffuse reflectance in each point of the facial surface. Our results include illumination correction, surface completion and face recognition on the FRGC database of scans
Data driven analysis of faces from images
This thesis proposes three new data-driven approaches to detect, analyze, or modify faces in images. All presented contributions are inspired by the use of prior knowledge and they derive information about facial appearances from pre-collected databases of images or 3D face models. First, we contribute an approach that extends a widely-used monocular face detector by an additional classifier that evaluates disparity maps of a passive stereo camera. The algorithm runs in real-time and significantly reduces the number of false positives compared to the monocular approach.
Next, with a many-core implementation of the detector, we train view-dependent face detectors based on tailored views which guarantee that the statistical variability is fully covered. These detectors are superior to the state of the art on a challenging dataset and can be trained in an automated procedure. Finally, we contribute a model describing the relation of facial appearance and makeup. The approach extracts makeup from before/after images of faces and allows to modify faces in images. Applications such as machine-suggested makeup can improve perceived attractiveness as shown in a perceptual study.
In summary, the presented methods help improve the outcome of face detection algorithms, ease and automate their training procedures and the modification of faces in images. Moreover, their data-driven nature enables new and powerful applications arising from the use of prior knowledge and statistical analyses.In der vorliegenden Arbeit werden drei neue, datengetriebene Methoden vorgestellt, die Gesichter in Abbildungen detektieren, analysieren oder modifizieren. Alle Algorithmen extrahieren dabei Vorwissen ĂŒber Gesichter und deren Erscheinungsformen aus zuvor erstellten Gesichts- Datenbanken, in 2-D oder 3-D.
ZunĂ€chst wird ein weit verbreiteter monokularer Gesichtsdetektions- Algorithmus um einen zweiten Klassifikator erweitert. In Echtzeit wertet dieser stereoskopische Tiefenkarten aus und fĂŒhrt so zu nachweislich weniger falsch detektierten Gesichtern. AnschlieĂend wird der Basis-Algorithmus durch Parallelisierung verbessert und mit synthetisch generierten Bilddaten trainiert. Diese garantieren die volle Nutzung des verfĂŒgbaren Varianzspektrums. So erzeugte Detektoren ĂŒbertreffen bisher prĂ€sentierte Detektoren auf einem schwierigen Datensatz und können automatisch erzeugt werden. AbschlieĂend wird ein Datenmodell fĂŒr Gesichts-Make-up vorgestellt. Dieses extrahiert Make-up aus Vorher/Nachher-Fotos und kann Gesichter in Abbildungen modifizieren. In einer Studie wird gezeigt, dass vom Computer empfohlenes Make-up die wahrgenommene AttraktivitĂ€t von Gesichtern steigert.
Zusammengefasst verbessern die gezeigten Methoden die Ergebnisse von Gesichtsdetektoren, erleichtern und automatisieren ihre Trainingsprozedur sowie die automatische VerÀnderung von Gesichtern in Abbildungen. Durch Extraktion von Vorwissen und statistische Datenanalyse entstehen zudem neuartige Anwendungsfelder
Uses of uncalibrated images to enrich 3D models information
The decrease in costs of semi-professional digital cameras has led to the possibility
for everyone to acquire a very detailed description of a scene in a very short time.
Unfortunately, the interpretation of the images is usually quite hard, due to the amount
of data and the lack of robust and generic image analysis methods. Nevertheless, if a
geometric description of the depicted scene is available, it gets much easier to extract
information from 2D data.
This information can be used to enrich the quality of the 3D data in several ways.
In this thesis, several uses of sets of unregistered images for the enrichment of 3D
models are shown.
In particular, two possible fields of application are presented: the color acquisition,
projection and visualization and the geometry modification.
Regarding color management, several practical and cheap solutions to overcome the
main issues in this field are presented. Moreover, some real applications, mainly related
to Cultural Heritage, show that provided methods are robust and effective.
In the context of geometry modification, two approaches are presented to modify already
existing 3D models. In the first one, information extracted from images is used
to deform a dummy model to obtain accurate 3D head models, used for simulation
in the context of three-dimensional audio rendering. The second approach presents
a method to fill holes in 3D models, with the use of registered images depicting a
pattern projected on the real object.
Finally, some useful indications about the possible future work in all the presented
fields are given, in order to delineate the developments of this promising direction of
research