Compensating inaccurate annotations to train 3D facial landmark localisation models

In this paper we investigate the impact of inconsistency in manual annotations when they are used to train automatic models for 3D facial landmark localization. We start by showing that it is possible to objectively measure the consistency of annotations in a database, provided that it contains replicates (i.e. repeated scans from the same person). Applying such measure to the widely used FRGC database we find that manual annotations currently available are suboptimal and can strongly impair the accuracy of automatic models learnt therefrom. To address this issue, we present a simple algorithm to automatically correct a set of annotations and show that it can help to significantly improve the accuracy of the models in terms of landmark localization errors. This improvement is observed even when errors are measured with respect to the original (not corrected) annotations. However, we also show that if errors are computed against an alternative set of manual annotations with higher consistency, the accuracy of the models constructed using the corrections from the presented algorithm tends to converge to the one achieved by building the models on the alternative,more consistent set

Rotationally invariant 3D shape contexts using asymmetry patterns

This paper presents an approach to resolve the azimuth ambiguity of 3D Shape Contexts (3DSC) based on asymmetry patterns. We show that it is possible to provide rotational invariance to 3DSC at the expense of a marginal increase in computational load, outperforming previous algorithms dealing with the azimuth ambiguity. We build on a recently presented measure of approximate rotational symmetry in 2D defined as the overlapping area between a shape and rotated versions of itself to extract asymmetry patterns from a 3DSC in a variety of ways, depending on the spatial relationships that need to be highlighted or disabled. Thus, we define Asymmetry Patterns Shape Contexts (APSC) from a subset of the possible spatial relations present in the spherical grid of 3DSC; hence they can be thought of as a family of descriptors that depend on the subset that is selected. This provides great flexibility to derive different descriptors. We show that choosing the appropriate spatial patterns can considerably reduce the errors obtained with 3DSC when targeting specific types of points

Translational genetic modelling of 3D craniofacial dysmorphology: elaborating the facial phenotype of neurodevelopmental disorders through the prism of schizophrenia

Purpose of Review: In the context of human developmental conditions, we review the conceptualisation of schizophrenia as a neurodevelopmental disorder, the status of craniofacial dysmorphology as a clinically accessible index of brain dysmorphogenesis, the ability of genetically modified mouse models of craniofacial dysmorphology to inform on the underlying dysmorphogenic process and how geometric morphometric techniques in mutant mice can extend quantitative analysis. Recent Findings: Mutant mice with disruption of neuregulin-1, a gene associated meta-analytically with risk for schizophrenia, constitute proof-of-concept studies of murine facial dysmorphology in a manner analogous to clinical studies in schizophrenia. Geometric morphometric techniques informed on the topography of facial dysmorphology and identified asymmetry therein. Summary: Targeted disruption in mice of genes involved in individual components of developmental processes and analysis of resultant facial dysmorphology using geometric morphometrics can inform on mechanisms of dysmorphogenesis at levels of incisiveness not possible in human subjects

Comparing 3D descriptors for local search of craniofacial landmarks

This paper presents a comparison of local descriptors for a set of 26 craniofacial landmarks annotated on 144 scans acquired in the context of clinical research. We focus on the accuracy of the different descriptors on a per-landmark basis when constrained to a local search. For most descriptors, we find that the curves of expected error against the search radius have a plateau that can be used to characterize their performance, both in terms of accuracy and maximum usable range for the local search. Six histograms-based descriptors were evaluated: three describing distances and three describing orientations. No descriptor dominated over the rest and the best accuracy per landmark was strongly distributed among 3 of the 6 algorithms evaluated. Ordering the descriptors by average error (over all landmarks) did not coincide with the ordering by most frequently selected, indicating that a comparison of descriptors based on their global behavior might be misleading when targeting facial landmarks

Craniofacial landmark localisation with asymmetry patterns shape objects

We present a new family of 3D geometry descriptors based on the asymmetry patterns present in the popular 3D Shape Contexts (3DSC)[1]. Our approach resolves the azimuth ambiguity of 3DSC, thus providing rotational invariance, at the expense of a marginal increase in computational load, outperforming previous algorithms dealing with the azimuth ambiguity. We build on a recently presented measure of approximate rotational symmetry in 2D [2] defined as the overlapping area between a shape and rotated versions of itself, to extract asymmetry patterns from a 3DSC in a variety of ways, depending on the spatial relationships that need to be highlighted or disabled. Thus, we define Asymmetry Patterns Shape Contexts (APSC) [3] from a subset of the possible spatial relations present in the spherical grid of 3DSC; hence they can be thought of as a family of descriptors that depend on the subset that is selected. This provides great flexibility to derive different descriptors. We quantify the performance of the proposed descriptors for craniofacial landmark localization, targeting 22 points relevant in the context of dysmorphology research [4]. Measuring the performance in terms of distance to expert annotations we show that APSC can achieve overall accuracy comparable to 3DSC; the rotational invariance of APSC is achieved at the expense of a small computation overhead to build the descriptor (typically < 10%) but implies a speedup during matching by a factor of twice the number of azimuth bins (typically 24 : 1). Moreover, the possibility to define APSC descriptors by selecting diverse spatial patterns from a 3DSC has two important advantages: 1) choosing the appropriate spatial patterns can considerably reduce the errors obtained with 3DSC when targeting specific types of points; 2) Once one APSC descriptor is built, additional ones can be built with only incremental cost. Therefore, it is possible to use a pool of APSC descriptors to maximize accuracy without a large increase in computational cost

3D facial landmark localization using combinatorial search and shape regression

This paper presents a method for the automatic detection of facial landmarks. The algorithm receives a set of 3D candidate points for each landmark (e.g. from a feature detector) and performs combinatorial search constrained by a deformable shape model. A key assumption of our approach is that for some landmarks there might not be an accurate candidate in the input set. This is tackled by detecting partial subsets of landmarks and inferring those that are missing so that the probability of the deformable model is maximized. The ability of the model to work with incomplete information makes it possible to limit the number of candidates that need to be retained, substantially reducing the number of possible combinations to be tested with respect to the alternative of trying to always detect the complete set of landmarks. We demonstrate the accuracy of the proposed method in a set of 144 facial scans acquired by means of a hand-held laser scanner in the context of clinical craniofacial dysmorphology research. Using spin images to describe the geometry and targeting 11 facial landmarks, we obtain an average error below 3 mm, which compares favorably with other state of the art approaches based on geometric descriptors

A quantitative assessment of 3D facial key point localization fitting 2D shape models to curvature information

This work addresses the localization of 11 prominent facial landmarks in 3D by fitting state of the art shape models to 2D data. Quantitative results are provided for 34 scans at high resolution (texture maps of 10 M-pixels) in terms of accuracy (with respect to manual measurements) and precision (repeatability on different images from the same individual). We obtain an average accuracy of approximately 3 mm, and median repeatability of inter-landmark distances typically below 2 mm, which are values comparable to current algorithms on automatic localization of facial landmarks. We also show that, in our experiments, the replacement of texture information by curvature features produced little change in performance, which is an important finding as it suggests the applicability of the method to any type of 3D data

Three-dimensional laser surface imaging and geometric morphometrics resolve frontonasal dysmorphology in schizophrenia.

BACKGROUND: Although a role for early developmental disturbance(s) in schizophrenia is postulated, it has proved difficult to identify hard, biological evidence. The brain and face emerge in embryologic intimacy, such that in neurodevelopmental disorders, brain dysmorphogenesis is accompanied by facial dysmorphogenesis. METHODS: Three-dimensional (3D) laser surface imaging was used to capture the facial surface of patients and control subjects in 37 male and 32 female patients who satisfied DSM-IV criteria for schizophrenia in comparison with 58 male and 34 female control subjects. Surface images were analyzed using geometric morphometrics and 3D visualizations to identify domains of facial shape that distinguish patients from control subjects. RESULTS: Both male and, particularly, female patients evidenced significant facial dysmorphology. There was narrowing and reduction of the mid to lower face and frontonasal prominences, including reduced width and posterior displacement of the mouth, lips, and chin; increased width of the upper face, mandible, and skull base, with lateral displacement of the cheeks, eyes, and orbits; and anterior displacement of the superior margins of the orbits. CONCLUSIONS: The frontonasal prominence, which enjoys the most intimate embryologic relationship with the anterior brain and also orchestrates aspects of development in maxillary and mandibular domains, evidences a characteristic topography of dysmorphogenesis in schizophrenia

Semaphorin 6A knockout mice display abnormalities across ethologically-based topographies of exploration and in motor learning

Semaphorins are secreted or membrane-bound proteins implicated in neurodevelopmental processes of axon guidance and cell migration. Exploratory behaviour and motor learning was examined ethologically in Semaphorin 6A (Sema6A) mutant mice. The ethogram of initial exploration in Sema6A knockout mice was characterised by increased rearing to wall with decreased sifting; over subsequent habituation, locomotion, sniffing and rearing to wall were increased, with reduced habituation of rearing seated. Rotarod analysis indicated delayed motor learning in Sema6A heterozygous mutants. Disruption to the axonal guidance and cell migration processes regulated by Sema6A is associated with topographically specific disruption to fundamental aspects of behaviour, namely the ethogram of initial exploration and subsequent habituation to the environment, and motor learning

Potentiation of latent inhibition by haloperidol and clozapine is attenuated in Dopamine D2 receptor (Drd-2) deficient mice: Do antipsychotics influence learning to ignore irrelevant stimuli via both Drd-2 and non-Drd-2 mechanisms?

Whether the dopamine Drd-2 receptor is necessary for the behavioural action of antipsychotic drugs is an important question, as Drd-2 antagonism is responsible for their debilitating motor side effects. Using Drd-2 null mice (Drd2 -/-) it has previously been shown that Drd-2 is not necessary for antipsychotic drugs to reverse D-amphetamine disruption of latent inhibition (LI), a behavioural measure of learning to ignore irrelevant stimuli. Weiner's 'two-headed' model indicates that antipsychotics not only reverse LI disruption, 'disrupted LI', but also potentiate LI when low/absent in controls, 'persistent' LI. We investigated whether antipsychotic drugs haloperidol or clozapine potentiated LI in wild-type controls or Drd2 -/-. Both drugs potentiated LI in wild-type but not in Drd2 -/- mice, suggesting moderation of this effect of antipsychotics in the absence of Drd-2. Haloperidol potentiated LI similarly in both Drd1 -/- and wild-type mice, indicating no such moderation in Drd1 -/-. These data suggest that antipsychotic drugs can have either Drd-2 or non-Drd-2 effects on learning to ignore irrelevant stimuli, depending on how the abnormality is produced. Identification of the non-Drd-2 mechanism may help to identify novel non-Drd2 based therapeutic strategies for psychosis