Automatic Landmarking for Non-cooperative 3D Face Recognition

This thesis describes a new framework for 3D surface landmarking and evaluates its performance for feature localisation on human faces. This framework has two main parts that can be designed and optimised independently. The first one is a keypoint detection system that returns positions of interest for a given mesh surface by using a learnt dictionary of local shapes. The second one is a labelling system, using model fitting approaches that establish a one-to-one correspondence between the set of unlabelled input points and a learnt representation of the class of object to detect. Our keypoint detection system returns local maxima over score maps that are generated from an arbitrarily large set of local shape descriptors. The distributions of these descriptors (scalars or histograms) are learnt for known landmark positions on a training dataset in order to generate a model. The similarity between the input descriptor value for a given vertex and a model shape is used as a descriptor-related score. Our labelling system can make use of both hypergraph matching techniques and rigid registration techniques to reduce the ambiguity attached to unlabelled input keypoints for which a list of model landmark candidates have been seeded. The soft matching techniques use multi-attributed hyperedges to reduce ambiguity, while the registration techniques use scale-adapted rigid transformation computed from 3 or more points in order to obtain one-to-one correspondences. Our final system achieves better or comparable (depending on the metric) results than the state-of-the-art while being more generic. It does not require pre-processing such as cropping, spike removal and hole filling and is more robust to occlusion of salient local regions, such as those near the nose tip and inner eye corners. It is also fully pose invariant and can be used with kinds of objects other than faces, provided that labelled training data is available

Unfamiliar facial identity registration and recognition performance enhancement

The work in this thesis aims at studying the problems related to the robustness of a face recognition system where specific attention is given to the issues of handling the image variation complexity and inherent limited Unique Characteristic Information (UCI) within the scope of unfamiliar identity recognition environment. These issues will be the main themes in developing a mutual understanding of extraction and classification tasking strategies and are carried out as a two interdependent but related blocks of research work. Naturally, the complexity of the image variation problem is built up from factors including the viewing geometry, illumination, occlusion and other kind of intrinsic and extrinsic image variation. Ideally, the recognition performance will be increased whenever the variation is reduced and/or the UCI is increased. However, the variation reduction on 2D facial images may result in loss of important clues or UCI data for a particular face alternatively increasing the UCI may also increase the image variation. To reduce the lost of information, while reducing or compensating the variation complexity, a hybrid technique is proposed in this thesis. The technique is derived from three conventional approaches for the variation compensation and feature extraction tasks. In this first research block, transformation, modelling and compensation approaches are combined to deal with the variation complexity. The ultimate aim of this combination is to represent (transformation) the UCI without losing the important features by modelling and discard (compensation) and reduce the level of the variation complexity of a given face image. Experimental results have shown that discarding a certain obvious variation will enhance the desired information rather than sceptical in losing the interested UCI. The modelling and compensation stages will benefit both variation reduction and UCI enhancement. Colour, gray level and edge image information are used to manipulate the UCI which involve the analysis on the skin colour, facial texture and features measurement respectively. The Derivative Linear Binary transformation (DLBT) technique is proposed for the features measurement consistency. Prior knowledge of input image with symmetrical properties, the informative region and consistency of some features will be fully utilized in preserving the UCI feature information. As a result, the similarity and dissimilarity representation for identity parameters or classes are obtained from the selected UCI representation which involves the derivative features size and distance measurement, facial texture and skin colour. These are mainly used to accommodate the strategy of unfamiliar identity classification in the second block of the research work. Since all faces share similar structure, classification technique should be able to increase the similarities within the class while increase the dissimilarity between the classes. Furthermore, a smaller class will result on less burden on the identification or recognition processes. The proposed method or collateral classification strategy of identity representation introduced in this thesis is by manipulating the availability of the collateral UCI for classifying the identity parameters of regional appearance, gender and age classes. In this regard, the registration of collateral UCI s have been made in such a way to collect more identity information. As a result, the performance of unfamiliar identity recognition positively is upgraded with respect to the special UCI for the class recognition and possibly with the small size of the class. The experiment was done using data from our developed database and open database comprising three different regional appearances, two different age groups and two different genders and is incorporated with pose and illumination image variations

The Role of Physical Image Properties in Facial Expression and Identity Perception

A number of attempts have been made to understand which physical image properties are important for the perception of different facial characteristics. These physical image properties have been broadly split in to two categories; namely facial shape and facial surface. Current accounts of face processing suggest that whilst judgements of facial identity rely approximately equally on facial shape and surface properties, judgements of facial expression are heavily shape dependent. This thesis presents behavioural experiments and fMRI experiments employing multi voxel pattern analysis (MVPA) to investigate the extent to which facial shape and surface properties underpin identity and expression perception and how these image properties are represented neurally. The first empirical chapter presents experiments showing that facial expressions are categorised approximately equally well when either facial shape or surface is the varying image cue. The second empirical chapter shows that neural patterns of response to facial expressions in the Occipital Face Area (OFA) and Superior Temporal Sulcus (STS) are reflected by patterns of perceptual similarity of the different expressions, in turn these patterns of perceptual similarity can be predicted by both facial shape and surface properties. The third empirical chapter demonstrates that distinct patterns of neural response can be found to shape based but not surface based cues to facial identity in the OFA and Fusiform Face Area (FFA). The final experimental chapter in this thesis demonstrates that the newly discovered contrast chimera effect is heavily dependent on the eye region and holistic face representations conveying facial identity. Taken together, these findings show the importance of facial surface as well as facial shape in expression perception. For facial identity both facial shape and surface cues are important for the contrast chimera effect although there are more consistent identity based neural response patterns to facial shape in face responsive brain regions

Electrophysiological Correlates of Processing Unattended Objects in Visual Cognition

Research is divided as to what degree visually unattended objects are processed (Lachter et al., 2008; Carrasco, 2011). The hybrid model of object recognition (Hummel, 2001) predicts that familiar objects are automatically recognised without attention. However under perceptual load theory (Lavie, 1995), when objects are rendered unattended due to exhausted attentional resources, they are not processed. The present work examined the visual processing of images of everyday objects in a short-lag repetition-priming paradigm. In Experiments 1-3 attention was cued to the location of one of two objects in the first (prime) display, with the unattended sometimes repeated in the second (probe) display. ERP repetition effects were observed which were insensitive to changes in scale (Experiment 1) but sensitive to slight scrambling of the image (Experiment 2). Increasing perceptual load did not modulate these view-specific repetition effects (Experiment 3), consistent with the predictions of automatic holistic processing. In Experiments 4-7 a letter search task was used to render the flanking object image unattended under high load. In Experiment 5 distractor processing was observed in ERP even under high load. In Experiments 4, 6 and 7 a pattern of view sensitive/insensitive and load sensitive/insensitive repetition effects on RT (Experiment 4) and ERP amplitude (Experiments 6, 7) were observed that were difficult to interpret under either the hybrid model or perceptual load theory, but may reflect fast view-based and slow view-independent processing of objects. Overall, the properties of the view-sensitive repetition effects were generally consistent with those associated with the automatic/pre-attentive processing of the holistic route of the hybrid model. However, differences between the processing of objects rendered unattended via a spatial cue or perceptual load indicate that the bottom-up driven hybrid model and perceptual load theory may benefit from the consideration of the interaction of top-down biasing of processing (Tsotsos et al., 2008)

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Shape classification: towards a mathematical description of the face

Recent advances in biostereometric techniques have led to the quick and easy acquisition of 3D data for facial and other biological surfaces. This has led facial surgeons to express dissatisfaction with landmark-based methods for analysing the shape of the face which use only a small part of the data available, and to seek a method for analysing the face which maximizes the use of this extensive data set. Scientists working in the field of computer vision have developed a variety of methods for the analysis and description of 2D and 3D shape. These methods are reviewed and an approach, based on differential geometry, is selected for the description of facial shape. For each data point, the Gaussian and mean curvatures of the surface are calculated. The performance of three algorithms for computing these curvatures are evaluated for mathematically generated standard 3D objects and for 3D data obtained from an optical surface scanner. Using the signs of these curvatures, the face is classified into eight 'fundamental surface types' - each of which has an intuitive perceptual meaning. The robustness of the resulting surface type description to errors in the data is determined together with its repeatability. Three methods for comparing two surface type descriptions are presented and illustrated for average male and average female faces. Thus a quantitative description of facial change, or differences between individual's faces, is achieved. The possible application of artificial intelligence techniques to automate this comparison is discussed. The sensitivity of the description to global and local changes to the data, made by mathematical functions, is investigated. Examples are given of the application of this method for describing facial changes made by facial reconstructive surgery and implications for defining a basis for facial aesthetics using shape are discussed. It is also applied to investigate the role played by the shape of the surface in facial recognition

Object Recognition

Vision-based object recognition tasks are very familiar in our everyday activities, such as driving our car in the correct lane. We do these tasks effortlessly in real-time. In the last decades, with the advancement of computer technology, researchers and application developers are trying to mimic the human's capability of visually recognising. Such capability will allow machine to free human from boring or dangerous jobs

Stability from variety: the prototype effect in face recognition

The central goal of the current thesis was to increase our understanding of how representations of individual faces are built from instances that vary. The prototype effect was used as a tool to probe the nature of our internal face representations. In face recognition, the prototype effect refers to the tendency to recognize, or find familiar, the average image of a face after having studied a series of similar face images. The experiments presented in this thesis investigated the modulating role of different variables on the prototype effect in face recognition. In the study phase, two or more different exemplars based on the same identity were presented. In the test phase, one of the seen exemplars, the unseen prototype, and an unseen exemplar of each studied identity were presented one at a time, and participants were asked to make a recognition judgement about the prior occurrence of either the exact image or the person’s face. Variants of each face identity were either unaltered images of real people’s faces, or they were created artificially by manipulating images of faces using several different techniques. All experiments using artificial variants produced strong prototype effects. The unseen prototype image was recognized more confidently than the actually studied images. This was true even when the variants were so similar that they were barely perceptually discriminable. Importantly, even when participants were given additional exposure to the studied exemplars, no weakening of the prototype effect was observed. Surprisingly, in the experiments using natural images of real people’s faces, no clear recognition advantage for the prototype image was observed. Results suggest that the prototype effect in face recognition might not be tapping an averaging mechanism that operates solely on variations within the same identity