Object Detection and Classification in the Visible and Infrared Spectrums

The over-arching theme of this dissertation is the development of automated detection and/or classification systems for challenging infrared scenarios. The six works presented herein can be categorized into four problem scenarios. In the first scenario, long-distance detection and classification of vehicles in thermal imagery, a custom convolutional network architecture is proposed for small thermal target detection. For the second scenario, thermal face landmark detection and thermal cross-spectral face verification, a publicly-available visible and thermal face dataset is introduced, along with benchmark results for several landmark detection and face verification algorithms. Furthermore, a novel visible-to-thermal transfer learning algorithm for face landmark detection is presented. The third scenario addresses near-infrared cross-spectral periocular recognition with a coupled conditional generative adversarial network guided by auxiliary synthetic loss functions. Finally, a deep sparse feature selection and fusion is proposed to detect the presence of textured contact lenses prior to near-infrared iris recognition

A Study on an Automatic System for Analyzing the Facial Beauty of Young Women

A Study on an Automatic System for Analyzing the Facial Beauty of Young Women Neha Sultan Beauty is one of the foremost ideas that define human personality. However, only recently has the concept of beauty been scientifically analyzed. This has mostly been due to extensive research done in the area of face recognition and image processing on identification and classification of human features as contributing to facial beauty. Current research aims at precisely and conclusively understanding how humans classify a given individual's face as beautiful. Due to the lack of published theoretical standards and ground truths for human facial beauty, this is often an ambiguous process. Current methods of analysis and classification of human facial beauty rely mainly on the geometric aspects of human facial beauty. The classifiers used in current research include the k-nearest neighbor algorithm, ridge regression, and basic principal component analysis. In this research, various approaches related to the comprehension and analysis of human beauty are presented and the use of these theories is outlined. Each set of theories is translated into a feature model that is tested for classification. Selecting the best set of features that result in the most accurate model for the representation of the human face is a key challenge. This research introduces the combined use of three main groups of features for classification of female facial beauty, to be used with classification through support vector machines. The classifier utilized is Support Vector Machine (SVM) and the accuracy obtained through this classifier is 86%. Current research in the field has produced algorithms with percentages of accuracy that are in the range of 75-85%. The approach used is one of analysis of the central tenets of beauty, the successive application of image processing techniques, and finally the usage of relevant machine learning methods to build an effective system for the automatic assessment of facial beauty. The ground truths used for verifying results are derived from ratings extracted from surveys conducted. The proposed methodology involves a novel algorithm for the representation of facial beauty, which combines the use of geometric, textural, and shape based features for the analysis of facial beauty. This algorithm initially develops an overall landmark model of the entire human face. A significant advantage of this methodology is the accurate model of the human face which synthesizes the geometric, textural and shape-related aspects of the face. The landmark model is then used for extracting critical characteristics which are then used in a feature vector for training using machine learning. The features extracted help to represent facial characteristics in three major areas. Geometric features help to represent the symmetrical properties and ratio-based properties of landmarks on the face. Textural features extracted help capture information related to skin texture and composition. Finally, face shape and outline features help to categorize the overall shape of a given face, which helps to represent the given female face shape and outline for further analysis of any deviations from the basic face shapes. These features are then used in a classifier to appropriately categorize each image. The database used for the source of images contains images of female subjects from a variety of backgrounds and levels of attractiveness

Discriminative Appearance Models for Face Alignment

The proposed face alignment algorithm uses local gradient features as the appearance representation. These features are obtained by pixel value comparison, which provide robustness against changes in illumination, as well as partial occlusion and local deformation due to the locality. The adopted features are modeled in three discriminative methods, which correspond to different alignment cost functions. The discriminative appearance modeling alleviate the generalization problem to some extent

Adaptive threshold optimisation for colour-based lip segmentation in automatic lip-reading systems

A thesis submitted to the Faculty of Engineering and the Built Environment, University of the Witwatersrand, Johannesburg, in ful lment of the requirements for the degree of Doctor of Philosophy. Johannesburg, September 2016Having survived the ordeal of a laryngectomy, the patient must come to terms with the resulting loss of speech. With recent advances in portable computing power, automatic lip-reading (ALR) may become a viable approach to voice restoration. This thesis addresses the image processing aspect of ALR, and focuses three contributions to colour-based lip segmentation. The rst contribution concerns the colour transform to enhance the contrast between the lips and skin. This thesis presents the most comprehensive study to date by measuring the overlap between lip and skin histograms for 33 di erent colour transforms. The hue component of HSV obtains the lowest overlap of 6:15%, and results show that selecting the correct transform can increase the segmentation accuracy by up to three times. The second contribution is the development of a new lip segmentation algorithm that utilises the best colour transforms from the comparative study. The algorithm is tested on 895 images and achieves percentage overlap (OL) of 92:23% and segmentation error (SE) of 7:39 %. The third contribution focuses on the impact of the histogram threshold on the segmentation accuracy, and introduces a novel technique called Adaptive Threshold Optimisation (ATO) to select a better threshold value. The rst stage of ATO incorporates -SVR to train the lip shape model. ATO then uses feedback of shape information to validate and optimise the threshold. After applying ATO, the SE decreases from 7:65% to 6:50%, corresponding to an absolute improvement of 1:15 pp or relative improvement of 15:1%. While this thesis concerns lip segmentation in particular, ATO is a threshold selection technique that can be used in various segmentation applications.MT201

Analysis of 3D Face Reconstruction

This thesis investigates the long standing problem of 3D reconstruction from a single 2D face image. Face reconstruction from a single 2D face image is an ill posed problem involving estimation of the intrinsic and the extrinsic camera parameters, light parameters, shape parameters and the texture parameters. The proposed approach has many potential applications in the law enforcement, surveillance, medicine, computer games and the entertainment industries. This problem is addressed using an analysis by synthesis framework by reconstructing a 3D face model from identity photographs. The identity photographs are a widely used medium for face identi cation and can be found on identity cards and passports. The novel contribution of this thesis is a new technique for creating 3D face models from a single 2D face image. The proposed method uses the improved dense 3D correspondence obtained using rigid and non-rigid registration techniques. The existing reconstruction methods use the optical ow method for establishing 3D correspondence. The resulting 3D face database is used to create a statistical shape model. The existing reconstruction algorithms recover shape by optimizing over all the parameters simultaneously. The proposed algorithm simplifies the reconstruction problem by using a step wise approach thus reducing the dimension of the parameter space and simplifying the opti- mization problem. In the alignment step, a generic 3D face is aligned with the given 2D face image by using anatomical landmarks. The texture is then warped onto the 3D model by using the spatial alignment obtained previously. The 3D shape is then recovered by optimizing over the shape parameters while matching a texture mapped model to the target image. There are a number of advantages of this approach. Firstly, it simpli es the optimization requirements and makes the optimization more robust. Second, there is no need to accurately recover the illumination parameters. Thirdly, there is no need for recovering the texture parameters by using a texture synthesis approach. Fourthly, quantitative analysis is used for improving the quality of reconstruction by improving the cost function. Previous methods use qualitative methods such as visual analysis, and face recognition rates for evaluating reconstruction accuracy. The improvement in the performance of the cost function occurs as a result of improvement in the feature space comprising the landmark and intensity features. Previously, the feature space has not been evaluated with respect to reconstruction accuracy thus leading to inaccurate assumptions about its behaviour. The proposed approach simpli es the reconstruction problem by using only identity images, rather than placing eff ort on overcoming the pose, illumination and expression (PIE) variations. This makes sense, as frontal face images under standard illumination conditions are widely available and could be utilized for accurate reconstruction. The reconstructed 3D models with texture can then be used for overcoming the PIE variations

A Methodology for Extracting Human Bodies from Still Images

Monitoring and surveillance of humans is one of the most prominent applications of today and it is expected to be part of many future aspects of our life, for safety reasons, assisted living and many others. Many efforts have been made towards automatic and robust solutions, but the general problem is very challenging and remains still open. In this PhD dissertation we examine the problem from many perspectives. First, we study the performance of a hardware architecture designed for large-scale surveillance systems. Then, we focus on the general problem of human activity recognition, present an extensive survey of methodologies that deal with this subject and propose a maturity metric to evaluate them. One of the numerous and most popular algorithms for image processing found in the field is image segmentation and we propose a blind metric to evaluate their results regarding the activity at local regions. Finally, we propose a fully automatic system for segmenting and extracting human bodies from challenging single images, which is the main contribution of the dissertation. Our methodology is a novel bottom-up approach relying mostly on anthropometric constraints and is facilitated by our research in the fields of face, skin and hands detection. Experimental results and comparison with state-of-the-art methodologies demonstrate the success of our approach

Ubiquitous Technologies for Emotion Recognition

Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now, with the advent of wearable, mobile, and ubiquitous technologies, that we can aim to sense and recognize emotions, continuously and in real time. This book brings together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and the recognition of human emotions

An automated computer-assisted approximation of the nose in South Africans from CBCT (Cone Beam Computed Tomography) scans

Thesis (PhD (Anatomy))--University of Pretoria, 2018.Each year in the Gauteng province of South Africa, approximately 1300 bodies are incinerated without a known identity (Bloom, 2015; Krüger et al., 2018). Because of various socio-economic reasons, identification is not always possible with conventional methods such as DNA comparisons and fingerprints. Therefore, more creative methods, including facial reconstruction, have been implemented to assist in the identification of unknown persons from their skeletal remains in South Africa. The aim of this thesis was to provide an automated computer-assisted method, independent of any forensic artistic interpretations, to create accurate statistical models for predicting nasal soft-tissue shape from information about the underlying skull substrate. The acquisition and extraction of the relevant anatomical structures (hard- and soft-tissue) were performed using an automatic dense landmarking procedure and analysed by geometric morphometrics. In this research, a validation of the precision of the automatic placement of landmarks, demonstrated its utilisation as a convenient prerequisite for geometric morphometric based shape analysis of the nasal complex. The automatic landmark positioning on hard- and soft-tissue 3D surfaces offered increased objectivity and the possibility of standardisation. In addition to reducing measurement errors in landmark placements, automatic landmarking, achieved a better precision for facial approximation, enabling the possibility to include more samples and populations with ease. A detailed study of the influence of factors (ancestry, sex, ageing and allometry) on the variability of the mid-facial skeleton among two South African ancestral groups were performed, revealing their statistically significant influences on the overall shape variation of the nose. Ancestry was found to be a very important factor in shape variation within the sample emphasising ancestral-specific differences. In addition, the expression of sexual dimorphism and effect of aging appeared to be different on distinct elements of the shape of the mid-facial region. From the findings, the two South African groups differed significantly regarding hard- and soft-tissue nasal complex morphology and their correlations, emphasising the importance of considering ancestry, sex and age as factors in the process of approximating the nose and highlighting the need for population specific accurate and reliable 3D statistical nose prediction methods. This study provided accurate statistical models using Partial Least Squared Regression (PLSR) algorithms which were optimised by including additional information such as ancestry, sex and age. Age and sex appeared to be important factors to be considered as additional information in order to improve the quality of the prediction. The predictions were based on a sample of 200 specimens resulting in an error when using the landmark-to-landmark distances on non-trained data, ranging between 2.139 mm and 2.833 mm for black South Africans at the tip of the nose and the alae, while they ranged from 2.575 mm to 2.859 mm for white South Africans. This research is the first attempt at a computer-assisted facial approximation of the nose with an automatic landmarking approach for the development of valid and reliable South African population specific standards using Cone Beam Computer-Tomography scans.AESOP + Erasmus Mundus ProgramAnatomyPhD (Anatomy)Unrestricte