Face image super-resolution using 2D CCA

In this paper a face super-resolution method using two-dimensional canonical correlation analysis (2D CCA) is presented. A detail compensation step is followed to add high-frequency components to the reconstructed high-resolution face. Unlike most of the previous researches on face super-resolution algorithms that first transform the images into vectors, in our approach the relationship between the high-resolution and the low-resolution face image are maintained in their original 2D representation. In addition, rather than approximating the entire face, different parts of a face image are super-resolved separately to better preserve the local structure. The proposed method is compared with various state-of-the-art super-resolution algorithms using multiple evaluation criteria including face recognition performance. Results on publicly available datasets show that the proposed method super-resolves high quality face images which are very close to the ground-truth and performance gain is not dataset dependent. The method is very efficient in both the training and testing phases compared to the other approaches. © 2013 Elsevier B.V

Robust Face Representation and Recognition Under Low Resolution and Difficult Lighting Conditions

This dissertation focuses on different aspects of face image analysis for accurate face recognition under low resolution and poor lighting conditions. A novel resolution enhancement technique is proposed for enhancing a low resolution face image into a high resolution image for better visualization and improved feature extraction, especially in a video surveillance environment. This method performs kernel regression and component feature learning in local neighborhood of the face images. It uses directional Fourier phase feature component to adaptively lean the regression kernel based on local covariance to estimate the high resolution image. For each patch in the neighborhood, four directional variances are estimated to adapt the interpolated pixels. A Modified Local Binary Pattern (MLBP) methodology for feature extraction is proposed to obtain robust face recognition under varying lighting conditions. Original LBP operator compares pixels in a local neighborhood with the center pixel and converts the resultant binary string to 8-bit integer value. So, it is less effective under difficult lighting conditions where variation between pixels is negligible. The proposed MLBP uses a two stage encoding procedure which is more robust in detecting this variation in a local patch. A novel dimensionality reduction technique called Marginality Preserving Embedding (MPE) is also proposed for enhancing the face recognition accuracy. Unlike Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), which project data in a global sense, MPE seeks for a local structure in the manifold. This is similar to other subspace learning techniques but the difference with other manifold learning is that MPE preserves marginality in local reconstruction. Hence it provides better representation in low dimensional space and achieves lower error rates in face recognition. Two new concepts for robust face recognition are also presented in this dissertation. In the first approach, a neural network is used for training the system where input vectors are created by measuring distance from each input to its class mean. In the second approach, half-face symmetry is used, realizing the fact that the face images may contain various expressions such as open/close eye, open/close mouth etc., and classify the top half and bottom half separately and finally fuse the two results. By performing experiments on several standard face datasets, improved results were observed in all the new proposed methodologies. Research is progressing in developing a unified approach for the extraction of features suitable for accurate face recognition in a long range video sequence in complex environments

A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal

Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images, which is a domain-specific image restoration problem in the low-level computer vision area. The early face restoration methods mainly use statistic priors and degradation models, which are difficult to meet the requirements of real-world applications in practice. In recent years, face restoration has witnessed great progress after stepping into the deep learning era. However, there are few works to study deep learning-based face restoration methods systematically. Thus, this paper comprehensively surveys recent advances in deep learning techniques for face restoration. Specifically, we first summarize different problem formulations and analyze the characteristic of the face image. Second, we discuss the challenges of face restoration. Concerning these challenges, we present a comprehensive review of existing FR methods, including prior based methods and deep learning-based methods. Then, we explore developed techniques in the task of FR covering network architectures, loss functions, and benchmark datasets. We also conduct a systematic benchmark evaluation on representative methods. Finally, we discuss future directions, including network designs, metrics, benchmark datasets, applications,etc. We also provide an open-source repository for all the discussed methods, which is available at https://github.com/TaoWangzj/Awesome-Face-Restoration.Comment: 21 pages, 19 figure

Face hallucination with application in far distance face recognition

In this thesis, faces captured in far distances are investigated. Face enhancement algorithms are studied. Hallucinating faces in holistic model and patch-based model are analysed respectively. The advantages and disadvantages of both models are discussed. An innovative holistic model and patch-based model are proposed separately. More investigation in practical surveillance environments are carried. And a new far face recognition model is proposed. Experiments demonstrate the improvement of proposed approaches

Novel Techniques for Automated Dental Identification

Automated dental identification is one of the best candidates for postmortem identification. With the large number of victims encountered in mass disasters, automating the process of postmortem identification is receiving an increased attention. This dissertation introduces new approaches for different stages of Automated Dental Identification system: These stages include segmentations, classification, labeling, and matching:;We modified the seam carving technique to adapt the problem of segmenting dental image records into individual teeth. We propose a two-stage teeth segmentation approach for segmenting the dental images. In the first stage, the teeth images are preprocessed by a two-step thresholding technique, which starts with an iterative thresholding followed by an adaptive thresholding to binarize the teeth images. In the second stage, we adapt the seam carving technique on the binary images, using both horizontal and vertical seams, to separate each individual tooth. We have obtained an optimality rate of 54.02% for the bitewing type images, which is superior to all existing fully automated dental segmentation algorithms in the literature, and a failure rate of 1.05%. For the periapical type images, we have obtained a high optimality rate of 58.13% and a low failure rate of 0.74 which also surpasses the performance of existing techniques. An important problem in automated dental identification is automatic classification of teeth into four classes (molars, premolars, canines, and incisors). A dental chart is a key to avoiding illogical comparisons that inefficiently consume the limited computational resources, and may mislead decision-making. We tackle this composite problem using a two-stage approach. The first stage, utilizes low computational-cost, appearance-based features, using Orthogonal Locality Preserving Projections (OLPP) for assigning an initial class. The second stage applies a string matching technique, based on teeth neighborhood rules, to validate initial teeth-classes and hence to assign each tooth a number corresponding to its location in the dental chart, even in the presence of a missed tooth. The experimental results of teeth classification show that on a large dataset of bitewing and periapical films, the proposed approach achieves overall classification accuracy of 77% and teeth class validation enhances the overall teeth classification accuracy to 87% which is slightly better than the performance obtained from previous methods based on EigenTeeth the performance of which is 75% and 86%, respectively.;We present a new technique that searches the dental database to find a candidate list. We use dental records of the FBI\u27s Criminal Justice Service (CJIC) ADIS database, that contains 104 records (about 500 bitewing and periapical films) involving more than 2000 teeth, 47 Antemortem (AM) records and 57 Postmortem (PM) records with 20 matched records.;The proposed approach consists of two main stages, the first stage is to preprocess the dental records (segmentation and teeth labeling classification) in order to get a reliable, appearance-based, low computational-cost feature. In the second stage, we developed a technique based on LaplacianTeeth using OLPP algorithm to produce a candidate list. The proposed technique can correctly retrieve the dental records 65% in the 5 top ranks while the method based on EigenTeeth remains at 60%. The proposed approach takes about 0.17 seconds to make record to record comparison while the other method based on EigenTeeth takes about 0.09 seconds.;Finally, we address the teeth matching problem by presenting a new technique for dental record retrieval. The technique is based on the matching of the Scale Invariant feature Transform (SIFT) descriptors guided by the teeth contour between the subject and reference dental records. Our fundamental objective is to accomplish a relatively short match list, with a high probability of having the correct match reference. The proposed technique correctly retrieves the dental records with performance rates of 35% and 75% in the 1 and 5 top ranks respectively, and takes only an average time of 4.18 minutes to retrieve a match list. This compares favorably with the existing technique shape-based (edge direction histogram) method which has the performance rates of 29% and 46% in the 1 and 5 top ranks respectively.;In summary, the proposed ADIS system accurately retrieves the dental record with an overall rate of 80% in top 5 ranks when a candidate list of 20 is used (from potential match search) whereas a candidate size of 10 yields an overall rate of 84% in top 5 ranks and takes only a few minutes to search the database, which compares favorably against most of the existing methods in the literature, when both accuracy and computational complexity are considered