Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement

We address the problem of restoring a high-resolution face image from a blurry low-resolution input. This problem is difficult as super-resolution and deblurring need to be tackled simultaneously. Moreover, existing algorithms cannot handle face images well as low-resolution face images do not have much texture which is especially critical for deblurring. In this paper, we propose an effective algorithm by utilizing the domain-specific knowledge of human faces to recover high-quality faces. We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image. However, the CNN based method cannot handle image details well. We further develop a novel exemplar-based detail enhancement algorithm via facial component matching. Extensive experiments show that the proposed method outperforms the state-of-the-art algorithms both quantitatively and qualitatively.Comment: In IJCV 201

Valvekaameratel põhineva inimseire täiustamine pildi resolutsiooni parandamise ning näotuvastuse abil

Due to importance of security in the society, monitoring activities and recognizing specific people through surveillance video camera is playing an important role. One of the main issues in such activity rises from the fact that cameras do not meet the resolution requirement for many face recognition algorithms. In order to solve this issue, in this work we are proposing a new system which super resolve the image. First, we are using sparse representation with the specific dictionary involving many natural and facial images to super resolve images. As a second method, we are using deep learning convulutional network. Image super resolution is followed by Hidden Markov Model and Singular Value Decomposition based face recognition. The proposed system has been tested on many well-known face databases such as FERET, HeadPose, and Essex University databases as well as our recently introduced iCV Face Recognition database (iCV-F). The experimental results shows that the recognition rate is increasing considerably after applying the super resolution by using facial and natural image dictionary. In addition, we are also proposing a system for analysing people movement on surveillance video. People including faces are detected by using Histogram of Oriented Gradient features and Viola-jones algorithm. Multi-target tracking system with discrete-continuouos energy minimization tracking system is then used to track people. The tracking data is then in turn used to get information about visited and passed locations and face recognition results for tracked people

Face Recognition Methodologies Using Component Analysis: The Contemporary Affirmation of The Recent Literature

This paper explored the contemporary affirmation of the recent literature in the context of face recognition systems, a review motivated by contradictory claims in the literature. This paper shows how the relative performance of recent claims based on methodologies such as PCA and ICA, which are depend on the task statement. It then explores the space of each model acclaimed in recent literature. In the process, this paper verifies the results of many of the face recognition models in the literature, and relates them to each other and to this work

HDTR-Net: A Real-Time High-Definition Teeth Restoration Network for Arbitrary Talking Face Generation Methods

Talking Face Generation (TFG) aims to reconstruct facial movements to achieve high natural lip movements from audio and facial features that are under potential connections. Existing TFG methods have made significant advancements to produce natural and realistic images. However, most work rarely takes visual quality into consideration. It is challenging to ensure lip synchronization while avoiding visual quality degradation in cross-modal generation methods. To address this issue, we propose a universal High-Definition Teeth Restoration Network, dubbed HDTR-Net, for arbitrary TFG methods. HDTR-Net can enhance teeth regions at an extremely fast speed while maintaining synchronization, and temporal consistency. In particular, we propose a Fine-Grained Feature Fusion (FGFF) module to effectively capture fine texture feature information around teeth and surrounding regions, and use these features to fine-grain the feature map to enhance the clarity of teeth. Extensive experiments show that our method can be adapted to arbitrary TFG methods without suffering from lip synchronization and frame coherence. Another advantage of HDTR-Net is its real-time generation ability. Also under the condition of high-definition restoration of talking face video synthesis, its inference speed is $300\%$ faster than the current state-of-the-art face restoration based on super-resolution.Comment: 15pages, 6 figures, PRCV202

A unified framework for subspace based face recognition.

Wang Xiaogang.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 88-91).Abstracts in English and Chinese.Abstract --- p.iAcknowledgments --- p.vTable of Contents --- p.viList of Figures --- p.viiiList of Tables --- p.xChapter Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Face recognition --- p.1Chapter 1.2 --- Subspace based face recognition technique --- p.2Chapter 1.3 --- Unified framework for subspace based face recognition --- p.4Chapter 1.4 --- Discriminant analysis in dual intrapersonal subspaces --- p.5Chapter 1.5 --- Face sketch recognition and hallucination --- p.6Chapter 1.6 --- Organization of this thesis --- p.7Chapter Chapter 2 --- Review of Subspace Methods --- p.8Chapter 2.1 --- PCA --- p.8Chapter 2.2 --- LDA --- p.9Chapter 2.3 --- Bayesian algorithm --- p.12Chapter Chapter 3 --- A Unified Framework --- p.14Chapter 3.1 --- PCA eigenspace --- p.16Chapter 3.2 --- Intrapersonal and extrapersonal subspaces --- p.17Chapter 3.3 --- LDA subspace --- p.18Chapter 3.4 --- Comparison of the three subspaces --- p.19Chapter 3.5 --- L-ary versus binary classification --- p.22Chapter 3.6 --- Unified subspace analysis --- p.23Chapter 3.7 --- Discussion --- p.26Chapter Chapter 4 --- Experiments on Unified Subspace Analysis --- p.28Chapter 4.1 --- Experiments on FERET database --- p.28Chapter 4.1.1 --- PCA Experiment --- p.28Chapter 4.1.2 --- Bayesian experiment --- p.29Chapter 4.1.3 --- Bayesian analysis in reduced PCA subspace --- p.30Chapter 4.1.4 --- Extract discriminant features from intrapersonal subspace --- p.33Chapter 4.1.5 --- Subspace analysis using different training sets --- p.34Chapter 4.2 --- Experiments on the AR face database --- p.36Chapter 4.2.1 --- "Experiments on PCA, LDA and Bayes" --- p.37Chapter 4.2.2 --- Evaluate the Bayesian algorithm for different transformation --- p.38Chapter Chapter 5 --- Discriminant Analysis in Dual Subspaces --- p.41Chapter 5.1 --- Review of LDA in the null space of and direct LDA --- p.42Chapter 5.1.1 --- LDA in the null space of --- p.42Chapter 5.1.2 --- Direct LDA --- p.43Chapter 5.1.3 --- Discussion --- p.44Chapter 5.2 --- Discriminant analysis in dual intrapersonal subspaces --- p.45Chapter 5.3 --- Experiment --- p.50Chapter 5.3.1 --- Experiment on FERET face database --- p.50Chapter 5.3.2 --- Experiment on the XM2VTS database --- p.53Chapter Chapter 6 --- Eigentransformation: Subspace Transform --- p.54Chapter 6.1 --- Face sketch recognition --- p.54Chapter 6.1.1 --- Eigentransformation --- p.56Chapter 6.1.2 --- Sketch synthesis --- p.59Chapter 6.1.3 --- Face sketch recognition --- p.61Chapter 6.1.4 --- Experiment --- p.63Chapter 6.2 --- Face hallucination --- p.69Chapter 6.2.1 --- Multiresolution analysis --- p.71Chapter 6.2.2 --- Eigentransformation for hallucination --- p.72Chapter 6.2.3 --- Discussion --- p.75Chapter 6.2.4 --- Experiment --- p.77Chapter 6.3 --- Discussion --- p.83Chapter Chapter 7 --- Conclusion --- p.85Publication List of This Thesis --- p.87Bibliography --- p.8