A Survey on Face Recognition Techniques

Face detection is a computer technology that determines the locations and sizes of human faces in arbitrary (digital) images. It detects facial features and ignores anything else, such as buildings, trees and bodies. Face detection can be regarded as a specific case of object-class detection. In object-class detection, the task is to find the locations and sizes of all objects in an image that belong to a given class. Examples include upper torsos, pedestrians, and cars. Face detection can be regarded as a more general case of face localization. These days face detection is current research area. The face detection is normally done using ANN, CBIR, LDA and PCA. Keywords:- ANN, CBIR, LDA and PC

Discriminant feature pursuit: from statistical learning to informative learning.

Lin Dahua.Thesis (M.Phil.)--Chinese University of Hong Kong, 2006.Includes bibliographical references (leaves 233-250).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- The Problem We are Facing --- p.1Chapter 1.2 --- Generative vs. Discriminative Models --- p.2Chapter 1.3 --- Statistical Feature Extraction: Success and Challenge --- p.3Chapter 1.4 --- Overview of Our Works --- p.5Chapter 1.4.1 --- New Linear Discriminant Methods: Generalized LDA Formulation and Performance-Driven Sub space Learning --- p.5Chapter 1.4.2 --- Coupled Learning Models: Coupled Space Learning and Inter Modality Recognition --- p.6Chapter 1.4.3 --- Informative Learning Approaches: Conditional Infomax Learning and Information Chan- nel Model --- p.6Chapter 1.5 --- Organization of the Thesis --- p.8Chapter I --- History and Background --- p.10Chapter 2 --- Statistical Pattern Recognition --- p.11Chapter 2.1 --- Patterns and Classifiers --- p.11Chapter 2.2 --- Bayes Theory --- p.12Chapter 2.3 --- Statistical Modeling --- p.14Chapter 2.3.1 --- Maximum Likelihood Estimation --- p.14Chapter 2.3.2 --- Gaussian Model --- p.15Chapter 2.3.3 --- Expectation-Maximization --- p.17Chapter 2.3.4 --- Finite Mixture Model --- p.18Chapter 2.3.5 --- A Nonparametric Technique: Parzen Windows --- p.21Chapter 3 --- Statistical Learning Theory --- p.24Chapter 3.1 --- Formulation of Learning Model --- p.24Chapter 3.1.1 --- Learning: Functional Estimation Model --- p.24Chapter 3.1.2 --- Representative Learning Problems --- p.25Chapter 3.1.3 --- Empirical Risk Minimization --- p.26Chapter 3.2 --- Consistency and Convergence of Learning --- p.27Chapter 3.2.1 --- Concept of Consistency --- p.27Chapter 3.2.2 --- The Key Theorem of Learning Theory --- p.28Chapter 3.2.3 --- VC Entropy --- p.29Chapter 3.2.4 --- Bounds on Convergence --- p.30Chapter 3.2.5 --- VC Dimension --- p.35Chapter 4 --- History of Statistical Feature Extraction --- p.38Chapter 4.1 --- Linear Feature Extraction --- p.38Chapter 4.1.1 --- Principal Component Analysis (PCA) --- p.38Chapter 4.1.2 --- Linear Discriminant Analysis (LDA) --- p.41Chapter 4.1.3 --- Other Linear Feature Extraction Methods --- p.46Chapter 4.1.4 --- Comparison of Different Methods --- p.48Chapter 4.2 --- Enhanced Models --- p.49Chapter 4.2.1 --- Stochastic Discrimination and Random Subspace --- p.49Chapter 4.2.2 --- Hierarchical Feature Extraction --- p.51Chapter 4.2.3 --- Multilinear Analysis and Tensor-based Representation --- p.52Chapter 4.3 --- Nonlinear Feature Extraction --- p.54Chapter 4.3.1 --- Kernelization --- p.54Chapter 4.3.2 --- Dimension reduction by Manifold Embedding --- p.56Chapter 5 --- Related Works in Feature Extraction --- p.59Chapter 5.1 --- Dimension Reduction --- p.59Chapter 5.1.1 --- Feature Selection --- p.60Chapter 5.1.2 --- Feature Extraction --- p.60Chapter 5.2 --- Kernel Learning --- p.61Chapter 5.2.1 --- Basic Concepts of Kernel --- p.61Chapter 5.2.2 --- The Reproducing Kernel Map --- p.62Chapter 5.2.3 --- The Mercer Kernel Map --- p.64Chapter 5.2.4 --- The Empirical Kernel Map --- p.65Chapter 5.2.5 --- Kernel Trick and Kernelized Feature Extraction --- p.66Chapter 5.3 --- Subspace Analysis --- p.68Chapter 5.3.1 --- Basis and Subspace --- p.68Chapter 5.3.2 --- Orthogonal Projection --- p.69Chapter 5.3.3 --- Orthonormal Basis --- p.70Chapter 5.3.4 --- Subspace Decomposition --- p.70Chapter 5.4 --- Principal Component Analysis --- p.73Chapter 5.4.1 --- PCA Formulation --- p.73Chapter 5.4.2 --- Solution to PCA --- p.75Chapter 5.4.3 --- Energy Structure of PCA --- p.76Chapter 5.4.4 --- Probabilistic Principal Component Analysis --- p.78Chapter 5.4.5 --- Kernel Principal Component Analysis --- p.81Chapter 5.5 --- Independent Component Analysis --- p.83Chapter 5.5.1 --- ICA Formulation --- p.83Chapter 5.5.2 --- Measurement of Statistical Independence --- p.84Chapter 5.6 --- Linear Discriminant Analysis --- p.85Chapter 5.6.1 --- Fisher's Linear Discriminant Analysis --- p.85Chapter 5.6.2 --- Improved Algorithms for Small Sample Size Problem . --- p.89Chapter 5.6.3 --- Kernel Discriminant Analysis --- p.92Chapter II --- Improvement in Linear Discriminant Analysis --- p.100Chapter 6 --- Generalized LDA --- p.101Chapter 6.1 --- Regularized LDA --- p.101Chapter 6.1.1 --- Generalized LDA Implementation Procedure --- p.101Chapter 6.1.2 --- Optimal Nonsingular Approximation --- p.103Chapter 6.1.3 --- Regularized LDA algorithm --- p.104Chapter 6.2 --- A Statistical View: When is LDA optimal? --- p.105Chapter 6.2.1 --- Two-class Gaussian Case --- p.106Chapter 6.2.2 --- Multi-class Cases --- p.107Chapter 6.3 --- Generalized LDA Formulation --- p.108Chapter 6.3.1 --- Mathematical Preparation --- p.108Chapter 6.3.2 --- Generalized Formulation --- p.110Chapter 7 --- Dynamic Feedback Generalized LDA --- p.112Chapter 7.1 --- Basic Principle --- p.112Chapter 7.2 --- Dynamic Feedback Framework --- p.113Chapter 7.2.1 --- Initialization: K-Nearest Construction --- p.113Chapter 7.2.2 --- Dynamic Procedure --- p.115Chapter 7.3 --- Experiments --- p.115Chapter 7.3.1 --- Performance in Training Stage --- p.116Chapter 7.3.2 --- Performance on Testing set --- p.118Chapter 8 --- Performance-Driven Subspace Learning --- p.119Chapter 8.1 --- Motivation and Principle --- p.119Chapter 8.2 --- Performance-Based Criteria --- p.121Chapter 8.2.1 --- The Verification Problem and Generalized Average Margin --- p.122Chapter 8.2.2 --- Performance Driven Criteria based on Generalized Average Margin --- p.123Chapter 8.3 --- Optimal Subspace Pursuit --- p.125Chapter 8.3.1 --- Optimal threshold --- p.125Chapter 8.3.2 --- Optimal projection matrix --- p.125Chapter 8.3.3 --- Overall procedure --- p.129Chapter 8.3.4 --- Discussion of the Algorithm --- p.129Chapter 8.4 --- Optimal Classifier Fusion --- p.130Chapter 8.5 --- Experiments --- p.131Chapter 8.5.1 --- Performance Measurement --- p.131Chapter 8.5.2 --- Experiment Setting --- p.131Chapter 8.5.3 --- Experiment Results --- p.133Chapter 8.5.4 --- Discussion --- p.139Chapter III --- Coupled Learning of Feature Transforms --- p.140Chapter 9 --- Coupled Space Learning --- p.141Chapter 9.1 --- Introduction --- p.142Chapter 9.1.1 --- What is Image Style Transform --- p.142Chapter 9.1.2 --- Overview of our Framework --- p.143Chapter 9.2 --- Coupled Space Learning --- p.143Chapter 9.2.1 --- Framework of Coupled Modelling --- p.143Chapter 9.2.2 --- Correlative Component Analysis --- p.145Chapter 9.2.3 --- Coupled Bidirectional Transform --- p.148Chapter 9.2.4 --- Procedure of Coupled Space Learning --- p.151Chapter 9.3 --- Generalization to Mixture Model --- p.152Chapter 9.3.1 --- Coupled Gaussian Mixture Model --- p.152Chapter 9.3.2 --- Optimization by EM Algorithm --- p.152Chapter 9.4 --- Integrated Framework for Image Style Transform --- p.154Chapter 9.5 --- Experiments --- p.156Chapter 9.5.1 --- Face Super-resolution --- p.156Chapter 9.5.2 --- Portrait Style Transforms --- p.157Chapter 10 --- Inter-Modality Recognition --- p.162Chapter 10.1 --- Introduction to the Inter-Modality Recognition Problem . . . --- p.163Chapter 10.1.1 --- What is Inter-Modality Recognition --- p.163Chapter 10.1.2 --- Overview of Our Feature Extraction Framework . . . . --- p.163Chapter 10.2 --- Common Discriminant Feature Extraction --- p.165Chapter 10.2.1 --- Formulation of the Learning Problem --- p.165Chapter 10.2.2 --- Matrix-Form of the Objective --- p.168Chapter 10.2.3 --- Solving the Linear Transforms --- p.169Chapter 10.3 --- Kernelized Common Discriminant Feature Extraction --- p.170Chapter 10.4 --- Multi-Mode Framework --- p.172Chapter 10.4.1 --- Multi-Mode Formulation --- p.172Chapter 10.4.2 --- Optimization Scheme --- p.174Chapter 10.5 --- Experiments --- p.176Chapter 10.5.1 --- Experiment Settings --- p.176Chapter 10.5.2 --- Experiment Results --- p.177Chapter IV --- A New Perspective: Informative Learning --- p.180Chapter 11 --- Toward Information Theory --- p.181Chapter 11.1 --- Entropy and Mutual Information --- p.181Chapter 11.1.1 --- Entropy --- p.182Chapter 11.1.2 --- Relative Entropy (Kullback Leibler Divergence) --- p.184Chapter 11.2 --- Mutual Information --- p.184Chapter 11.2.1 --- Definition of Mutual Information --- p.184Chapter 11.2.2 --- Chain rules --- p.186Chapter 11.2.3 --- Information in Data Processing --- p.188Chapter 11.3 --- Differential Entropy --- p.189Chapter 11.3.1 --- Differential Entropy of Continuous Random Variable . --- p.189Chapter 11.3.2 --- Mutual Information of Continuous Random Variable . --- p.190Chapter 12 --- Conditional Infomax Learning --- p.191Chapter 12.1 --- An Overview --- p.192Chapter 12.2 --- Conditional Informative Feature Extraction --- p.193Chapter 12.2.1 --- Problem Formulation and Features --- p.193Chapter 12.2.2 --- The Information Maximization Principle --- p.194Chapter 12.2.3 --- The Information Decomposition and the Conditional Objective --- p.195Chapter 12.3 --- The Efficient Optimization --- p.197Chapter 12.3.1 --- Discrete Approximation Based on AEP --- p.197Chapter 12.3.2 --- Analysis of Terms and Their Derivatives --- p.198Chapter 12.3.3 --- Local Active Region Method --- p.200Chapter 12.4 --- Bayesian Feature Fusion with Sparse Prior --- p.201Chapter 12.5 --- The Integrated Framework for Feature Learning --- p.202Chapter 12.6 --- Experiments --- p.203Chapter 12.6.1 --- A Toy Problem --- p.203Chapter 12.6.2 --- Face Recognition --- p.204Chapter 13 --- Channel-based Maximum Effective Information --- p.209Chapter 13.1 --- Motivation and Overview --- p.209Chapter 13.2 --- Maximizing Effective Information --- p.211Chapter 13.2.1 --- Relation between Mutual Information and Classification --- p.211Chapter 13.2.2 --- Linear Projection and Metric --- p.212Chapter 13.2.3 --- Channel Model and Effective Information --- p.213Chapter 13.2.4 --- Parzen Window Approximation --- p.216Chapter 13.3 --- Parameter Optimization on Grassmann Manifold --- p.217Chapter 13.3.1 --- Grassmann Manifold --- p.217Chapter 13.3.2 --- Conjugate Gradient Optimization on Grassmann Manifold --- p.219Chapter 13.3.3 --- Computation of Gradient --- p.221Chapter 13.4 --- Experiments --- p.222Chapter 13.4.1 --- A Toy Problem --- p.222Chapter 13.4.2 --- Face Recognition --- p.223Chapter 14 --- Conclusion --- p.23

Discriminant feature analysis for pattern recognition

Ph.DDOCTOR OF PHILOSOPH

Discriminant analysis based feature extraction for pattern recognition

Fisher's linear discriminant analysis (FLDA) has been widely used in pattern recognition applications. However, this method cannot be applied for solving the pattern recognition problems if the within-class scatter matrix is singular, a condition that occurs when the number of the samples is small relative to the dimension of the samples. This problem is commonly known as the small sample size (SSS) problem and many of the FLDA variants proposed in the past to deal with this problem suffer from excessive computational load because of the high dimensionality of patterns or lose some useful discriminant information. This study is concerned with developing efficient techniques for discriminant analysis of patterns while at the same time overcoming the small sample size problem. With this objective in mind, the work of this research is divided into two parts. In part 1, a technique by solving the problem of generalized singular value decomposition (GSVD) through eigen-decomposition is developed for linear discriminant analysis (LDA). The resulting algorithm referred to as modified GSVD-LDA (MGSVD-LDA) algorithm is thus devoid of the singularity problem of the scatter matrices of the traditional LDA methods. A theorem enunciating certain properties of the discriminant subspace derived by the proposed GSVD-based algorithms is established. It is shown that if the samples of a dataset are linearly independent, then the samples belonging to different classes are linearly separable in the derived discriminant subspace; and thus, the proposed MGSVD-LDA algorithm effectively captures the class structure of datasets with linearly independent samples. Inspired by the results of this theorem that essentially establishes a class separability of linearly independent samples in a specific discriminant subspace, in part 2, a new systematic framework for the pattern recognition of linearly independent samples is developed. Within this framework, a discriminant model, in which the samples of the individual classes of the dataset lie on parallel hyperplanes and project to single distinct points of a discriminant subspace of the underlying input space, is shown to exist. Based on this model, a number of algorithms that are devoid of the SSS problem are developed to obtain this discriminant subspace for datasets with linearly independent samples. For the discriminant analysis of datasets for which the samples are not linearly independent, some of the linear algorithms developed in this thesis are also kernelized. Extensive experiments are conducted throughout this investigation in order to demonstrate the validity and effectiveness of the ideas developed in this study. It is shown through simulation results that the linear and nonlinear algorithms for discriminant analysis developed in this thesis provide superior performance in terms of the recognition accuracy and computational complexit

A Learning-based Approach to Exploiting Sensing Diversity in Performance Critical Sensor Networks

Wireless sensor networks for human health monitoring, military surveillance, and disaster warning all have stringent accuracy requirements for detecting and classifying events while maximizing system lifetime. to meet high accuracy requirements and maximize system lifetime, we must address sensing diversity: sensing capability differences among both heterogeneous and homogeneous sensors in a specific deployment. Existing approaches either ignore sensing diversity entirely and assume all sensors have similar capabilities or attempt to overcome sensing diversity through calibration. Instead, we use machine learning to take advantage of sensing differences among heterogeneous sensors to provide high accuracy and energy savings for performance critical applications.;In this dissertation, we provide five major contributions that exploit the nuances of specific sensor deployments to increase application performance. First, we demonstrate that by using machine learning for event detection, we can explore the sensing capability of a specific deployment and use only the most capable sensors to meet user accuracy requirements. Second, we expand our diversity exploiting approach to detect multiple events using a distributed manner. Third, we address sensing diversity in body sensor networks, providing a practical, user friendly solution for activity recognition. Fourth, we further increase accuracy and energy savings in body sensor networks by sharing sensing resources among neighboring body sensor networks. Lastly, we provide a learning-based approach for forwarding event detection decisions to data sinks in an environment with mobile sensor nodes

Telecom customer segmentation and precise package design by using data mining

Changes in the form of communication have prompted the telecommunications industry to flourish. In the "big data era" of information explosion, as one of the leading industries in the information age, the development of the telecommunications industry depends not only on communication technology, but also on the ability of enterprises to optimize resource allocation. At present, the information resources owned by telecom companies mainly come from customers. During the development process, they have accumulated a large amount of customer data, which truly and objectively reflects the behavior of consumers. This paper is dedicated to combining data mining technology with the rich data resources of the telecom industry and the latest marketing theories, not only effectively helping subdivide the telecommunications customer market, but also supporting telecommunications companies in developing more accurate and efficient marketing strategies. In addition, data analysis method such as factor analysis, regression analysis and discriminant analysis are used to analyze the demographic, business, SMS messages and expense characteristics of telecom customers, providing a new vision and reference for the telecom industry to achieve accurate packaging design. Based on the above research results, a discriminant model for the loss of telecom customers is constructed, which will help telecommunications companies to obtain a control method for telecom customer management risk. At last, data mining technology is used to optimize the combination design of telecommunication services, which offer effective advice on precise telecom package design to telecommunications companies

Machine learning algorithms for cognitive radio wireless networks

In this thesis new methods are presented for achieving spectrum sensing in cognitive radio wireless networks. In particular, supervised, semi-supervised and unsupervised machine learning based spectrum sensing algorithms are developed and various techniques to improve their performance are described. Spectrum sensing problem in multi-antenna cognitive radio networks is considered and a novel eigenvalue based feature is proposed which has the capability to enhance the performance of support vector machines algorithms for signal classification. Furthermore, spectrum sensing under multiple primary users condition is studied and a new re-formulation of the sensing task as a multiple class signal detection problem where each class embeds one or more states is presented. Moreover, the error correcting output codes based multi-class support vector machines algorithms is proposed and investigated for solving the multiple class signal detection problem using two different coding strategies. In addition, the performance of parametric classifiers for spectrum sensing under slow fading channel is studied. To address the attendant performance degradation problem, a Kalman filter based channel estimation technique is proposed for tracking the temporally correlated slow fading channel and updating the decision boundary of the classifiers in real time. Simulation studies are included to assess the performance of the proposed schemes. Finally, techniques for improving the quality of the learning features and improving the detection accuracy of sensing algorithms are studied and a novel beamforming based pre-processing technique is presented for feature realization in multi-antenna cognitive radio systems. Furthermore, using the beamformer derived features, new algorithms are developed for multiple hypothesis testing facilitating joint spatio-temporal spectrum sensing. The key performance metrics of the classifiers are evaluated to demonstrate the superiority of the proposed methods in comparison with previously proposed alternatives

A generic face processing framework: technologies, analyses and applications.

Jang Kim-fung.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 108-124).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Background --- p.1Chapter 1.2 --- Introduction about Face Processing Framework --- p.4Chapter 1.2.1 --- Basic architecture --- p.4Chapter 1.2.2 --- Face detection --- p.5Chapter 1.2.3 --- Face tracking --- p.6Chapter 1.2.4 --- Face recognition --- p.6Chapter 1.3 --- The scope and contributions of the thesis --- p.7Chapter 1.4 --- The outline of the thesis --- p.8Chapter 2 --- Facial Feature Representation --- p.10Chapter 2.1 --- Facial feature analysis --- p.10Chapter 2.1.1 --- Pixel information --- p.11Chapter 2.1.2 --- Geometry information --- p.13Chapter 2.2 --- Extracting and coding of facial feature --- p.14Chapter 2.2.1 --- Face recognition --- p.15Chapter 2.2.2 --- Facial expression classification --- p.38Chapter 2.2.3 --- Other related work --- p.44Chapter 2.3 --- Discussion about facial feature --- p.48Chapter 2.3.1 --- Performance evaluation for face recognition --- p.49Chapter 2.3.2 --- Evolution of the face recognition --- p.52Chapter 2.3.3 --- Evaluation of two state-of-the-art face recog- nition methods --- p.53Chapter 2.4 --- Problem for current situation --- p.58Chapter 3 --- Face Detection Algorithms and Committee Ma- chine --- p.61Chapter 3.1 --- Introduction about face detection --- p.62Chapter 3.2 --- Face Detection Committee Machine --- p.64Chapter 3.2.1 --- Review of three approaches for committee machine --- p.65Chapter 3.2.2 --- The approach of FDCM --- p.68Chapter 3.3 --- Evaluation --- p.70Chapter 4 --- Facial Feature Localization --- p.73Chapter 4.1 --- Algorithm for gray-scale image: template match- ing and separability filter --- p.73Chapter 4.1.1 --- Position of face and eye region --- p.74Chapter 4.1.2 --- Position of irises --- p.75Chapter 4.1.3 --- Position of lip --- p.79Chapter 4.2 --- Algorithm for color image: eyemap and separa- bility filter --- p.81Chapter 4.2.1 --- Position of eye candidates --- p.81Chapter 4.2.2 --- Position of mouth candidates --- p.83Chapter 4.2.3 --- Selection of face candidates by cost function --- p.84Chapter 4.3 --- Evaluation --- p.85Chapter 4.3.1 --- Algorithm for gray-scale image --- p.86Chapter 4.3.2 --- Algorithm for color image --- p.88Chapter 5 --- Face Processing System --- p.92Chapter 5.1 --- System architecture and limitations --- p.92Chapter 5.2 --- Pre-processing module --- p.93Chapter 5.2.1 --- Ellipse color model --- p.94Chapter 5.3 --- Face detection module --- p.96Chapter 5.3.1 --- Choosing the classifier --- p.96Chapter 5.3.2 --- Verifying the candidate region --- p.97Chapter 5.4 --- Face tracking module --- p.99Chapter 5.4.1 --- Condensation algorithm --- p.99Chapter 5.4.2 --- Tracking the region using Hue color model --- p.101Chapter 5.5 --- Face recognition module --- p.102Chapter 5.5.1 --- Normalization --- p.102Chapter 5.5.2 --- Recognition --- p.103Chapter 5.6 --- Applications --- p.104Chapter 6 --- Conclusion --- p.106Bibliography --- p.10