Automatic Facial Feature Detection for Facial Expression Recognition

International audienceThis paper presents a real-time automatic facial feature point detection method for facial expression recognition. The system is capable of detecting seven facial feature points (eyebrows, pupils, nose, and corners of mouth) in grayscale images extracted from a given video. Extracted feature points then used for facial expression recognition. Neutral, happiness and surprise emotions have been studied on the Bosphorus dataset and tested on FG-NET video dataset using OpenCV. We compared our results with previous studies on this dataset. Our experiments showed that proposed method has the advantage of locating facial feature points automatically and accurately in real-time

Survey Paper on Emotion Recognition

Facial expressions give important information about emotions of a person. Understanding facial expressions accurately is one of the challenging tasks for interpersonal relationships. Automatic emotion detection using facial expressions recognition is now a main area of interest within various fields such as computer science, medicine, and psychology. HCI research communities also use automated facial expression recognition system for better results. Various feature extraction techniques have been developed for recognition of expressions from static images as well as real time videos. This paper provides a review of research work carried out and published in the field of facial expression recognition and various techniques used for facial expression recognition

Facial Action Units Intensity Estimation by the Fusion of Features with Multi-kernel Support Vector Machine

International audience— Automatic facial expression recognition has emerged over two decades. The recognition of the posed facial expressions and the detection of Action Units (AUs) of facial expression have already made great progress. More recently, the automatic estimation of the variation of facial expression, either in terms of the intensities of AUs or in terms of the values of dimensional emotions, has emerged in the field of the facial expression analysis. However, discriminating different intensities of AUs is a far more challenging task than AUs detection due to several intractable problems. Aiming to continuing standardized evaluation procedures and surpass the limits of the current research, the second Facial Expression Recognition and Analysis challenge (FERA2015) is presented. In this context, we propose a method using the fusion of the different appearance and geometry features based on a multi-kernel Support Vector Machine (SVM) for the automatic estimation of the intensities of the AUs. The result of our approach benefiting from taking advantages of the different features adapting to a multi-kernel SVM is shown to outperform the conventional methods based on the mono-type feature with single kernel SVM

Discriminant feature extraction and selection for person-independent facial expression recognition

This thesis is to develop new facial expression recognition techniques based on 2D/3D images or videos, with the purpose to improve the recognition efficiency and accuracy of the current state-of-art. A fully automatic facial expression recognition system is designed, including real-time landmark detection, spatio-temporal feature extraction, hierarchical classification, and most discriminant facial regions identification for expression recognition. In general, the proposed system improved the facial expression recognition state-of-art

Facial expression based emotion detection-A Review

Emotion detection is the task of recognizing a person’s emotion state. Understanding facial expression accurately is one of the challenging task for interpersonal relationship. Automatic emotion detection using facial expressions recognition is now a main area of interest within various fields such as computer vision, medicine and psychology. Various feature extraction techniques have been developed for recognition of expression from static images as well as real time videos. Artificial Neural Network (ANN) based detection for emotion like anger, confusion, happy, sad, annoyed, stressed etc. is now a days, gathering more popularity among the researchers as it provides better results. Human emotion can be detected image through digital processing. Few work done on emotion detection, those are published recently reviewed are summarizes here briefly.Keywords—Emotion, interpersonal, facial expression, ANN, Recognition, extraction, digital processin

Facial Expression Recognition Using Euclidean Distance Method

Facial expression recognition is found to be useful for emotion science, clinical psychology and pain assessment. In the proposed method, the face detection algorithm involves lighting compensation for getting uniformity on face and morphological operations for retaining the required face portion. After retaining the face portion in the image, the facial features like eyes, nose, and mouth are extracted using AAM (Active Appearance Model) method. For automatic facial expression recognition, simple Euclidean Distance method is used. In this method, the Euclidean distance between the feature points of the training images and that of the query image is compared. Based on minimum Euclidean distance, output image expression is decided

Timing is everything: A spatio-temporal approach to the analysis of facial actions

This thesis presents a fully automatic facial expression analysis system based on the Facial Action Coding System (FACS). FACS is the best known and the most commonly used system to describe facial activity in terms of facial muscle actions (i.e., action units, AUs). We will present our research on the analysis of the morphological, spatio-temporal and behavioural aspects of facial expressions. In contrast with most other researchers in the field who use appearance based techniques, we use a geometric feature based approach. We will argue that that approach is more suitable for analysing facial expression temporal dynamics. Our system is capable of explicitly exploring the temporal aspects of facial expressions from an input colour video in terms of their onset (start), apex (peak) and offset (end). The fully automatic system presented here detects 20 facial points in the first frame and tracks them throughout the video. From the tracked points we compute geometry-based features which serve as the input to the remainder of our systems. The AU activation detection system uses GentleBoost feature selection and a Support Vector Machine (SVM) classifier to find which AUs were present in an expression. Temporal dynamics of active AUs are recognised by a hybrid GentleBoost-SVM-Hidden Markov model classifier. The system is capable of analysing 23 out of 27 existing AUs with high accuracy. The main contributions of the work presented in this thesis are the following: we have created a method for fully automatic AU analysis with state-of-the-art recognition results. We have proposed for the first time a method for recognition of the four temporal phases of an AU. We have build the largest comprehensive database of facial expressions to date. We also present for the first time in the literature two studies for automatic distinction between posed and spontaneous expressions

Automatic Facial Expression Recognition and Operator Functional State

The prevalence of human error in safety-critical occupations remains a major challenge to mission success despite increasing automation in control processes. Although various methods have been proposed to prevent incidences of human error, none of these have been developed to employ the detection and regulation of Operator Functional State (OFS), or the optimal condition of the operator while performing a task, in work environments due to drawbacks such as obtrusiveness and impracticality. A video-based system with the ability to infer an individual's emotional state from facial feature patterning mitigates some of the problems associated with other methods of detecting OFS, like obtrusiveness and impracticality in integration with the mission environment. This paper explores the utility of facial expression recognition as a technology for inferring OFS by first expounding on the intricacies of OFS and the scientific background behind emotion and its relationship with an individual's state. Then, descriptions of the feedback loop and the emotion protocols proposed for the facial recognition program are explained. A basic version of the facial expression recognition program uses Haar classifiers and OpenCV libraries to automatically locate key facial landmarks during a live video stream. Various methods of creating facial expression recognition software are reviewed to guide future extensions of the program. The paper concludes with an examination of the steps necessary in the research of emotion and recommendations for the creation of an automatic facial expression recognition program for use in real-time, safety-critical missions