LEARNet Dynamic Imaging Network for Micro Expression Recognition

Unlike prevalent facial expressions, micro expressions have subtle, involuntary muscle movements which are short-lived in nature. These minute muscle movements reflect true emotions of a person. Due to the short duration and low intensity, these micro-expressions are very difficult to perceive and interpret correctly. In this paper, we propose the dynamic representation of micro-expressions to preserve facial movement information of a video in a single frame. We also propose a Lateral Accretive Hybrid Network (LEARNet) to capture micro-level features of an expression in the facial region. The LEARNet refines the salient expression features in accretive manner by incorporating accretion layers (AL) in the network. The response of the AL holds the hybrid feature maps generated by prior laterally connected convolution layers. Moreover, LEARNet architecture incorporates the cross decoupled relationship between convolution layers which helps in preserving the tiny but influential facial muscle change information. The visual responses of the proposed LEARNet depict the effectiveness of the system by preserving both high- and micro-level edge features of facial expression. The effectiveness of the proposed LEARNet is evaluated on four benchmark datasets: CASME-I, CASME-II, CAS(ME)^2 and SMIC. The experimental results after investigation show a significant improvement of 4.03%, 1.90%, 1.79% and 2.82% as compared with ResNet on CASME-I, CASME-II, CAS(ME)^2 and SMIC datasets respectively.Comment: Dynamic imaging, accretion, lateral, micro expression recognitio

Spontaneous Subtle Expression Detection and Recognition based on Facial Strain

Optical strain is an extension of optical flow that is capable of quantifying subtle changes on faces and representing the minute facial motion intensities at the pixel level. This is computationally essential for the relatively new field of spontaneous micro-expression, where subtle expressions can be technically challenging to pinpoint. In this paper, we present a novel method for detecting and recognizing micro-expressions by utilizing facial optical strain magnitudes to construct optical strain features and optical strain weighted features. The two sets of features are then concatenated to form the resultant feature histogram. Experiments were performed on the CASME II and SMIC databases. We demonstrate on both databases, the usefulness of optical strain information and more importantly, that our best approaches are able to outperform the original baseline results for both detection and recognition tasks. A comparison of the proposed method with other existing spatio-temporal feature extraction approaches is also presented.Comment: 21 pages (including references), single column format, accepted to Signal Processing: Image Communication journa

An innovative metaheuristic strategy for solar energy management through a neural networks framework

Proper management of solar energy as an effective renewable source is of high importance toward sustainable energy harvesting. This paper offers a novel sophisticated method for predicting solar irradiance (SIr) from environmental conditions. To this end, an efficient metaheuristic technique, namely electromagnetic field optimization (EFO), is employed for optimizing a neural network. This algorithm quickly mines a publicly available dataset for nonlinearly tuning the network parameters. To suggest an optimal configuration, five influential parameters of the EFO are optimized by an extensive trial and error practice. Analyzing the results showed that the proposed model can learn the SIr pattern and predict it for unseen conditions with high accuracy. Furthermore, it provided about 10% and 16% higher accuracy compared to two benchmark optimizers, namely shuffled complex evolution and shuffled frog leaping algorithm. Hence, the EFO-supervised neural network can be a promising tool for the early prediction of SIr in practice. The findings of this research may shed light on the use of advanced intelligent models for efficient energy development

SMEConvNet: A Convolutional Neural Network for Spotting Spontaneous Facial Micro-Expression from Long Videos

Micro-expression is a subtle and involuntary facial expression that may reveal the hidden emotion of human beings. Spotting micro-expression means to locate the moment when the microexpression happens, which is a primary step for micro-expression recognition. Previous work in microexpression expression spotting focus on spotting micro-expression from short video, and with hand-crafted features. In this paper, we present a methodology for spotting micro-expression from long videos. Specifically, a new convolutional neural network named as SMEConvNet (Spotting Micro-Expression Convolutional Network) was designed for extracting features from video clips, which is the first time that deep learning is used in micro-expression spotting. Then a feature matrix processing method was proposed for spotting the apex frame from long video, which uses a sliding window and takes the characteristics of micro-expression into account to search the apex frame. Experimental results demonstrate that the proposed method can achieve better performance than existing state-of-art methods

Face Recognition and Micro-expression Recognition Based on Discriminant Tensor Subspace Analysis Plus Extreme Learning Machine

In this paper, a novel recognition algorithm based on discriminant tensor subspace analysis (DTSA) and extreme learning machine (ELM) is introduced. DTSA treats a gray facial image as a second order tensor and adopts two-sided transformations to reduce dimensionality. One of the many advantages of DTSA is its ability to preserve the spatial structure information of the images. In order to deal with micro-expression video clips, we extend DTSA to a high-order tensor. Discriminative features are generated using DTSA to further enhance the classification performance of ELM classifier. Another notable contribution of the proposed method includes significant improvements in face and micro-expression recognition accuracy. The experimental results on the ORL, Yale, YaleB facial databases and CASME micro-expression database show the effectiveness of the proposed method

Reconocimiento de expresiones con Kinect

Los progresos recientes en análisis automáticos de rasgos faciales y comportamentales han abierto una puerta a nuevas aplicaciones para la identificación de conductas no verbales. Por ejemplo, en el área de la salud, los algoritmos de visión por ordenador pueden ayudar al personal sanitario a mejorar la comunicación a través de la telemedicina. Los descriptores automáticos del comportamiento pueden además añadir información cuantitativa a las interacciones profesional - paciente. Otro campo de aplicación es el de la psicología, donde una aplicación relevante para este tipo de tecnología no verbal es el desarrollo de descriptores robustos de comportamiento que se correlacionan con trastornos psicológicos como la depresión, ansiedad, trastornos de estrés postraumático o de hiperactividad, etc. De esta forma, el objetivo de este proyecto fin de carrera ha sido el de detectar variaciones faciales que se presentan en la cara de una persona ante diferentes estímulos. En este caso, los estímulos han consistido en vídeos que se han mostrado a las personas que se sometieron al estudio piloto que se llevó a cabo en la Fundación Jiménez Díaz. Estos estímulos son recogidos a través del nuevo sensor Kinect, disponible en el mercado desde mediados del 2014, y que permite obtener un análisis detallado de los movimientos faciales. Una colección de 3 vídeos de corta duración fue mostrada a los participantes, cada uno de ellos con un objetivo en concreto. Los resultados del estudio serán analizados en términos de FACS, considerado como uno de los métodos más eficaces para medir los comportamientos faciales. Este sistema divide las expresiones faciales en unidades de acción (UAs) en vez de hacer una clasificación en unas cuantas emociones básicasIngeniería de Telecomunicació

3D FACE RECOGNITION USING LOCAL FEATURE BASED METHODS

Face recognition has attracted many researchers’ attention compared to other biometrics due to its non-intrusive and friendly nature. Although several methods for 2D face recognition have been proposed so far, there are still some challenges related to the 2D face including illumination, pose variation, and facial expression. In the last few decades, 3D face research area has become more interesting since shape and geometry information are used to handle challenges from 2D faces. Existing algorithms for face recognition are divided into three different categories: holistic feature-based, local feature-based, and hybrid methods. According to the literature, local features have shown better performance relative to holistic feature-based methods under expression and occlusion challenges. In this dissertation, local feature-based methods for 3D face recognition have been studied and surveyed. In the survey, local methods are classified into three broad categories which consist of keypoint-based, curve-based, and local surface-based methods. Inspired by keypoint-based methods which are effective to handle partial occlusion, structural context descriptor on pyramidal shape maps and texture image has been proposed in a multimodal scheme. Score-level fusion is used to combine keypoints’ matching score in both texture and shape modalities. The survey shows local surface-based methods are efficient to handle facial expression. Accordingly, a local derivative pattern is introduced to extract distinct features from depth map in this work. In addition, the local derivative pattern is applied on surface normals. Most 3D face recognition algorithms are focused to utilize the depth information to detect and extract features. Compared to depth maps, surface normals of each point can determine the facial surface orientation, which provides an efficient facial surface representation to extract distinct features for recognition task. An Extreme Learning Machine (ELM)-based auto-encoder is used to make the feature space more discriminative. Expression and occlusion robust analysis using the information from the normal maps are investigated by dividing the facial region into patches. A novel hybrid classifier is proposed to combine Sparse Representation Classifier (SRC) and ELM classifier in a weighted scheme. The proposed algorithms have been evaluated on four widely used 3D face databases; FRGC, Bosphorus, Bu-3DFE, and 3D-TEC. The experimental results illustrate the effectiveness of the proposed approaches. The main contribution of this work lies in identification and analysis of effective local features and a classification method for improving 3D face recognition performance