Distinguishing Posed and Spontaneous Smiles by Facial Dynamics

Smile is one of the key elements in identifying emotions and present state of mind of an individual. In this work, we propose a cluster of approaches to classify posed and spontaneous smiles using deep convolutional neural network (CNN) face features, local phase quantization (LPQ), dense optical flow and histogram of gradient (HOG). Eulerian Video Magnification (EVM) is used for micro-expression smile amplification along with three normalization procedures for distinguishing posed and spontaneous smiles. Although the deep CNN face model is trained with large number of face images, HOG features outperforms this model for overall face smile classification task. Using EVM to amplify micro-expressions did not have a significant impact on classification accuracy, while the normalizing facial features improved classification accuracy. Unlike many manual or semi-automatic methodologies, our approach aims to automatically classify all smiles into either `spontaneous' or `posed' categories, by using support vector machines (SVM). Experimental results on large UvA-NEMO smile database show promising results as compared to other relevant methods.Comment: 16 pages, 8 figures, ACCV 2016, Second Workshop on Spontaneous Facial Behavior Analysi

Are there nonverbal cues to commitment? An exploratory study using the zero-acquaintance video presentation paradigm

Altruism is difficult to explain evolutionarily if subtle cheaters exist in a population (Trivers, 1971). A pathway to the evolutionary maintenance of cooperation is nonverbal altruist-detection. One adaptive advantage of nonverbal altruist-detection is the formation of trustworthy division of labour partnerships (Frank, 1988). Three studies were designed to test a fundamental assumption behind altruistic partner preference models. In the first experiment perceivers (blind with respect to target altruism level) made assessments of video-clips depicting self-reported altruists and self-reported non-altruists. Video-clips were designed with attempts to control for attractiveness, expressiveness, role-playing ability, and verbal content. Overall perceivers rated altruists as more “helpful” than non-altruists. In a second experiment manipulating the payoffs for cooperation, perceivers (blind with respect to payoff condition and altruism level) assessed altruists who were helping others as more “concerned” and “attentive” than non-altruists. However perceivers assessed the same altruists as less “concerned” and “attentive” than non-altruists when the payoffs were for self. This finding suggests that perceivers are sensitive to nonverbal indicators of selfishness. Indeed the self-reported non-altruists were more likely than self-reported altruists to retain resources for themselves in an objective measure of cooperative tendencies (i.e. a dictator game). In a third study altruists and non-altruists’ facial expressions were analyzed. The smile emerged as a consistent cue to altruism. In addition, altruists exhibited more expressions that are under involuntary control (e.g., orbicularis oculi) compared to non-altruists. Findings Are there nonverbal cues to commitment? suggest that likelihood to cooperate is signaled nonverbally and the putative cues may be under involuntary control as predicted by Frank (1988)

Less is More: Facial Landmarks can Recognize a Spontaneous Smile

Smile veracity classification is a task of interpreting social interactions. Broadly, it distinguishes between spontaneous and posed smiles. Previous approaches used hand-engineered features from facial landmarks or considered raw smile videos in an end-to-end manner to perform smile classification tasks. Feature-based methods require intervention from human experts on feature engineering and heavy pre-processing steps. On the contrary, raw smile video inputs fed into end-to-end models bring more automation to the process with the cost of considering many redundant facial features (beyond landmark locations) that are mainly irrelevant to smile veracity classification. It remains unclear to establish discriminative features from landmarks in an end-to-end manner. We present a MeshSmileNet framework, a transformer architecture, to address the above limitations. To eliminate redundant facial features, our landmarks input is extracted from Attention Mesh, a pre-trained landmark detector. Again, to discover discriminative features, we consider the relativity and trajectory of the landmarks. For the relativity, we aggregate facial landmark that conceptually formats a curve at each frame to establish local spatial features. For the trajectory, we estimate the movements of landmark composed features across time by self-attention mechanism, which captures pairwise dependency on the trajectory of the same landmark. This idea allows us to achieve state-of-the-art performances on UVA-NEMO, BBC, MMI Facial Expression, and SPOS datasets

Machine Analysis of Facial Expressions

No abstract

Timing is everything: A spatio-temporal approach to the analysis of facial actions

This thesis presents a fully automatic facial expression analysis system based on the Facial Action Coding System (FACS). FACS is the best known and the most commonly used system to describe facial activity in terms of facial muscle actions (i.e., action units, AUs). We will present our research on the analysis of the morphological, spatio-temporal and behavioural aspects of facial expressions. In contrast with most other researchers in the field who use appearance based techniques, we use a geometric feature based approach. We will argue that that approach is more suitable for analysing facial expression temporal dynamics. Our system is capable of explicitly exploring the temporal aspects of facial expressions from an input colour video in terms of their onset (start), apex (peak) and offset (end). The fully automatic system presented here detects 20 facial points in the first frame and tracks them throughout the video. From the tracked points we compute geometry-based features which serve as the input to the remainder of our systems. The AU activation detection system uses GentleBoost feature selection and a Support Vector Machine (SVM) classifier to find which AUs were present in an expression. Temporal dynamics of active AUs are recognised by a hybrid GentleBoost-SVM-Hidden Markov model classifier. The system is capable of analysing 23 out of 27 existing AUs with high accuracy. The main contributions of the work presented in this thesis are the following: we have created a method for fully automatic AU analysis with state-of-the-art recognition results. We have proposed for the first time a method for recognition of the four temporal phases of an AU. We have build the largest comprehensive database of facial expressions to date. We also present for the first time in the literature two studies for automatic distinction between posed and spontaneous expressions