37,615 research outputs found

    Coding, Analysis, Interpretation, and Recognition of Facial Expressions

    Get PDF
    We describe a computer vision system for observing facial motion by using an optimal estimation optical flow method coupled with a geometric and a physical (muscle) model describing the facial structure. Our method produces a reliable parametric representation of the face's independent muscle action groups, as well as an accurate estimate of facial motion. Previous efforts at analysis of facial expression have been based on the Facial Action Coding System (FACS), a representation developed in order to allow human psychologists to code expression from static pictures. To avoid use of this heuristic coding scheme, we have used our computer vision system to probabilistically characterize facial motion and muscle activation in an experimental population, thus deriving a new, more accurate representation of human facial expressions that we call FACS+. We use this new representation for recognition in two different ways. The first method uses the physics-based model directly, by recognizing..

    Subspace Representations for Robust Face and Facial Expression Recognition

    Get PDF
    Analyzing human faces and modeling their variations have always been of interest to the computer vision community. Face analysis based on 2D intensity images is a challenging problem, complicated by variations in pose, lighting, blur, and non-rigid facial deformations due to facial expressions. Among the different sources of variation, facial expressions are of interest as important channels of non-verbal communication. Facial expression analysis is also affected by changes in view-point and inter-subject variations in performing different expressions. This dissertation makes an attempt to address some of the challenges involved in developing robust algorithms for face and facial expression recognition by exploiting the idea of proper subspace representations for data. Variations in the visual appearance of an object mostly arise due to changes in illumination and pose. So we first present a video-based sequential algorithm for estimating the face albedo as an illumination-insensitive signature for face recognition. We show that by knowing/estimating the pose of the face at each frame of a sequence, the albedo can be efficiently estimated using a Kalman filter. Then we extend this to the case of unknown pose by simultaneously tracking the pose as well as updating the albedo through an efficient Bayesian inference method performed using a Rao-Blackwellized particle filter. Since understanding the effects of blur, especially motion blur, is an important problem in unconstrained visual analysis, we then propose a blur-robust recognition algorithm for faces with spatially varying blur. We model a blurred face as a weighted average of geometrically transformed instances of its clean face. We then build a matrix, for each gallery face, whose column space spans the space of all the motion blurred images obtained from the clean face. This matrix representation is then used to define a proper objective function and perform blur-robust face recognition. To develop robust and generalizable models for expression analysis one needs to break the dependence of the models on the choice of the coordinate frame of the camera. To this end, we build models for expressions on the affine shape-space (Grassmann manifold), as an approximation to the projective shape-space, by using a Riemannian interpretation of deformations that facial expressions cause on different parts of the face. This representation enables us to perform various expression analysis and recognition algorithms without the need for pose normalization as a preprocessing step. There is a large degree of inter-subject variations in performing various expressions. This poses an important challenge on developing robust facial expression recognition algorithms. To address this challenge, we propose a dictionary-based approach for facial expression analysis by decomposing expressions in terms of action units (AUs). First, we construct an AU-dictionary using domain experts' knowledge of AUs. To incorporate the high-level knowledge regarding expression decomposition and AUs, we then perform structure-preserving sparse coding by imposing two layers of grouping over AU-dictionary atoms as well as over the test image matrix columns. We use the computed sparse code matrix for each expressive face to perform expression decomposition and recognition. Most of the existing methods for the recognition of faces and expressions consider either the expression-invariant face recognition problem or the identity-independent facial expression recognition problem. We propose joint face and facial expression recognition using a dictionary-based component separation algorithm (DCS). In this approach, the given expressive face is viewed as a superposition of a neutral face component with a facial expression component, which is sparse with respect to the whole image. This assumption leads to a dictionary-based component separation algorithm, which benefits from the idea of sparsity and morphological diversity. The DCS algorithm uses the data-driven dictionaries to decompose an expressive test face into its constituent components. The sparse codes we obtain as a result of this decomposition are then used for joint face and expression recognition

    Machine Analysis of Facial Expressions

    Get PDF
    No abstract

    Cultural differences in the decoding and representation of facial expression signals

    Get PDF
    Summary. In this thesis, I will challenge one of the most fundamental assumptions of psychological science – the universality of facial expressions. I will do so by first reviewing the literature to reveal major flaws in the supporting arguments for universality. I will then present new data demonstrating how culture has shaped the decoding and transmission of facial expression signals. A summary of both sections are presented below. Review of the Literature To obtain a clear understanding of how the universality hypothesis developed, I will present the historical course of the emotion literature, reviewing relevant works supporting notions of a ‘universal language of emotion.’ Specifically, I will examine work on the recognition of facial expressions across cultures as it constitutes a main component of the evidence for universality. First, I will reveal that a number of ‘seminal’ works supporting the universality hypothesis are critically flawed, precluding them from further consideration. Secondly, by questioning the validity of the statistical criteria used to demonstrate ‘universal recognition,’ I will show that long-standing claims of universality are both misleading and unsubstantiated. On a related note, I will detail the creation of the ‘universal’ facial expression stimulus set (Facial Action Coding System -FACS- coded facial expressions) to reveal that it is in fact a biased, culture-specific representation of Western facial expressions of emotion. The implications for future cross-cultural work are discussed in relation to the limited FACS-coded stimulus set. Experimental Work In reviewing the literature, I will reveal a latent phenomenon which has so far remained unexplained – the East Asian (EA) recognition deficit. Specifically, EA observers consistently perform significantly poorer when categorising certain ‘universal’ facial expressions compared to Western Caucasian (WC) observers – a surprisingly neglected finding given the importance of emotion communication for human social interaction. To address this neglected issue, I examined both the decoding and transmission of facial expression signals in WC and EA observers. Experiment 1: Cultural Decoding of ‘Universal’ Facial Expressions of Emotion To examine the decoding of ‘universal’ facial expressions across cultures, I used eye tracking technology to record the eye movements of WC and EA observers while they categorised the 6 ‘universal’ facial expressions of emotion. My behavioural results demonstrate the robustness of the phenomenon by replicating the EA recognition deficit (i.e., EA observers are significantly poorer at recognizing facial expressions of ‘fear’ and ‘disgust’). Further inspection of the data also showed that EA observers systematically miscategorise ‘fear’ as ‘surprise’ and ‘disgust’ as ‘anger.’ Using spatio-temporal analyses of fixations, I will show that WC and EA observers use culture-specific fixation strategies to decode ‘universal’ facial expressions of emotion. Specifically, while WC observers distribute fixations across the face, sampling the eyes and mouth, EA observers persistently bias fixations towards the eyes and neglect critical features, especially for facial expressions eliciting significant confusion (i.e., ‘fear,’ ‘disgust,’ and ‘anger’). My behavioural data showed that EA observers systematically miscategorise ‘fear’ as ‘surprise’ and ‘disgust’ as ‘anger.’ Analysis of my eye movement data also showed that EA observers repetitively sample information from the eye region during facial expression decoding, particularly for those eliciting significant behavioural confusions (i.e., ‘fear,’ ‘disgust,’ and ‘anger’). To objectively examine whether the EA culture-specific fixation pattern could give rise to the reported behavioural confusions, I built a model observer that samples information from the face to categorise facial expressions. Using this model observer, I will show that the EA decoding strategy is inadequate to distinguish ‘fear’ from ‘surprise’ and ‘disgust’ from ‘anger,’ thus giving rise to the reported EA behavioural confusions. For the first time, I will reveal the origins of a latent phenomenon - the EA recognition deficit. I discuss the implications of culture-specific decoding strategies during facial expression categorization in light of current theories of cross-cultural emotion communication. Experiment 2: Cultural Internal Representations of Facial Expressions of Emotion In the previous two experiments, I presented data that questions the universality of facial expressions. As replicated in Experiment 1, WC and EA observers differ significantly in their recognition performance for certain ‘universal’ facial expressions. In Experiment 1, I showed culture-specific fixation patterns, demonstrating cultural differences in the predicted locations of diagnostic information. Together, these data predict cultural specificity in facial expression signals, supporting notions of cultural ‘accents’ and/or ‘dialects.’ To examine whether facial expression signals differ across cultures, I used a powerful reverse correlation (RC) technique to reveal the internal representations of the 6 ‘basic’ facial expressions of emotion in WC and EA observers. Using complementary statistical image processing techniques to examine the signal properties of each internal representation, I will directly reveal cultural specificity in the representations of the 6 ‘basic’ facial expressions of emotion. Specifically, I will show that while WC representations of facial expressions predominantly featured the eyebrows and mouth, EA representations were biased towards the eyes, as predicted by my eye movement data in Experiment 1. I will also show gaze avoidance as unique feature of the EA group. In sum, this data shows clear cultural contrasts in facial expression signals by showing that culture shapes the internal representations of emotion. Future Work My review of the literature will show that pivotal concepts such as ‘recognition’ and ‘universality’ are currently flawed and have misled both the interpretation of empirical work the direction of theoretical developments. Here, I will examine each concept in turn and propose more accurate criteria with which to demonstrate ‘universal recognition’ in future studies. In doing so, I will also detail possible future studies designed to address current gaps in knowledge created by use of inappropriate criteria. On a related note, having questioned the validity of FACS-coded facial expressions as ‘universal’ facial expressions, I will highlight an area for empirical development – the creation of a culturally valid facial expression stimulus set – and detail future work required to address this question. Finally, I will discuss broader areas of interest (i.e., lexical structure of emotion) which could elevate current knowledge of cross-cultural facial expression recognition and emotion communication in the future

    Facial Expression Recognition

    Get PDF

    Cultural dialects of real and synthetic emotional facial expressions

    Get PDF
    In this article we discuss the aspects of designing facial expressions for virtual humans (VHs) with a specific culture. First we explore the notion of cultures and its relevance for applications with a VH. Then we give a general scheme of designing emotional facial expressions, and identify the stages where a human is involved, either as a real person with some specific role, or as a VH displaying facial expressions. We discuss how the display and the emotional meaning of facial expressions may be measured in objective ways, and how the culture of displayers and the judges may influence the process of analyzing human facial expressions and evaluating synthesized ones. We review psychological experiments on cross-cultural perception of emotional facial expressions. By identifying the culturally critical issues of data collection and interpretation with both real and VHs, we aim at providing a methodological reference and inspiration for further research

    A survey on mouth modeling and analysis for Sign Language recognition

    Get PDF
    © 2015 IEEE.Around 70 million Deaf worldwide use Sign Languages (SLs) as their native languages. At the same time, they have limited reading/writing skills in the spoken language. This puts them at a severe disadvantage in many contexts, including education, work, usage of computers and the Internet. Automatic Sign Language Recognition (ASLR) can support the Deaf in many ways, e.g. by enabling the development of systems for Human-Computer Interaction in SL and translation between sign and spoken language. Research in ASLR usually revolves around automatic understanding of manual signs. Recently, ASLR research community has started to appreciate the importance of non-manuals, since they are related to the lexical meaning of a sign, the syntax and the prosody. Nonmanuals include body and head pose, movement of the eyebrows and the eyes, as well as blinks and squints. Arguably, the mouth is one of the most involved parts of the face in non-manuals. Mouth actions related to ASLR can be either mouthings, i.e. visual syllables with the mouth while signing, or non-verbal mouth gestures. Both are very important in ASLR. In this paper, we present the first survey on mouth non-manuals in ASLR. We start by showing why mouth motion is important in SL and the relevant techniques that exist within ASLR. Since limited research has been conducted regarding automatic analysis of mouth motion in the context of ALSR, we proceed by surveying relevant techniques from the areas of automatic mouth expression and visual speech recognition which can be applied to the task. Finally, we conclude by presenting the challenges and potentials of automatic analysis of mouth motion in the context of ASLR
    corecore