Automatic Recognition of Facial Displays of Unfelt Emotions

Humans modify their facial expressions in order to communicate their internal states and sometimes to mislead observers regarding their true emotional states. Evidence in experimental psychology shows that discriminative facial responses are short and subtle. This suggests that such behavior would be easier to distinguish when captured in high resolution at an increased frame rate. We are proposing SASE-FE, the first dataset of facial expressions that are either congruent or incongruent with underlying emotion states. We show that overall the problem of recognizing whether facial movements are expressions of authentic emotions or not can be successfully addressed by learning spatio-temporal representations of the data. For this purpose, we propose a method that aggregates features along fiducial trajectories in a deeply learnt space. Performance of the proposed model shows that on average it is easier to distinguish among genuine facial expressions of emotion than among unfelt facial expressions of emotion and that certain emotion pairs such as contempt and disgust are more difficult to distinguish than the rest. Furthermore, the proposed methodology improves state of the art results on CK+ and OULU-CASIA datasets for video emotion recognition, and achieves competitive results when classifying facial action units on BP4D datas

Automatic Recognition of Facial Displays of Unfelt Emotions

Humans modify their facial expressions in order to communicate their internal states and sometimes to mislead observers regarding their true emotional states. Evidence in experimental psychology shows that discriminative facial responses are short and subtle. This suggests that such behavior would be easier to distinguish when captured in high resolution at an increased frame rate. We are proposing SASE-FE, the first dataset of facial expressions that are either congruent or incongruent with underlying emotion states. We show that overall the problem of recognizing whether facial movements are expressions of authentic emotions or not can be successfully addressed by learning spatio-temporal representations of the data. For this purpose, we propose a method that aggregates features along fiducial trajectories in a deeply learnt space. Performance of the proposed model shows that on average, it is easier to distinguish among genuine facial expressions of emotion than among unfelt facial expressions of emotion and that certain emotion pairs such as contempt and disgust are more difficult to distinguish than the rest. Furthermore, the proposed methodology improves state of the art results on CK+ and OULU-CASIA datasets for video emotion recognition, and achieves competitive results when classifying facial action units on BP4D datase

Subjective and objective measures

One of the greatest challenges in the study of emotions and emotional states is their measurement. The techniques used to measure emotions depend essentially on the authors’ definition of the concept of emotion. Currently, two types of measures are used: subjective and objective. While subjective measures focus on assessing the conscious recognition of one’s own emotions, objective measures allow researchers to quantify and assess the conscious and unconscious emotional processes. In this sense, when the objective is to evaluate the emotional experience from the subjective point of view of an individual in relation to a given event, then subjective measures such as self-report should be used. In addition to this, when the objective is to evaluate the emotional experience at the most unconscious level of processes such as the physiological response, objective measures should be used. There are no better or worse measures, only measures that allow access to the same phenomenon from different points of view. The chapter’s main objective is to make a survey of the main measures of evaluation of the emotions and emotional states more relevant in the current scientific panorama.info:eu-repo/semantics/acceptedVersio

Facial-Expression Affective Attributes and their Configural Correlates: Components and Categories

The present study investigates the perception of facial expressions of emotion, and explores the relation between the configural properties of expressions and their subjective attribution. Stimuli were a male and a female series of morphed facial expressions, interpolated between prototypes of seven emotions (happiness, sadness, fear, anger, surprise and disgust, and neutral) from Ekman and Friesen (1976). Topographical properties of the stimuli were quantified using the Facial Expression Measurement (FACEM) scheme. Perceived dissimilarities between the emotional expressions were elicited using a sorting procedure and processed with multidimensional scaling. Four dimensions were retained in the reconstructed facial-expression space, with positive and negative expressions opposed along D1, while the other three dimensions were interpreted as affective attributes distinguishing clusters of expressions categorized as “Surprise-Fear,” “Anger,” and “Disgust.” Significant relationships were found between these affective attributes and objective facial measures of the stimuli. The findings support a componential explanatory scheme for expression processing, wherein each component of a facial stimulus conveys an affective value separable from its context, rather than a categorical-gestalt scheme. The findings further suggest that configural information is closely involved in the decoding of affective attributes of facial expressions. Configural measures are also suggested as a common ground for dimensional as well as categorical perception of emotional faces.Este estudio investiga la percepción de las expresiones faciales de la emoción y explora la relación entre las propiedades configurales de las expresiones y su atribución subjetiva. Los estímulos eran una serie de expresiones faciales transformadas por ordenador, interpuestas entre los prototipos de siete emociones (felicidad, tristeza, miedo, ira, sorpresa, asco y neutral) tomados de Ekman y Friesen (1976). Las propiedades topográficas de los estímulos se cuantificaron mediante el esquema Facial Expression Measurement (FACEM). Las disimilaridades percibidas entre las expresiones emocionales se elicitaron mediante un procedimiento de clasificación y se procesaron con escalonamiento multidimensional. Se retuvieron cuatro dimensiones en el espacio facial-expresión reconstruido, con expresiones positivas y negativas contrapuestas a lo largo de D1, y las restantes tres dimensiones se interpretaron como atributos afectivos, distinguiendo clusters de expresiones clasificadas como “Sorpresa/Miedo”, “Ira”, y “Asco”. Se hallaron relaciones significativas entre estos atributos afectivos y las medidas faciales objetivas de los estímulos. Los resultados apoyan un esquema explicativo componencial para el procesamiento de las expresiones, en el que cada componente de un estímulo facial conlleva un valor afectivo separable de su contexto, más que un esquema categórico de tipo Gestalt. Además sugieren que la información configural juega un papel importante en la decodificación de los atributos afectivos de las expresiones faciales Además, sugieren que las medidas configurales constituyen en terreno común de la percepción dimensional y categórica de las caras emocionales

Music Playlist Generation using Facial Expression Analysis and Task Extraction

In day to day stressful environment of IT Industry, there is a truancy for the appropriate relaxation time for all working professionals. To keep a person stress free, various technical or non-technical stress releasing methods are now being adopted. We can categorize the people working on computers as administrators, programmers, etc. each of whom require varied ways in order to ease themselves. The work pressure and the vexation of any kind for a person can be depicted by their emotions. Facial expressions are the key to analyze the current psychology of the person. In this paper, we discuss a user intuitive smart music player. This player will capture the facial expressions of a person working on the computer and identify the current emotion. Intuitively the music will be played for the user to relax them. The music player will take into account the foreground processes which the person is executing on the computer. Since various sort of music is available to boost one's enthusiasm, taking into consideration the tasks executed on the system by the user and the current emotions they carry, an ideal playlist of songs will be created and played for the person. The person can browse the playlist and modify it to make the system more flexible. This music player will thus allow the working professionals to stay relaxed in spite of their workloads

Feature Tracking and Expression Recognition of Face Using Dynamic Bayesian Network

Abstract- The human face plays a central role in social interaction, hence it is not surprising that facial information processing is an important and highly active subfield of cognitive science research. The face is a complex stimulus displaying information about identity, age, gender, as well as emotional and attention state. Here we consider the problem of extracting information about emotional state (facial expression) from single images. Due to the difficulty of obtaining controlled video sequences of standard facial expressions, many psychological and neurophysiologic studies of facial expression processing have used single image motivations. In proposed system, in contrast to the mainstream approaches, we are trying to build a probabilistic model based on the Dynamic Bayesian Network (DBN) to capture the facial interactions at different levels. Hence the proposed system deal with the identification of facial expression on the image captured through camera

Sistema de reconhecimento de expressões faciais para deteção de stress

Stress is the body's natural reaction to external and internal stimuli. Despite being something natural, prolonged exposure to stressors can contribute to serious health problems. These reactions are reflected not only physiologically, but also psychologically, translating into emotions and facial expressions. Once this relationship between the experience of stressful situations and the demonstration of certain emotions in response was understood, it was decided to develop a system capable of classifying facial expressions and thereby creating a stress detector. The proposed solution consists of two main blocks. A convolutional neural network capable of classifying facial expressions, and an application that uses this model to classify real-time images of the user's face and thereby verify whether or not it shows signs of stress. The application consists in capturing real-time images from the webcam, extract the user's face, classify which facial expression he expresses, and with these classifications assess whether or not he shows signs of stress in a given time interval. As soon as the application determines the presence of signs of stress, it notifies the user. For the creation of the classification model, was used transfer learning, together with finetuning. In this way, we took advantage of the pre-trained networks VGG16, VGG19, and Inception-ResNet V2 to solve the problem at hand. For the transfer learning process, were also tried two classifier architectures. After several experiments, it was determined that VGG16, together with a classifier made up of a convolutional layer, was the candidate with the best performance at classifying stressful emotions. Having presented an MCC of 0.8969 in the test images of the KDEF dataset, 0.5551 in the Net Images dataset, and 0.4250 in the CK +.O stress é uma reação natural do corpo a estímulos externos e internos. Apesar de ser algo natural, a exposição prolongada a stressors pode contribuir para sérios problemas de saúde. Essas reações refletem-se não só fisiologicamente, mas também psicologicamente. Traduzindose em emoções e expressões faciais. Uma vez compreendida esta relação entre a experiência de situações stressantes e a demonstração de determinadas emoções como resposta, decidiu-se desenvolver um sistema capaz de classificar expressões faciais e com isso criar um detetor de stress. A solução proposta é constituida por dois blocos fundamentais. Uma rede neuronal convolucional capaz de classificar expressões faciais e uma aplicação que utiliza esse modelo para classificar imagens em tempo real do rosto do utilizador e assim averiguar se este apresenta ou não sinais de stress. A aplicação consiste em captar imagens em tempo real a partir da webcam, extrair o rosto do utilizador, classificar qual a expressão facial que este manifesta, e com essas classificações avaliar se num determinado intervalo temporal este apresenta ou não sinais de stress. Assim que a aplicação determine a presença de sinais de stress, esta irá notificar o utilizador. Para a criação do modelo de classificação, foi utilizado transfer learning, juntamente com finetuning. Desta forma tirou-se partido das redes pre-treinadas VGG16, VGG19, e InceptionResNet V2 para a resolução do problema em mãos. Para o processo de transfer learning foram também experimentadas duas arquiteturas de classificadores. Após várias experiências, determinou-se que a VGG16, juntamente com um classificador constituido por uma camada convolucional era a candidata com melhor desempenho a classificar emoções stressantes. Tendo apresentado um MCC de 0,8969 nas imagens de teste do conjunto de dados KDEF, 0,5551 no conjunto de dados Net Images, e 0,4250 no CK+

Nonverbal communication: Micro expressions

Pojam mikroekspresija naglo je u javnosti postao popularan zahvaljujući američkoj televizijskoj seriji Laži mi (Lie to me) koja se prikazivala od 2009. do 2011. godine, a snimljena je prema teoriji i naučavanju Paula Ekmana, jednog od ključnih znanstvenika koji se bave tom temom. Sâm Ekman redovito je davao svoj osvrt na emitirane epizode te sa stručnog stajališta objasnio prikazane analize nečijih emocija i ukazao na eventualne nedostatke u postupcima. To je mikroekspresije približilo široj publici, ali u određenim krugovima njihovo postojanje je zadobilo pažnju puno prije emitiranja serije. Mikroekspresije su brze nenamjerne ekspresije lica koje otkrivaju emocije koje osoba pokušava prikriti. Većina ljudi ih ne uočava, ali moguće ih je naučiti zamjećivati. Zanimljive su iz teoretskog aspekta, no većina današnjih istraživanja ide u smjeru unaprjeđivanja praktične primjene znanja o njima. Znanje je široko primjenjivo – od sigurnosnih službi do svakodnevnih privatnih odnosa, a korist je značajna. Ekman je svoja istraživanja uspješno popularizirao, pa tako nudi i nekoliko vrsta tečajeva prepoznavanja mikroekspresija koji uključuju video materijale i skripta za pomoć u učenju i primjeni znanja o dešifriranju tuđih emocija, ali mogu služiti i za pomoć u prenošenju vlastitih. Ekman smatra da je analiza mikroekspresija metoda koja puno obećava kada je u pitanju otkrivanje prijevare. S dobrim uvježbavanjem „lovaca“, kako ih Ekman obično naziva, postiže se velika uspješnost pri detektiranju laži, od preko 80 posto u prvih sat vremena uvježbavanja (2009a: 351). U ovom ćemo se radu baviti različitim tipovima neverbalne komunikacije te teorijama na kojima su one zasnovane