1,530 research outputs found
Timing is everything: A spatio-temporal approach to the analysis of facial actions
This thesis presents a fully automatic facial expression analysis system based on the Facial Action
Coding System (FACS). FACS is the best known and the most commonly used system to describe
facial activity in terms of facial muscle actions (i.e., action units, AUs). We will present our research
on the analysis of the morphological, spatio-temporal and behavioural aspects of facial expressions.
In contrast with most other researchers in the field who use appearance based techniques, we use a
geometric feature based approach. We will argue that that approach is more suitable for analysing
facial expression temporal dynamics. Our system is capable of explicitly exploring the temporal
aspects of facial expressions from an input colour video in terms of their onset (start), apex (peak)
and offset (end).
The fully automatic system presented here detects 20 facial points in the first frame and tracks them
throughout the video. From the tracked points we compute geometry-based features which serve as
the input to the remainder of our systems. The AU activation detection system uses GentleBoost
feature selection and a Support Vector Machine (SVM) classifier to find which AUs were present in an
expression. Temporal dynamics of active AUs are recognised by a hybrid GentleBoost-SVM-Hidden
Markov model classifier. The system is capable of analysing 23 out of 27 existing AUs with high
accuracy.
The main contributions of the work presented in this thesis are the following: we have created a
method for fully automatic AU analysis with state-of-the-art recognition results. We have proposed
for the first time a method for recognition of the four temporal phases of an AU. We have build the
largest comprehensive database of facial expressions to date. We also present for the first time in the
literature two studies for automatic distinction between posed and spontaneous expressions
Spotting Agreement and Disagreement: A Survey of Nonverbal Audiovisual Cues and Tools
While detecting and interpreting temporal patterns of non–verbal behavioral cues in a given context is a natural and often unconscious process for humans, it remains a rather difficult task for computer systems. Nevertheless, it is an important one to achieve if the goal is to realise a naturalistic communication between humans and machines. Machines that are able to sense social attitudes like agreement and disagreement and respond to them in a meaningful way are likely to be welcomed by users due to the more natural, efficient and human–centered interaction they are bound to experience. This paper surveys the nonverbal cues that could be present during agreement and disagreement behavioural displays and lists a number of tools that could be useful in detecting them, as well as a few publicly available databases that could be used to train these tools for analysis of spontaneous, audiovisual instances of agreement and disagreement
Machine Analysis of Facial Expressions
No abstract
Less is More: Facial Landmarks can Recognize a Spontaneous Smile
Smile veracity classification is a task of interpreting social interactions.
Broadly, it distinguishes between spontaneous and posed smiles. Previous
approaches used hand-engineered features from facial landmarks or considered
raw smile videos in an end-to-end manner to perform smile classification tasks.
Feature-based methods require intervention from human experts on feature
engineering and heavy pre-processing steps. On the contrary, raw smile video
inputs fed into end-to-end models bring more automation to the process with the
cost of considering many redundant facial features (beyond landmark locations)
that are mainly irrelevant to smile veracity classification. It remains unclear
to establish discriminative features from landmarks in an end-to-end manner. We
present a MeshSmileNet framework, a transformer architecture, to address the
above limitations. To eliminate redundant facial features, our landmarks input
is extracted from Attention Mesh, a pre-trained landmark detector. Again, to
discover discriminative features, we consider the relativity and trajectory of
the landmarks. For the relativity, we aggregate facial landmark that
conceptually formats a curve at each frame to establish local spatial features.
For the trajectory, we estimate the movements of landmark composed features
across time by self-attention mechanism, which captures pairwise dependency on
the trajectory of the same landmark. This idea allows us to achieve
state-of-the-art performances on UVA-NEMO, BBC, MMI Facial Expression, and SPOS
datasets
Machine Understanding of Human Behavior
A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior
A study of the temporal relationship between eye actions and facial expressions
A dissertation submitted in ful llment of the requirements for the
degree of Master of Science
in the
School of Computer Science and Applied Mathematics
Faculty of Science
August 15, 2017Facial expression recognition is one of the most common means of communication used
for complementing spoken word. However, people have grown to master ways of ex-
hibiting deceptive expressions. Hence, it is imperative to understand di erences in
expressions mostly for security purposes among others. Traditional methods employ
machine learning techniques in di erentiating real and fake expressions. However, this
approach does not always work as human subjects can easily mimic real expressions with
a bit of practice. This study presents an approach that evaluates the time related dis-
tance that exists between eye actions and an exhibited expression. The approach gives
insights on some of the most fundamental characteristics of expressions. The study fo-
cuses on nding and understanding the temporal relationship that exists between eye
blinks and smiles. It further looks at the relationship that exits between eye closure and
pain expressions. The study incorporates active appearance models (AAM) for feature
extraction and support vector machines (SVM) for classi cation. It tests extreme learn-
ing machines (ELM) in both smile and pain studies, which in turn, attains excellent
results than predominant algorithms like the SVM. The study shows that eye blinks
are highly correlated with the beginning of a smile in posed smiles while eye blinks are
highly correlated with the end of a smile in spontaneous smiles. A high correlation is
observed between eye closure and pain in spontaneous pain expressions. Furthermore,
this study brings about ideas that lead to potential applications such as lie detection
systems, robust health care monitoring systems and enhanced animation design systems
among others.MT 201
- …