This thesis presents a fully automatic facial expression analysis system based on the Facial Action
Coding System (FACS). FACS is the best known and the most commonly used system to describe
facial activity in terms of facial muscle actions (i.e., action units, AUs). We will present our research
on the analysis of the morphological, spatio-temporal and behavioural aspects of facial expressions.
In contrast with most other researchers in the field who use appearance based techniques, we use a
geometric feature based approach. We will argue that that approach is more suitable for analysing
facial expression temporal dynamics. Our system is capable of explicitly exploring the temporal
aspects of facial expressions from an input colour video in terms of their onset (start), apex (peak)
and offset (end).
The fully automatic system presented here detects 20 facial points in the first frame and tracks them
throughout the video. From the tracked points we compute geometry-based features which serve as
the input to the remainder of our systems. The AU activation detection system uses GentleBoost
feature selection and a Support Vector Machine (SVM) classifier to find which AUs were present in an
expression. Temporal dynamics of active AUs are recognised by a hybrid GentleBoost-SVM-Hidden
Markov model classifier. The system is capable of analysing 23 out of 27 existing AUs with high
accuracy.
The main contributions of the work presented in this thesis are the following: we have created a
method for fully automatic AU analysis with state-of-the-art recognition results. We have proposed
for the first time a method for recognition of the four temporal phases of an AU. We have build the
largest comprehensive database of facial expressions to date. We also present for the first time in the
literature two studies for automatic distinction between posed and spontaneous expressions
