Objective Classes for Micro-Facial Expression Recognition

Micro-expressions are brief spontaneous facial expressions that appear on a face when a person conceals an emotion, making them different to normal facial expressions in subtlety and duration. Currently, emotion classes within the CASME II dataset are based on Action Units and self-reports, creating conflicts during machine learning training. We will show that classifying expressions using Action Units, instead of predicted emotion, removes the potential bias of human reporting. The proposed classes are tested using LBP-TOP, HOOF and HOG 3D feature descriptors. The experiments are evaluated on two benchmark FACS coded datasets: CASME II and SAMM. The best result achieves 86.35\% accuracy when classifying the proposed 5 classes on CASME II using HOG 3D, outperforming the result of the state-of-the-art 5-class emotional-based classification in CASME II. Results indicate that classification based on Action Units provides an objective method to improve micro-expression recognition.Comment: 11 pages, 4 figures and 5 tables. This paper will be submitted for journal revie

Micro-facial movement detection using spatio-temporal features

Micro-facial expressions are fast, subtle movements of facial muscles that occur when someone is attempting to conceal their true emotion. Detecting these movements for a human is di�cult, as the movement could appear and disappear within half of a second. Recently, research into detecting micro-facial movements using computer vision and other techniques has emerged with the aim of outperforming a human. The motivation behind a lot of this research is the potential applications in security, healthcare and emotional-based training. The research has also introduced some ethical concerns on whether it is okay to detect micro-movements when people do not know they are showing them. The main aim of this thesis is to investigate and develop novel ways of detecting micro-facial movements using features based in the spatial and temporal domains. The contributions towards this aim are: an extended feature descriptor to describe micro-facial movement namely Local Binary Patterns on Three Orthogonal Planes (LBP-TOP) combined with Gaussian Derivatives (GD); a dataset of spontaneously induced micro-facial movements, namely Spontaneous Activity of Micro-Movements (SAMM); an individualised baseline method for micromovement detection that forms an Adaptive Baseline Threshold (ABT); Facial Action Coding System (FACS)-based regions are proposed to focus on the local movement of relevant facial areas. The LBP-TOP with GD feature was developed to improve on an established feature and use the GD to enhance the facial features. Using machine learning, the method performs well achieving an accuracy of 92.6%. Next a new dataset, SAMM, was introduced that improved on the limitations of previous sets, including a wider demographic, increased resolution and comprehensively FACS coded. An individualised baseline method was the introduced and tested using the new dataset. Using feature di�erence instead of machine learning, the performance increased with a recall of 0.8429 on the maximum thresholding and a further increase of the recall to 0.9125 when using the ABT. To increase the relevance of what is being processed on the face, FACS-based regions were created. By focusing on local regions and individualised baselines, this method outperformed similar state-of-the-art with an Area Under Curve (AUC) of 0.7513. The research into detecting micro-movements is still in it's infancy, and much more can be done to advance this �eld. While machine learning can �nd patterns in normal facial expressions, it is the feature di�erence methods that perform the best when detecting the subtle changes of the face. By using this and comparing the movement against a person's baseline, the micro-movements can �nally be accurately detected

3D-CNN for Facial Micro- and Macro-expression Spotting on Long Video Sequences using Temporal Oriented Reference Frame

Facial expression spotting is the preliminary step for micro- and macro-expression analysis. The task of reliably spotting such expressions in video sequences is currently unsolved. The current best systems depend upon optical flow methods to extract regional motion features, before categorisation of that motion into a specific class of facial movement. Optical flow is susceptible to drift error, which introduces a serious problem for motions with long-term dependencies, such as high frame-rate macro-expression. We propose a purely deep learning solution which, rather than track frame differential motion, compares via a convolutional model, each frame with two temporally local reference frames. Reference frames are sampled according to calculated micro- and macro-expression durations. We show that our solution achieves state-of-the-art performance (F1-score of 0.126) in a dataset of high frame-rate (200 fps) long video sequences (SAMM-LV) and is competitive in a low frame-rate (30 fps) dataset (CAS(ME)2). In this paper, we document our deep learning model and parameters, including how we use local contrast normalisation, which we show is critical for optimal results. We surpass a limitation in existing methods, and advance the state of deep learning in the domain of facial expression spotting

Sheridan School of Architectural Technology Volume 3 [W2018+S2018]

This volume shows the work of the graduating students of the Architectural Technology programme. Once again, their hard work over three years of study shows both in the variety and in the quality of their work. The work presented here was prepared in a single course but it draws from all the courses in the programme. It reflects their capabilities in design, building science, legislation, regulations, graphical representation and technology. Each piece of work represents the individual blend of these that each student possesses. As they leave Sheridan they take with them a range of knowledge and skills that will start their careers. As their careers develop some will do exactly what they thought and more will move in completely new and unexpected directions. Please enjoy the work presented here and think of the future to come.https://source.sheridancollege.ca/fast_books/1005/thumbnail.jp

Combining physiological, environmental and locational sensors for citizen-oriented health applications

This work investigates the potential of combining the outputs of multiple low-cost sensor technologies for the direct measurement of spatio-temporal variations in phenomena that exist at the interface between our bodies and the environment. The example used herein is the measurement of personal exposure to traffic pollution, which may be considered as a function of the concentration of pollutants in the air and the frequency and volume of that air which enters our lungs. The sensor-based approach described in this paper removes the ‘traditional’ requirements either to model or interpolate pollution levels or to make assumptions about the physiology of an individual. Rather, a wholly empirical analysis into pollution exposure is possible, based upon high-resolution spatio-temporal data drawn from sensors for NO2, nasal airflow and location (GPS). Data are collected via a custom smartphone application and mapped to give an unprecedented insight into exposure to traffic pollution at the individual level. Whilst the quality of data from low-cost miniaturised sensors is not suitable for all applications, there certainly are many applications for which these data would be well suited, particularly those in the field of citizen science. This paper demonstrates both the potential and limitations of sensor-based approaches and discusses the wider relevance of these technologies for the advancement of citizen science

Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks

Breast lesion detection using ultrasound imaging is considered an important step of Computer-Aided Diagnosis systems. Over the past decade, researchers have demonstrated the possibilities to automate the initial lesion detection. However, the lack of a common dataset impedes research when comparing the performance of such algorithms. This paper proposes the use of deep learning approaches for breast ultrasound lesion detection and investigates three different methods: a Patch-based LeNet, a U-Net, and a transfer learning approach with a pretrained FCN-AlexNet. Their performance is compared against four state-of-the-art lesion detection algorithms (i.e. Radial Gradient Index, Multifractal Filtering, Rule-based Region Ranking and Deformable Part Models). In addition, this paper compares and contrasts two conventional ultrasound image datasets acquired from two different ultrasound systems. Dataset A comprises 306 (60 malignant and 246 benign) images and Dataset B comprises 163 (53 malignant and 110 benign) images. To overcome the lack of public datasets in this domain, Dataset B will be made available for research purposes. The results demonstrate an overall improvement by the deep learning approaches when assessed on both datasets in terms of True Positive Fraction, False Positives per image, and F-measure.authorsversionPeer reviewe

3D-CNN for Facial Micro- and Macro-expression Spotting on Long Video Sequences using Temporal Oriented Reference Frame

Facial expression spotting is the preliminary step for micro- and macro-expression analysis. The task of reliably spotting such expressions in video sequences is currently unsolved. The current best systems depend upon optical flow methods to extract regional motion features, before categorisation of that motion into a specific class of facial movement. Optical flow is susceptible to drift error, which introduces a serious problem for motions with long-term dependencies, such as high frame-rate macro-expression. We propose a purely deep learning solution which, rather than track frame differential motion, compares via a convolutional model, each frame with two temporally local reference frames. Reference frames are sampled according to calculated micro- and macro-expression durations. We show that our solution achieves state-of-the-art performance (F1-score of 0.126) in a dataset of high frame-rate (200 fps) long video sequences (SAMM-LV) and is competitive in a low frame-rate (30 fps) dataset (CAS(ME)2). In this paper, we document our deep learning model and parameters, including how we use local contrast normalisation, which we show is critical for optimal results. We surpass a limitation in existing methods, and advance the state of deep learning in the domain of facial expression spotting