Objective Classes for Micro-Facial Expression Recognition

Micro-expressions are brief spontaneous facial expressions that appear on a face when a person conceals an emotion, making them different to normal facial expressions in subtlety and duration. Currently, emotion classes within the CASME II dataset are based on Action Units and self-reports, creating conflicts during machine learning training. We will show that classifying expressions using Action Units, instead of predicted emotion, removes the potential bias of human reporting. The proposed classes are tested using LBP-TOP, HOOF and HOG 3D feature descriptors. The experiments are evaluated on two benchmark FACS coded datasets: CASME II and SAMM. The best result achieves 86.35\% accuracy when classifying the proposed 5 classes on CASME II using HOG 3D, outperforming the result of the state-of-the-art 5-class emotional-based classification in CASME II. Results indicate that classification based on Action Units provides an objective method to improve micro-expression recognition.Comment: 11 pages, 4 figures and 5 tables. This paper will be submitted for journal revie

Dynamic texture recognition using time-causal and time-recursive spatio-temporal receptive fields

This work presents a first evaluation of using spatio-temporal receptive fields from a recently proposed time-causal spatio-temporal scale-space framework as primitives for video analysis. We propose a new family of video descriptors based on regional statistics of spatio-temporal receptive field responses and evaluate this approach on the problem of dynamic texture recognition. Our approach generalises a previously used method, based on joint histograms of receptive field responses, from the spatial to the spatio-temporal domain and from object recognition to dynamic texture recognition. The time-recursive formulation enables computationally efficient time-causal recognition. The experimental evaluation demonstrates competitive performance compared to state-of-the-art. Especially, it is shown that binary versions of our dynamic texture descriptors achieve improved performance compared to a large range of similar methods using different primitives either handcrafted or learned from data. Further, our qualitative and quantitative investigation into parameter choices and the use of different sets of receptive fields highlights the robustness and flexibility of our approach. Together, these results support the descriptive power of this family of time-causal spatio-temporal receptive fields, validate our approach for dynamic texture recognition and point towards the possibility of designing a range of video analysis methods based on these new time-causal spatio-temporal primitives.Comment: 29 pages, 16 figure

The Historical Origin of the Pulfrich Effect: A Serendipitous Astronomic Observation at the Border of the Milky Way

Interested in star movement the founder of Heidelberg's astronomy observatory, Max Wolf, faced the dilemma that the hitherto used 'Blinkmikrosop' of his Institution was damaged beyond repair following the first world war. He therefore used a new method, stereoscopy, to systematically classify 1053 moving stars between 1915 and 1918. The key problem Wolf identified with the new method was that variation in brightness of the same star on different photographic plates gave rise to an illusory movement. This was a particularly frequent problem with smaller stars close to the very bright Milky Way such as those in the proximity of Cygni or fade-out stars such as R Coronae Borealis. Carl Pulfrich, the world-leading expert on stereoscopy at the time, picked up immediately on the technical limitations Wolf published on stereoscopy in 1920. Pulfrich, who was blind in one eye, could not see the effect himself and designed a projection device to demonstrate Wolf's serendipitous observation to an audience which was equipped with a monocular neutral density filter. Pulfrich performed detailed investigations on the relationship of spatial perception and object movement, naming the phenomenon stereo effect, but it became widely known as the Pulfrich effect. The neuro-anatomical basis of the Pulfrich effect lies in the joint encoding of motion and depth within the visual cortex. Recognising Pulfrich effect is relevant for the management of patients in whom pathology of the visual pathways impairs judgment of object movement/position (e.g., in traffic or sport). Fitting a unilateral tinted lens or contact lens in front of the good eye can abolish the problem

Antiferromagnetism in NiO Observed by Transmission Electron Diffraction

Neutron diffraction has been used to investigate antiferromagnetism since 1949. Here we show that antiferromagnetic reflections can also be seen in transmission electron diffraction patterns from NiO. The diffraction patterns taken here came from regions as small as 10.5 nm and such patterns could be used to form an image of the antiferromagnetic structure with a nanometre resolution.Comment: 10 pages, 7 figures. Typos corrected. To appear in Physical Review Letter

Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition

In the recent year, state-of-the-art for facial micro-expression recognition have been significantly advanced by deep neural networks. The robustness of deep learning has yielded promising performance beyond that of traditional handcrafted approaches. Most works in literature emphasized on increasing the depth of networks and employing highly complex objective functions to learn more features. In this paper, we design a Shallow Triple Stream Three-dimensional CNN (STSTNet) that is computationally light whilst capable of extracting discriminative high level features and details of micro-expressions. The network learns from three optical flow features (i.e., optical strain, horizontal and vertical optical flow fields) computed based on the onset and apex frames of each video. Our experimental results demonstrate the effectiveness of the proposed STSTNet, which obtained an unweighted average recall rate of 0.7605 and unweighted F1-score of 0.7353 on the composite database consisting of 442 samples from the SMIC, CASME II and SAMM databases.Comment: 5 pages, 1 figure, Accepted and published in IEEE FG 201

A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences

Facial expression causes different parts of the facial region to change over time and thus dynamic descriptors are inherently more suitable than static descriptors for recognising facial expressions. In this paper, we extend the spatial pyramid histogram of gradients to spatio-temporal domain to give 3-dimensional facial features and integrate them with dense optical flow to give a spatio-temporal descriptor which extracts both the spatial and dynamic motion information of facial expressions. A multi-class support vector machine based classifier with one-to-one strategy is used to recognise facial expressions. Experiments on the CK+ and MMI datasets using leave-one-out cross validation scheme demonstrate that the integrated framework achieves a better performance than using individual descriptor separately. Compared with six state of the art methods, the proposed framework demonstrates a superior performance

Associating low-level features with semantic concepts using video objects and relevance feedback

The holy grail of multimedia indexing and retrieval is developing algorithms capable of imitating human abilities in distinguishing and recognising semantic concepts within the content, so that retrieval can be based on ”real world” concepts that come naturally to users. In this paper, we discuss an approach to using segmented video objects as the midlevel connection between low-level features and semantic concept description. In this paper, we consider a video object as a particular instance of a semantic concept and we model the semantic concept as an average representation of its instances. A system supporting object-based search through a test corpus is presented that allows matching presegmented objects based on automatically extracted lowlevel features. In the system, relevance feedback is employed to drive the learning of the semantic model during a regular search process