47,523 research outputs found
Cognitive behaviour analysis based on facial information using depth sensors
Cognitive behaviour analysis is considered of high impor- tance with many innovative applications in a range of sectors including healthcare, education, robotics and entertainment. In healthcare, cogni- tive and emotional behaviour analysis helps to improve the quality of life of patients and their families. Amongst all the different approaches for cognitive behaviour analysis, significant work has been focused on emo- tion analysis through facial expressions using depth and EEG data. Our work introduces an emotion recognition approach using facial expres- sions based on depth data and landmarks. A novel dataset was created that triggers emotions from long or short term memories. This work uses novel features based on a non-linear dimensionality reduction, t-SNE, applied on facial landmarks and depth data. Its performance was eval- uated in a comparative study, proving that our approach outperforms other state-of-the-art features
Recommended from our members
MGEED: A Multimodal Genuine Emotion and Expression Detection Database
Multimodal emotion recognition has attracted increasing interest from academia and industry in recent years, since it enables emotion detection using various modalities, such as facial expression images, speech and physiological signals. Although research in this field has grown rapidly, it is still challenging to create a multimodal database containing facial electrical information due to the difficulty in capturing natural and subtle facial expression signals, such as optomyography (OMG) signals. To this end, we present a newly developed Multimodal Genuine Emotion and Expression Detection (MGEED) database in this paper, which is the first publicly available database containing the facial OMG signals. MGEED consists of 17 subjects with over 150K facial images, 140K depth maps and different modalities of physiological signals including OMG, electroencephalography (EEG) and electrocardiography (ECG) signals. The emotions of the participants are evoked by video stimuli and the data are collected by a multimodal sensing system. With the collected data, an emotion recognition method is developed based on multimodal signal synchronisation, feature extraction, fusion and emotion prediction. The results show that superior performance can be achieved by fusing the visual, EEG and OMG features. The database can be obtained from https://github.com/YMPort/MGEED.Engineering and Physical Sciences Research Council (EPSRC) through the Project 4D Facial Sensing and Modelling under Grant EP/N025849/1
Investigating multi-modal features for continuous affect recognition using visual sensing
Emotion plays an essential role in human cognition, perception and rational decisionmaking.
In the information age, people spend more time then ever before interacting
with computers, however current technologies such as Artificial Intelligence (AI) and
Human-Computer Interaction (HCI) have largely ignored the implicit information of
a user’s emotional state leading to an often frustrating and cold user experience. To
bridge this gap between human and computer, the field of affective computing has
become a popular research topic. Affective computing is an interdisciplinary field
encompassing computer, social, cognitive, psychology and neural science. This thesis
focuses on human affect recognition, which is one of the most commonly investigated
areas in affective computing. Although from a psychology point of view, emotion
is usually defined differently from affect, for this thesis the terms emotion, affect,
emotional state and affective state are used interchangeably.
Both visual and vocal cues have been used in previous research to recognise a
human’s affective states. For visual cues, information from the face is often used.
Although these systems achieved good performance under laboratory settings, it
has proved a challenging task to translate these to unconstrained environments due
to variations in head pose and lighting conditions. Since a human face is a threedimensional (3D) object whose 2D projection is sensitive to the aforementioned
variations, recent trends have shifted towards using 3D facial information to improve
the accuracy and robustness of the systems. However these systems are still
focused on recognising deliberately displayed affective states, mainly prototypical
expressions of six basic emotions (happiness, sadness, fear, anger, surprise and disgust). To our best knowledge, no research has been conducted towards continuous
recognition of spontaneous affective states using 3D facial information.
The main goal of this thesis is to investigate the use of 2D (colour) and 3D
(depth) facial information to recognise spontaneous affective states continuously.
Due to a lack of an existing continuous annotated spontaneous data set, which
contains both colour and depth information, such a data set was created. To better
understand the processes in affect recognition and to compare results of the proposed
methods, a baseline system was implemented. Then the use of colour and depth
information for affect recognition were examined separately. For colour information,
an investigation was carried out to explore the performance of various state-of-art 2D
facial features using different publicly available data sets as well as the captured data set. Experiments were also carried out to study if it is possible to predict a human’s affective state using 2D features extracted from individual facial parts (E.g. eyes and mouth). For depth information, a number of histogram based features were used and their performance was evaluated. Finally a multi-modal affect recognition framework
utilising both colour and depth information is proposed and its performance was
evaluated using the captured data set
Web-based visualisation of head pose and facial expressions changes: monitoring human activity using depth data
Despite significant recent advances in the field of head pose estimation and
facial expression recognition, raising the cognitive level when analysing human
activity presents serious challenges to current concepts. Motivated by the need
of generating comprehensible visual representations from different sets of
data, we introduce a system capable of monitoring human activity through head
pose and facial expression changes, utilising an affordable 3D sensing
technology (Microsoft Kinect sensor). An approach build on discriminative
random regression forests was selected in order to rapidly and accurately
estimate head pose changes in unconstrained environment. In order to complete
the secondary process of recognising four universal dominant facial expressions
(happiness, anger, sadness and surprise), emotion recognition via facial
expressions (ERFE) was adopted. After that, a lightweight data exchange format
(JavaScript Object Notation-JSON) is employed, in order to manipulate the data
extracted from the two aforementioned settings. Such mechanism can yield a
platform for objective and effortless assessment of human activity within the
context of serious gaming and human-computer interaction.Comment: 8th Computer Science and Electronic Engineering, (CEEC 2016),
University of Essex, UK, 6 page
- …