Robust Modeling of Epistemic Mental States

This work identifies and advances some research challenges in the analysis of facial features and their temporal dynamics with epistemic mental states in dyadic conversations. Epistemic states are: Agreement, Concentration, Thoughtful, Certain, and Interest. In this paper, we perform a number of statistical analyses and simulations to identify the relationship between facial features and epistemic states. Non-linear relations are found to be more prevalent, while temporal features derived from original facial features have demonstrated a strong correlation with intensity changes. Then, we propose a novel prediction framework that takes facial features and their nonlinear relation scores as input and predict different epistemic states in videos. The prediction of epistemic states is boosted when the classification of emotion changing regions such as rising, falling, or steady-state are incorporated with the temporal features. The proposed predictive models can predict the epistemic states with significantly improved accuracy: correlation coefficient (CoERR) for Agreement is 0.827, for Concentration 0.901, for Thoughtful 0.794, for Certain 0.854, and for Interest 0.913.Comment: Accepted for Publication in Multimedia Tools and Application, Special Issue: Socio-Affective Technologie

Sensitizing a Gifted Child with Autism Spectrum Disorder towards Social Cognition: From Assessment to Treatment

Social cognition difficulties are well documented in children with Autism Spectrum Disorder (ASD). This clinical case study reports on social cognition assessment and treatment of a gifted child, SC (9 years), with ASD and an extraordinarily high verbal IQ (146). The assessment of theory of mind, emotion recognition and pragmatic abilities showed some weaknesses in these areas. The 4- month treatment was divided into 14 sessions and was aimed at helping SC to improve his abilities to recognize emotions and comprehend mental states. The main technique used in the intervention was “social reading” with video clips. The initial assessment was repeated after the treatment and in a follow up session. The results of this case study suggest that “social reading” can be considered as a beneficial technique for children with high-functioning ASD, and especially in giftedness conditions

Machine Analysis of Facial Expressions

No abstract

FACETEQ interface demo for emotion expression in VR

© 2017 IEEE.Faceteq prototype v.05 is a wearable technology for measuring facial expressions and biometric responses for experimental studies in Virtual Reality. Developed by Emteq Ltd laboratory, Faceteq can enable new avenues for virtual reality research through combination of high performance patented dry sensor technologies, proprietary algorithms and real-time data acquisition and streaming. Emteq founded the Faceteq project with the aim to provide a human-centered additional tool for emotion expression, affective human-computer interaction and social virtual environments. The proposed demonstration will exhibit the hardware and its functionality by allowing attendees to experience three of the showcasing applications we developed this year

Helping Autistic Children Understand Their Emotions Using Facial Expression Recognition and Mobile Technologies

One of the main challenges for autistic children is to identify and express emotions. Many emotion-learning apps are available for smartphones and tablets to assist autistic children and their carers. However, they do not use the full potential offered by mobile technology, such as using facial expression recognition and wireless biosensors to recognise and sense emotions. To fill this gap we developed CaptureMyEmotion, an Android App that uses wireless sensors to capture physiological data together with facial expression recognition to provide a very personalised way to help autistic children learn about their emotions. The App enables children to capture photos, videos or sounds, and simultaneously attach emotion data and a self-portrait photo. The material can then be reviewed and discussed together with a carer at a later stage. CaptureMyEmotion has the potential to help autistic children integrate better in the society by providing a new way for them to understand their emotions