Robust Modeling of Epistemic Mental States

This work identifies and advances some research challenges in the analysis of facial features and their temporal dynamics with epistemic mental states in dyadic conversations. Epistemic states are: Agreement, Concentration, Thoughtful, Certain, and Interest. In this paper, we perform a number of statistical analyses and simulations to identify the relationship between facial features and epistemic states. Non-linear relations are found to be more prevalent, while temporal features derived from original facial features have demonstrated a strong correlation with intensity changes. Then, we propose a novel prediction framework that takes facial features and their nonlinear relation scores as input and predict different epistemic states in videos. The prediction of epistemic states is boosted when the classification of emotion changing regions such as rising, falling, or steady-state are incorporated with the temporal features. The proposed predictive models can predict the epistemic states with significantly improved accuracy: correlation coefficient (CoERR) for Agreement is 0.827, for Concentration 0.901, for Thoughtful 0.794, for Certain 0.854, and for Interest 0.913.Comment: Accepted for Publication in Multimedia Tools and Application, Special Issue: Socio-Affective Technologie

Using Applied Behavior Analysis in Software to help Tutor Individuals with Autism Spectrum Disorder

There are currently many tutoring software systems which have been designed for neurotypical children. These systems cover academic topics such as reading and math, and are made available through various technological mediums. The majority of these systems were not designed for use by children with special needs, in particular those who are diagnosed with Autism Spectrum Disorder. Since the 1970's, studies have been conducted on the use of Applied Behavior Analysis to help autistic children learn [1]. This teaching methodology is proven to be very effective, with many patients having their diagnosis of autism dropped after a few years of treatment. With the advent of ubiquitous technologies such as mobile devices, it has become apparent that these devices could also be used to help tutor autistic children on academic subjects such as reading and math. Though the delivery of tutoring material must be made using Applied Behavior Analysis techniques, given that ABA therapy is currently the only form of treatment for Autism Spectrum Disorder endorsed by the US Surgeon General [2], which further makes the case for incorporating it into an academics tutoring system tailored for autistic children. In this paper, we present a mobile software system which can be utilized to tutor children who are diagnosed with Autism Spectrum Disorder in the subjects of reading and math. The software makes use of Applied Behavior Analysis techniques such as a Token Economy system, visual and audible reinforcers, and generalization. Furthermore, we explore how combining Applied Behavior Analysis and technology, could help extend the reach of tutoring systems to these children.Comment: 8 pages, 7 figure

Proceedings of QG2010: The Third Workshop on Question Generation

These are the peer-reviewed proceedings of "QG2010, The Third Workshop on Question Generation". The workshop included a special track for "QGSTEC2010: The First Question Generation Shared Task and Evaluation Challenge". QG2010 was held as part of The Tenth International Conference on Intelligent Tutoring Systems (ITS2010)

Assessing the Effectiveness of Automated Emotion Recognition in Adults and Children for Clinical Investigation

Recent success stories in automated object or face recognition, partly fuelled by deep learning artificial neural network (ANN) architectures, has led to the advancement of biometric research platforms and, to some extent, the resurrection of Artificial Intelligence (AI). In line with this general trend, inter-disciplinary approaches have taken place to automate the recognition of emotions in adults or children for the benefit of various applications such as identification of children emotions prior to a clinical investigation. Within this context, it turns out that automating emotion recognition is far from being straight forward with several challenges arising for both science(e.g., methodology underpinned by psychology) and technology (e.g., iMotions biometric research platform). In this paper, we present a methodology, experiment and interesting findings, which raise the following research questions for the recognition of emotions and attention in humans: a) adequacy of well-established techniques such as the International Affective Picture System (IAPS), b) adequacy of state-of-the-art biometric research platforms, c) the extent to which emotional responses may be different among children or adults. Our findings and first attempts to answer some of these research questions, are all based on a mixed sample of adults and children, who took part in the experiment resulting into a statistical analysis of numerous variables. These are related with, both automatically and interactively, captured responses of participants to a sample of IAPS pictures

Machine Analysis of Facial Expressions

No abstract