Measuring instant emotions based on facial expression during computer-based assessment

Abstract Emotions are very important during learning and assessment procedures. However, measuring emotions is a very demanding task. Several tools have been developed and used for this purpose. In this paper, the efficiency of the FaceReader during a computer-based assessment (CBA) was evaluated. Instant measurements of the FaceReader were compared with the researchers' estimations regarding students' emotions. The observations took place in a properly designed room in real time. Statistical analysis showed that there are some differences between FaceReader's and researchers' estimations regarding Disgusted and Angry emotions. Results showed that FaceReader is capable of measuring emotions with an efficacy of over 87% during a CBA and that it could be successfully integrated into a computer-aided learning system for the purpose of emotion recognition. Moreover, this study provides useful results for the emotional states of students during CBA and learning procedures. This is actually the first time that student's instant emotions were measured during a CBA, based on their facial expressions. Results showed that most of the time students were experiencing Neutral, Angry, and Sad emotions. Furthermore, gender analysis highlights differences between genders' instant emotions

Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing

Effects of Differences of Middle Eastern and American Cultural Backgrounds on Salt Perception Using FaceReader : A Pioneering Tool to Capture Instant Emotional Reactions

A preliminary study was conducted (n=65) to compare the consistency of measurements of emotional reactions to salt intake using FaceReader (facial expression analyzer) and hedonic self-reports. Study participants tasted six different samples of mashed potatoes with three different concentrations of sodium (Na) (two samples of 0 mg, 178 mg, and 356 mg/15g each). Emotional reactions were measured with FaceReader and self-reported on a 10-point scale, and the mean for each of the different sodium concentrations was calculated. Six different emotional expressions were measured (happy, sad, angry, surprised, scared, and disgusted) for each of the samples. Differences between the means for the two samples with the same sodium concentration were then compared. Our findings have indicated that the differences between the means were lower for FaceReader measurements. These findings occurred more frequently with 16 instances of greater consistency for FaceReader compared with two for hedonic self-reports. Therefore, FaceReader measurements provided more consistent emotional measurements for comparative sodium concentrations than were provided by hedonic self-reports. In particular, the emotions of “happy,” “sad,” “scared,” and “disgusted” were measured more consistently by FaceReader for all concentrations of sodium. FaceReader and hedonic self-reports provide similar results for “angry” and “surprised” . Furthermore, the cultural background was examined and Primary Study was conducted (n=100) to find if cultural backgrounds have an effect on salt perception between American and immigrant groups, using FaceReader to measure their facial emotional reactions. The American group was comprised of 50 participants and, comparably, the immigrant group was comprised of 50 participants, in total the Primary Study included 54 males and 46 females. Our findings demonstrate that cultural background has a significant effect on people’s taste preference for salt. In particular, Americans have a higher salt preference than immigrants. The fast-food culture, and the increased availability of highly processed and affordable foods manufactured by big food companies in the United States are the main contributors for this difference. Future research in these areas are needed to explore these topics further

What Twitter Profile and Posted Images Reveal About Depression and Anxiety

Previous work has found strong links between the choice of social media images and users' emotions, demographics and personality traits. In this study, we examine which attributes of profile and posted images are associated with depression and anxiety of Twitter users. We used a sample of 28,749 Facebook users to build a language prediction model of survey-reported depression and anxiety, and validated it on Twitter on a sample of 887 users who had taken anxiety and depression surveys. We then applied it to a different set of 4,132 Twitter users to impute language-based depression and anxiety labels, and extracted interpretable features of posted and profile pictures to uncover the associations with users' depression and anxiety, controlling for demographics. For depression, we find that profile pictures suppress positive emotions rather than display more negative emotions, likely because of social media self-presentation biases. They also tend to show the single face of the user (rather than show her in groups of friends), marking increased focus on the self, emblematic for depression. Posted images are dominated by grayscale and low aesthetic cohesion across a variety of image features. Profile images of anxious users are similarly marked by grayscale and low aesthetic cohesion, but less so than those of depressed users. Finally, we show that image features can be used to predict depression and anxiety, and that multitask learning that includes a joint modeling of demographics improves prediction performance. Overall, we find that the image attributes that mark depression and anxiety offer a rich lens into these conditions largely congruent with the psychological literature, and that images on Twitter allow inferences about the mental health status of users.Comment: ICWSM 201

Machine Analysis of Facial Expressions

No abstract

Investigating facial animation production through artistic inquiry

Studies into dynamic facial expressions tend to make use of experimental methods based on objectively manipulated stimuli. New techniques for displaying increasingly realistic facial movement and methods of measuring observer responses are typical of computer animation and psychology facial expression research. However, few projects focus on the artistic nature of performance production. Instead, most concentrate on the naturalistic appearance of posed or acted expressions. In this paper, the authors discuss a method for exploring the creative process of emotional facial expression animation, and ask whether anything can be learned about authentic dynamic expressions through artistic inquiry

Measuring user emotionality on online videos: A comparison between self-report and facial expression analysis

One common factor that unites the popularity of online video viewers is their virality. Marketers and academics have been involved in the contemporary research not only to understand how online virality occurs but in addition how it can be measured. Thus, the aim of this paper is threefold: a) to advance the understanding of what online video virality is b) to propose a conceptual framework for measuring video virality c) to evaluate two main contrasting methods for measuring video virality. The conceptual framework identifies key elements to video virality as emotions and social groups, and the tools proposed to be used for measuring online video virality is the FaceReader and the online web questionnaire. The findings from the study indicate the existence of discriminant validity between the two methods which inherently adds to the theoretical advancement with the notion that video marketers or researchers cannot use self-report to measure emotions or use it synchronously with facial expression analysis on online videos

Unleashing the Power of VGG16: Advancements in Facial Emotion Recognization

In facial emotion detection, researchers are actively exploring effective methods to identify and understand facial expressions. This study introduces a novel mechanism for emotion identification using diverse facial photos captured under varying lighting conditions. A meticulously pre-processed dataset ensures data consistency and quality. Leveraging deep learning architectures, the study utilizes feature extraction techniques to capture subtle emotive cues and build an emotion classification model using convolutional neural networks (CNNs). The proposed methodology achieves an impressive 97% accuracy on the validation set, outperforming previous methods in terms of accuracy and robustness. Challenges such as lighting variations, head posture, and occlusions are acknowledged, and multimodal approaches incorporating additional modalities like auditory or physiological data are suggested for further improvement. The outcomes of this research have wide-ranging implications for affective computing, human-computer interaction, and mental health diagnosis, advancing the field of facial emotion identification and paving the way for sophisticated technology capable of understanding and responding to human emotions across diverse domains

Emotions used in Learning Analytics: a state-of-the-art review

Emotions play a critical role in the learning and teaching process because learners’ feelings impact motivation, self-regulation and academic achievement. In this literature review of 100+ studies, we identify approximately 100 different emotions that may have a positive, negative or neutral impact on learners’ attitudes, behaviour and cognition. In this review, we explore seven methods of data gathering approaches to measure and understand emotions (i.e., content analysis, natural language processing, behavioural indicators, quantitative instruments, qualitative approaches, well-being word clouds, and intelligent tutoring systems). With increased affordances of technologies to continuously measure emotions (e.g., facial and voice expressions with tablets and smart phones), it might become feasible to monitor learners’ emotions on a real-time basis in the near future