7 research outputs found
Using system and user performance features to improve emotion detection in spoken tutoring dialogs
In this study, we incorporate automatically obtained system/user performance features into machine learning experiments to detect student emotion in computer tutoring dialogs. Our results show a relative improvement of 2.7% on classification accuracy and 8.08% on Kappa over using standard lexical, prosodie, sequential, and identification features. This level of improvement is comparable to the performance improvement shown in previous studies by applying dialog acts or lexical/prosodic-/discourse- level contextual features
Knowledge Elicitation Methods for Affect Modelling in Education
Research on the relationship between affect and cognition in Artificial Intelligence in Education (AIEd) brings an important dimension to our understanding of how learning occurs and how it can be facilitated. Emotions are crucial to learning, but their nature, the conditions under which they occur, and their exact impact on learning for different learners in diverse contexts still needs to be mapped out. The study of affect during learning can be challenging, because emotions are subjective, fleeting phenomena that are often difficult for learners to report accurately and for observers to perceive reliably. Context forms an integral part of learnersâ affect and the study thereof. This review provides a synthesis of the current knowledge elicitation methods that are used to aid the study of learnersâ affect and to inform the design of intelligent technologies for learning. Advantages and disadvantages of the specific methods are discussed along with their respective potential for enhancing research in this area, and issues related to the interpretation of data that emerges as the result of their use. References to related research are also provided together with illustrative examples of where the individual methods have been used in the past. Therefore, this review is intended as a resource for methodological decision making for those who want to study emotions and their antecedents in AIEd contexts, i.e. where the aim is to inform the design and implementation of an intelligent learning environment or to evaluate its use and educational efficacy
The role of learning theory in multimodal learning analytics
This study presents the outcomes of a semi-systematic
literature review on the role of learning theory in multimodal learning analytics (MMLA) research. Based on
previous systematic literature reviews in MMLA and
an additional new search, 35MMLA works were identified that use theory. The results show that MMLA
studies do not always discuss their findings within
an established theoretical framework. Most of the
theory-driven MMLA studies are positioned in the
cognitive and affective domains, and the three most
frequently used theories are embodied cognition,
cognitive load theory and controlâvalue theory of
achievement emotions. Often, the theories are only
used to inform the study design, but there is a relationship between the most frequently used theories
and the data modalities used to operationalize those
theories. Although studies such as these are rare, the
findings indicate that MMLA affordances can, indeed,
lead to theoretical contributions to learning sciences.
In this work, we discuss methods of accelerating
theory-driven MMLA research and how this acceleration can extend or even create new theoretical
knowledge
Carelessness and Affect in an Intelligent Tutoring System for Mathematics
We investigate the relationship between students' affect and their frequency of careless errors while using an Intelligent Tutoring System for middle school mathematics. A student is said to have committed a careless error when the student's answer is wrong despite knowing the skill required to provide the correct answer. We operationalize the probability that an error is careless through the use of an automated detector, developed using educational data mining, which infers the probability that an error involves carelessness rather than not knowing the relevant skill. This detector is then applied to log data produced by high-school students in the Philippines using a Cognitive Tutor for scatterplots. We study the relationship between carelessness and affect, triangulating between the detector of carelessness and field observations of affect. Surprisingly, we find that carelessness is common among students who frequently experience engaged concentration. This finding implies that a highly engaged student may paradoxically become overconfident or impulsive, leading to more careless errors. In contrast, students displaying confusion or boredom make fewer careless errors. Further analysis over time suggests that confused and bored students have lower learning overall. Thus, their mistakes appear to stem from a genuine lack of knowledge rather than carelessness
Enhancing electronic intelligent tutoring systems by responding to affective states
The overall aim of this research is the exploration mechanisms which allow an understanding of the emotional state of students and the selection of an appropriate cognitive and affective feedback for students on the basis of students' emotional state and cognitive state in an affective learning environment. The learning environment in which this research is based is one in which students learn by watching an instructional video. The main contributions in the thesis include:
- A video study was carried out to gather data in order to construct the emotional models in this research. This video study adopted a methodology in qualitative research called âQuick and Dirty Ethnographyâ(Hughes et al., 1995). In the video study, the emotional states, including boredom, frustration, confusion, flow, happiness, interest, were identified as being the most important to a learner in learning. The results of the video study indicates that blink frequencies can reflect the learner's emotional states and it is necessary to intervene when students are in self-learning through watching an instructional video in order to ensure that attention levels do not decrease.
- A novel emotional analysis model for modeling studentâs cognitive and emotional state in an affective learning system was constructed. It is an appraisal model which is on the basis of an instructional theory called Gagneâs theory (Gagne, 1965).
- A novel emotion feedback model for producing appropriate feedback tactics in affective learning system was developed by Ontology and Influence Diagram ii approach. On the basis of the tutor-remediation hypothesis and the self-remediation hypothesis (Hausmann et al., 2013), two feedback tactic selection algorithms were designed and implemented.
The evaluation results show: the emotion analysis model can be used to classify negative emotion and hence deduce the learnerâs cognitive state; the degree of satisfaction with the feedback based on the tutor-remediation hypothesis is higher than the feedback based on self-remediation hypothesis; the results indicated a higher degree of satisfaction with the combined cognitive and emotional feedback than cognitive feedback on its own
ModĂ©lisation des Ă©motions de lâapprenant et interventions implicites pour les systĂšmes tutoriels intelligents
La modĂ©lisation de lâexpĂ©rience de lâutilisateur dans les Interactions Homme-Machine est un enjeu important pour la conception et le dĂ©veloppement des systĂšmes adaptatifs intelligents. Dans ce contexte, une attention particuliĂšre est portĂ©e sur les rĂ©actions Ă©motionnelles de lâutilisateur, car elles ont une influence capitale sur ses aptitudes cognitives, comme la perception et la prise de dĂ©cision. La modĂ©lisation des Ă©motions est particuliĂšrement pertinente pour les SystĂšmes Tutoriels Ămotionnellement Intelligents (STEI). Ces systĂšmes cherchent Ă identifier les Ă©motions de lâapprenant lors des sessions dâapprentissage, et Ă optimiser son expĂ©rience dâinteraction en recourant Ă diverses stratĂ©gies dâinterventions.
Cette thĂšse vise Ă amĂ©liorer les mĂ©thodes de modĂ©lisation des Ă©motions et les stratĂ©gies Ă©motionnelles utilisĂ©es actuellement par les STEI pour agir sur les Ă©motions de lâapprenant. Plus prĂ©cisĂ©ment, notre premier objectif a Ă©tĂ© de proposer une nouvelle mĂ©thode pour dĂ©tecter lâĂ©tat Ă©motionnel de lâapprenant, en utilisant diffĂ©rentes sources dâinformations qui permettent de mesurer les Ă©motions de façon prĂ©cise, tout en tenant compte des variables individuelles qui peuvent avoir un impact sur la manifestation des Ă©motions. Pour ce faire, nous avons dĂ©veloppĂ© une approche multimodale combinant plusieurs mesures physiologiques (activitĂ© cĂ©rĂ©brale, rĂ©actions galvaniques et rythme cardiaque) avec des variables individuelles, pour dĂ©tecter une Ă©motion trĂšs frĂ©quemment observĂ©e lors des sessions dâapprentissage, Ă savoir lâincertitude. Dans un premier lieu, nous avons identifiĂ© les indicateurs physiologiques clĂ©s qui sont associĂ©s Ă cet Ă©tat, ainsi que les caractĂ©ristiques individuelles qui contribuent Ă sa manifestation. Puis, nous avons dĂ©veloppĂ© des modĂšles prĂ©dictifs permettant de dĂ©tecter automatiquement cet Ă©tat Ă partir des diffĂ©rentes variables analysĂ©es, Ă travers lâentrainement dâalgorithmes dâapprentissage machine.
Notre deuxiĂšme objectif a Ă©tĂ© de proposer une approche unifiĂ©e pour reconnaĂźtre simultanĂ©ment une combinaison de plusieurs Ă©motions, et Ă©valuer explicitement lâimpact de ces Ă©motions sur lâexpĂ©rience dâinteraction de lâapprenant. Pour cela, nous avons dĂ©veloppĂ© une plateforme hiĂ©rarchique, probabiliste et dynamique permettant de suivre les changements Ă©motionnels de l'apprenant au fil du temps, et dâinfĂ©rer automatiquement la tendance gĂ©nĂ©rale qui caractĂ©rise son expĂ©rience dâinteraction Ă savoir : lâimmersion, le blocage ou le dĂ©crochage. Lâimmersion correspond Ă une expĂ©rience optimale : un Ă©tat dans lequel l'apprenant est complĂštement concentrĂ© et impliquĂ© dans lâactivitĂ© dâapprentissage. LâĂ©tat de blocage correspond Ă une tendance dâinteraction non optimale oĂč l'apprenant a de la difficultĂ© Ă se concentrer. Finalement, le dĂ©crochage correspond Ă un Ă©tat extrĂȘmement dĂ©favorable oĂč lâapprenant nâest plus du tout impliquĂ© dans lâactivitĂ© dâapprentissage. La plateforme proposĂ©e intĂšgre trois modalitĂ©s de variables diagnostiques permettant dâĂ©valuer lâexpĂ©rience de lâapprenant Ă savoir : des variables physiologiques, des variables comportementales, et des mesures de performance, en combinaison avec des variables prĂ©dictives qui reprĂ©sentent le contexte courant de lâinteraction et les caractĂ©ristiques personnelles de l'apprenant. Une Ă©tude a Ă©tĂ© rĂ©alisĂ©e pour valider notre approche Ă travers un protocole expĂ©rimental permettant de provoquer dĂ©libĂ©rĂ©ment les trois tendances ciblĂ©es durant lâinteraction des apprenants avec diffĂ©rents environnements dâapprentissage.
Enfin, notre troisiĂšme objectif a Ă©tĂ© de proposer de nouvelles stratĂ©gies pour influencer positivement lâĂ©tat Ă©motionnel de lâapprenant, sans interrompre la dynamique de la session dâapprentissage. Nous avons Ă cette fin introduit le concept de stratĂ©gies Ă©motionnelles implicites : une nouvelle approche pour agir subtilement sur les Ă©motions de lâapprenant, dans le but dâamĂ©liorer son expĂ©rience dâapprentissage. Ces stratĂ©gies utilisent la perception subliminale, et plus prĂ©cisĂ©ment une technique connue sous le nom dâamorçage affectif. Cette technique permet de solliciter inconsciemment les Ă©motions de lâapprenant, Ă travers la projection dâamorces comportant certaines connotations affectives. Nous avons mis en Ćuvre une stratĂ©gie Ă©motionnelle implicite utilisant une forme particuliĂšre dâamorçage affectif Ă savoir : le conditionnement Ă©valuatif, qui est destinĂ© Ă amĂ©liorer de façon inconsciente lâestime de soi. Une Ă©tude expĂ©rimentale a Ă©tĂ© rĂ©alisĂ©e afin dâĂ©valuer lâimpact de cette stratĂ©gie sur les rĂ©actions Ă©motionnelles et les performances des apprenants.Modeling the userâs experience within Human-Computer Interaction is an important challenge for the design and development of intelligent adaptive systems. In this context, a particular attention is given to the userâs emotional reactions, as they decisively influence his cognitive abilities, such as perception and decision-making. Emotion modeling is particularly relevant for Emotionally Intelligent Tutoring Systems (EITS). These systems seek to identify the learnerâs emotions during tutoring sessions, and to optimize his interaction experience using a variety of intervention strategies.
This thesis aims to improve current methods on emotion modeling, as well as the emotional strategies that are presently used within EITS to influence the learnerâs emotions. More precisely, our first objective was to propose a new method to recognize the learnerâs emotional state, using different sources of information that allow to measure emotions accurately, whilst taking account of individual characteristics that can have an impact on the manifestation of emotions. To that end, we have developed a multimodal approach combining several physiological measures (brain activity, galvanic responses and heart rate) with individual variables, to detect a specific emotion, which is frequently observed within computer tutoring, namely : uncertainty. First, we have identified the key physiological indicators that are associated to this state, and the individual characteristics that contribute to its manifestation. Then, we have developed predictive models to automatically detect this state from the analyzed variables, trough machine learning algorithm training.
Our second objective was to propose a unified approach to simultaneously recognize a combination of several emotions, and to explicitly evaluate the impact of these emotions on the learnerâs interaction experience. For this purpose, we have developed a hierarchical, probabilistic and dynamic framework, which allows one to track the learnerâs emotional changes over time, and to automatically infer the trend that characterizes his interaction experience namely : flow, stuck or off-task. Flow is an optimal experience : a state in which the learner is completely focused and involved within the learning activity. The state of stuck is a non-optimal trend of the interaction where the learner has difficulty to maintain focused attention. Finally, the off-task behavior is an extremely unfavorable state where the learner is not involved anymore within the learning session. The proposed framework integrates three-modality diagnostic variables that sense the learnerâs experience including : physiology, behavior and performance, in conjunction with predictive variables that represent the current context of the interaction and the learnerâs personal characteristics. A human-subject study was conducted to validate our approach through an experimental protocol designed to deliberately elicit the three targeted trends during the learnersâ interaction with different learning environments.
Finally, our third objective was to propose new strategies to positively influence the learnerâs emotional state, without interrupting the dynamics of the learning session. To this end, we have introduced the concept of implicit emotional strategies : a novel approach to subtly impact the learnerâs emotions, in order to improve his learning experience. These strategies use the subliminal perception, and more precisely a technique known as affective priming. This technique aims to unconsciously solicit the learnerâs emotions, through the projection of primes charged with specific affective connotations. We have implemented an implicit emotional strategy using a particular form of affective priming namely : the evaluative conditioning, which is designed to unconsciously enhance self-esteem. An experimental study was conducted in order to evaluate the impact of this strategy on the learnersâ emotional reactions and performance