Affective Computational Model to Extract Natural Affective States of Students with Asperger Syndrome (AS) in Computer-based Learning Environment

This study was inspired by looking at the central role of emotion in the learning process, its impact on students’ performance; as well as the lack of affective computing models to detect and infer affective-cognitive states in real time for students with and without Asperger Syndrome (AS). This model overcomes gaps in other models that were designed for people with autism, which needed the use of sensors or physiological instrumentations to collect data. The model uses a webcam to capture students’ affective-cognitive states of confidence, uncertainty, engagement, anxiety, and boredom. These states have a dominant effect on the learning process. The model was trained and tested on a natural-spontaneous affective dataset for students with and without AS, which was collected for this purpose. The dataset was collected in an uncontrolled environment and included variations in culture, ethnicity, gender, facial and hairstyle, head movement, talking, glasses, illumination changes and background variation. The model structure used deep learning (DL) techniques like convolutional neural network (CNN) and long short-term memory (LSTM). DL is the-state-of-art tool that used to reduce data dimensionality and capturing non-linear complex features from simpler representations. The affective model provide reliable results with accuracy 90.06%. This model is the first model to detected affective states for adult students with AS without physiological or wearable instruments. For the first time, the occlusions in this model, like hand over face or head were considered an important indicator for affective states like boredom, anxiety, and uncertainty. These occlusions have been ignored in most other affective models. The essential information channels in this model are facial expressions, head movement, and eye gaze. The model can serve as an aided-technology for tutors to monitor and detect the behaviors of all students at the same time and help in predicting negative affective states during learning process

An Evaluation of Mouse and Keyboard Interaction Indicators towards Non-intrusive and Low Cost Affective Modeling in an Educational Context

AbstractIn this paper we propose a series of indicators, which derive from user's interactions with mouse and keyboard. The goal is to evaluate their use in identifying affective states and behavior changes in an e-learning platform by means of non-intrusive and low cost methods. The approach we have followed study user's interactions regardless of the task being performed and its presentation, aiming at finding a solution applicable in any domain. In particular, mouse movements and clicks, as well as keystrokes were recorded during a math problem solving activity where users involved in the experiment had not only to score their degree of valence (i.e., pleasure versus displeasure) and arousal (i.e., high activation versus low activation) of their affective states after each problem by using the Self-Assessment-Manikin scale, but also type a description of their own feelings. By using that affective labeling, we evaluated the information provided by these different indicators processed from the original user's interactions logs. In total, we computed 42 keyboard indicators and 96 mouse indicators

Bodily sensation maps: Exploring a new direction for detecting emotions from user self-reported data

The ability of detecting emotions is essential in different fields such as user experience (UX), affective computing, and psychology. This paper explores the possibility of detecting emotions through user-generated bodily sensation maps (BSMs). The theoretical basis that inspires this work is the proposal by Nummenmaa et al. (2014) of BSMs for 14 emotions. To make it easy for users to create a BSM of how they feel, and convenient for researchers to acquire and classify users’ BSMs, we created a mobile app, called EmoPaint. The app includes an interface for BSM creation, and an automatic classifier that matches the created BSM with the BSMs for the 14 emotions. We conducted a user study aimed at evaluating both components of EmoPaint. First, it shows that the app is easy to use, and is able to classify BSMs consistently with the considered theoretical approach. Second, it shows that using EmoPaint increases accuracy of users’ emotion classification when compared with an adaptation of the well-known method of using the Affect Grid with the Circumplex Model, focused on the same set of 14 emotions of Nummenmaa et al. Overall, these results indicate that the novel approach of using BSMs in the context of automatic emotion detection is promising, and encourage further developments and studies of BSM-based methods

Supporting students’ confidence judgement through visualising alignment in open learner models

Supporting students’ knowledge monitoring skills, a component of metacognition, can help students regulate their own learning. This thesis investigates the alignment of learners’ confidence in their knowledge with a computer’s assessment of their knowledge, visualised using an Open Learner Model (OLM). The research explored students’ preferred method for visualising inconsistent data (e.g. misalignment) in an OLM, and the ways in which visualising alignment can influence student interaction with the computer. The thesis demonstrates that visualising alignment in Open Learner Models signifi-cantly increases students’ confidence compared to a control condition. In particular, visualising alignment benefited low-achieving students, in terms of knowledge monitoring and this was associated with improvements in their performance. Students showed a preference towards the visualisations that provides an overview of the in-formation (i.e. opacity) rather than ones, which provide detailed information. Graph-ical representation is shown to be more beneficial in motivating students to interact with the system than text-based representation of the same information in the con-text of representing the alignment within OLMs

Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

This paper presents an approach to flexible and adaptive dialogue management driven by cognitive modelling of human dialogue behaviour. Artificial intelligent agents, based on the ACT-R cognitive architecture, together with human actors are participating in a (meta)cognitive skills training within a negotiation scenario. The agent  employs instance-based learning to decide about its own actions and to reflect on the behaviour of the opponent. We show that task-related actions can be handled by a cognitive agent who is a plausible dialogue partner.  Separating task-related and dialogue control actions enables the application of sophisticated models along with a flexible architecture  in which  various alternative modelling methods can be combined. We evaluated the proposed approach with users assessing  the relative contribution of various factors to the overall usability of a dialogue system. Subjective perception of effectiveness, efficiency and satisfaction were correlated with various objective performance metrics, e.g. number of (in)appropriate system responses, recovery strategies, and interaction pace. It was observed that the dialogue system usability is determined most by the quality of agreements reached in terms of estimated Pareto optimality, by the user's negotiation strategies selected, and by the quality of system recognition, interpretation and responses. We compared human-human and human-agent performance with respect to the number and quality of agreements reached, estimated cooperativeness level, and frequency of accepted negative outcomes. Evaluation experiments showed promising, consistently positive results throughout the range of the relevant scales

Análisis de bases de datos de expresiones faciales para la identificación automática de emociones centradas en el aprendizaje

This work presents the analysis of the state of the art of facial expressions databases for the automatic identification of learning-centered emotions. Obtaining data for automatic recognition processes in a specific context is essential for their success. Thus, this project begins by reviewing the information available to carry out the training and classification stages of emotions with the proposed computational techniques. The search activities of the databases of facial expressions that capture learning-centered emotions are described. These activities were part of the stages of the work methodology to recognize students' emotions while they carried out online learning activities. This allowed justifying the creation of the database, formalizing a protocol from its capture to its digitization.Este trabajo presenta el análisis del estado del arte de bases de datos de expresiones faciales para la identificación automática de emociones centradas en el aprendizaje. La obtención de datos para los procesos de reconocimiento automático en un contexto específico es esencial para su éxito. Así, este tipo de proyectos inician haciendo una revisión de la información disponible para llevar a cabo las etapas de entrenamiento y clasificación de las emociones con las técnicas computacionales que se propongan. Se describen las actividades de búsqueda de las bases de datos de expresiones faciales que capturan emociones centradas en el aprendizaje. Estas actividades formaron parte de las etapas de la metodología del trabajo para reconocer las emociones de estudiantes mientras realizaban actividades de aprendizaje en línea. Esto permitió justificar la creación de la base de datos desde la formalización de un protocolo para su captura hasta su digitalización

Conditions and the effects of an intelligent tutoring system usage for Russian high-stakes exam in English

The aim of the proposed study was to dwell on the field of intelligent tutoring systems as applied to high-stakes exam settings in foreign languages. The main research hypothesis of this paper was the following: Does the study attempt frequency within the suggested intelligent tutoring system affect the overall students’ learning performance in preparation for the Speaking part of the Russian high-stakes exam in the English language? Addressing this research hypothesis also resulted in acquiring understanding on key stakeholders’ perception of preparation for the Russian high-stakes exam in English. Research literature was thoroughly analyzed and the suggested intelligent system was described in detail. Data was collected through a computer-based automated procedure with further randomization and sampling. As a result of the study, three cohorts of users of the intelligent tutoring system were defined. Each cohort maintained a positive study dynamics experienced through the use of the intelligent tutoring system. Also, continuous aspiration for implementing online self-training environments was identified within the majority of a foreign language teachers’ community. The framework developed for the research can be used in future research as a foundation for investigating self-regulated learning environments created for the Speaking part preparation of high-stakes exam in foreign languages

Cognitive architecture of multimodal multidimensional dialogue management

Numerous studies show that participants of real-life dialogues happen to get involved in rather dynamic non-sequential interactions. This challenges the dialogue system designs based on a reactive interlocutor paradigm and calls for dialog systems that can be characterised as a proactive learner, accomplished multitasking planner and adaptive decision maker. Addressing this call, the thesis brings innovative integration of cognitive models into the human-computer dialogue systems. This work utilises recent advances in Instance-Based Learning of Theory of Mind skills and the established Cognitive Task Analysis and ACT-R models. Cognitive Task Agents, producing detailed simulation of human learning, prediction, adaption and decision making, are integrated in the multi-agent Dialogue Man-ager. The manager operates on the multidimensional information state enriched with representations based on domain- and modality-specific semantics and performs context-driven dialogue acts interpretation and generation. The flexible technical framework for modular distributed dialogue system integration is designed and tested. The implemented multitasking Interactive Cognitive Tutor is evaluated as showing human-like proactive and adaptive behaviour in setting goals, choosing appropriate strategies and monitoring processes across contexts, and encouraging the user exhibit similar metacognitive competences

Enhanced Living Environments

This open access book was prepared as a Final Publication of the COST Action IC1303 “Algorithms, Architectures and Platforms for Enhanced Living Environments (AAPELE)”. The concept of Enhanced Living Environments (ELE) refers to the area of Ambient Assisted Living (AAL) that is more related with Information and Communication Technologies (ICT). Effective ELE solutions require appropriate ICT algorithms, architectures, platforms, and systems, having in view the advance of science and technology in this area and the development of new and innovative solutions that can provide improvements in the quality of life for people in their homes and can reduce the financial burden on the budgets of the healthcare providers. The aim of this book is to become a state-of-the-art reference, discussing progress made, as well as prompting future directions on theories, practices, standards, and strategies related to the ELE area. The book contains 12 chapters and can serve as a valuable reference for undergraduate students, post-graduate students, educators, faculty members, researchers, engineers, medical doctors, healthcare organizations, insurance companies, and research strategists working in this area