Affective Environment for Java Programming Using Facial and EEG Recognition

Abstract. We have developed an affective and intelligent learning environment that helps students to improve their Java programming skills. This environment evaluates cognitive and affective aspects of students in order to define the level of difficulty of the exercises that are more suitable for the them in its current condition. The cognitive aspects are: the number of mistakes, the difficulty level of the current exercise and the time spent in the solution. The affective aspects are: the acquired emotion from a facial expression and the acquired valence from electroencephalogram signals. This environment also uses a neural network for face recognition of basic emotions, a support vector machine to define the valence of emotion and a fuzzy inference engine to evaluate the cognitive and affective aspects

Affective e-learning approaches, technology and implementation model: a systematic review

A systematic literature study including articles from 2016 to 2022 was done to evaluate the various approaches, technologies, and implementation models involved in measuring student engagement during learning. The review’s objective was to compile and analyze all studies that investigated how instructors can gauge students’ mental states while teaching and assess the most effective teaching methods. Additionally, it aims to extract and assess expanded methodologies from chosen research publications to offer suggestions and answers to researchers and practitioners. Planning, carrying out the analysis, and publishing the results have all received significant attention in the research approach. The study’s findings indicate that more needs to be done to evaluate student participation objectively and follow their development for improved academic performance. Physiological approaches should be given more support among the alternatives. While deep learning implementation models and contactless technology should interest more researchers. And, the recommender system should be integrated into e-learning system. Other approaches, technologies, and methodology articles, on the other hand, lacked authenticity in conveying student feeling

The Possibilities of Classification of Emotional States Based on User Behavioral Characteristics

The classification of user's emotions based on their behavioral characteristic, namely their keyboard typing and mouse usage pattern is an effective and non-invasive way of gathering user's data without imposing any limitations on their ability to perform tasks. To gather data for the classifier we used an application, the Emotnizer, which we had developed for this purpose. The output of the classification is categorized into 4 emotional categories from Russel's complex circular model - happiness, anger, sadness and the state of relaxation. The sample of the reference database consisted of 50 students. Multiple regression analyses gave us a model, that allowed us to predict the valence and arousal of the subject based on the input from the keyboard and mouse. Upon re-testing with another test group of 50 students and processing the data we found out our Emotnizer program can classify emotional states with an average success rate of 82.31%

Pushing the boundaries of EEG-based emotion classification using consumer-grade wearable brain-computer interfacing devices and ensemble classifiers

Emotion classification using features derived from electroencephalography (EEG) is currently one of the major research areas in big data. Although this area of research is not new, the current challenge is now to move from medical-grade EEG acquisition devices to consumer-grade EEG devices. The overwhelmingly large majority of reported studies that have achieved high success rates in such research uses equipment that is beyond the reach of the everyday consumer. Subsequently, EEG-based emotion classification applications, though highly promising and worthwhile to research, largely remain as academic research and not as deployable solutions. In this study, we attempt to use consumer-grade EEG devices commonly referred to as wearable EEG devices that are very economical in cost but have a limited number of sensor electrodes as well as limited signal resolution. Hence, this greatly reduces the number and quality of available EEG signals that can be used as classification features. Additionally, we also attempt to classify into 4 distinct classes as opposed to the more common 2 or 3 class emotion classification task. Moreover, we also additionally attempt to conduct inter-subject classification rather than just intra-subject classification, which again the former is much more challenging than the latter. Using a test cohort of 31 users with stimuli presented via an immersive virtual reality environment, we present results that show that classification accuracies were able to be pushed to beyond 85% using ensemble classification methods in the form of Random Forest

A conceptual framework for an affective tutoring system using unobtrusive affect sensing for enhanced tutoring outcomes

PhD ThesisAffect plays a pivotal role in influencing the student’s motivation and learning achievements. The ability of expert human tutors to achieve enhanced learning outcomes is widely attributed to their ability to sense the affect of their tutees and to continually adapt their tutoring strategies in response to the dynamically changing affect throughout the tutoring session. In this thesis, I explore the feasibility of building an Affective Tutoring System (ATS) which senses the student’s affect on a moment-to-moment basis with the use of unobtrusive sensors in the context of computer programming tutoring. The novel use of keystrokes and mouse clicks for affect sensing is proposed here as they are ubiquitous and unobtrusive. I first establish the viability of using keystrokes and contextual logs for affect sensing first on a per exercise session level and then on a more granular basis of 30 seconds. Subsequently, I move on to investigate the use of multiple sensing channels e.g. facial, keystrokes, mouse clicks, contextual logs and head postures to enhance the availability and accuracy of sensing. The results indicated that it is viable to use keystrokes for affect sensing. In addition, the combination of multiple sensor modes enhances the accuracy of affect sensing. From the results, the sensor modes that are most significant for affect sensing are the head postures and facial modes. Nevertheless, keystrokes make up for the periods of unavailability of the former. With the affect sensing (both sensing of frustration and disengagement) in place, I moved on to architect and design the ATS and conducted an experimental study and a series of focus group discussions to evaluate the ATS. The results showed that the ATS is rated positively by the participants for usability and acceptance. The ATS is also effective in enhancing the learning of the studentsNanyang Polytechni

Leveraging Targeted Regions of Interest by Analyzing Code Comprehension With AI-Enabled Eye-Tracking

Code comprehension studies techniques for extracting information that give insights on how code is understood. For educators teaching programming courses, this is an important but often difficult task, especially given the challenges of large class sizes, limited time, and grading resources. By analyzing where a student looks during a code comprehension task, instructors can gain insights into what information the student deems important and assess whether they are looking in the right areas of the code. The proportion of time spent viewing a part of the code is also a useful indicator of the student\u27s decision-making process. The goal of this research is to analyze the differences in how students\u27 eyes traverse across code during coding comprehension activities and to offer a systematic way for distinguishing students with a solid understanding of the exercise from those who require further assistance. The study uses coding exercises seeded with errors, measured fixation counts, and average fixation durations of the students\u27 eyes within targeted regions of interest (TROI) using an AI-Enabled Eye-Tracking System (NiCATS). The results of the study showed that students\u27 grades (as a proxy for understanding of the code\u27s context and their decision-making skills) were positively correlated with a higher ratio of the number of fixations in the TROI

Affective modelling and feedback in programming practice systems

指導教員：角　

Socio-Cognitive and Affective Computing

Social cognition focuses on how people process, store, and apply information about other people and social situations. It focuses on the role that cognitive processes play in social interactions. On the other hand, the term cognitive computing is generally used to refer to new hardware and/or software that mimics the functioning of the human brain and helps to improve human decision-making. In this sense, it is a type of computing with the goal of discovering more accurate models of how the human brain/mind senses, reasons, and responds to stimuli. Socio-Cognitive Computing should be understood as a set of theoretical interdisciplinary frameworks, methodologies, methods and hardware/software tools to model how the human brain mediates social interactions. In addition, Affective Computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human affects, a fundamental aspect of socio-cognitive neuroscience. It is an interdisciplinary field spanning computer science, electrical engineering, psychology, and cognitive science. Physiological Computing is a category of technology in which electrophysiological data recorded directly from human activity are used to interface with a computing device. This technology becomes even more relevant when computing can be integrated pervasively in everyday life environments. Thus, Socio-Cognitive and Affective Computing systems should be able to adapt their behavior according to the Physiological Computing paradigm. This book integrates proposals from researchers who use signals from the brain and/or body to infer people's intentions and psychological state in smart computing systems. The design of this kind of systems combines knowledge and methods of ubiquitous and pervasive computing, as well as physiological data measurement and processing, with those of socio-cognitive and affective computing