Socio-Cognitive and Affective Computing

Social cognition focuses on how people process, store, and apply information about other people and social situations. It focuses on the role that cognitive processes play in social interactions. On the other hand, the term cognitive computing is generally used to refer to new hardware and/or software that mimics the functioning of the human brain and helps to improve human decision-making. In this sense, it is a type of computing with the goal of discovering more accurate models of how the human brain/mind senses, reasons, and responds to stimuli. Socio-Cognitive Computing should be understood as a set of theoretical interdisciplinary frameworks, methodologies, methods and hardware/software tools to model how the human brain mediates social interactions. In addition, Affective Computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human affects, a fundamental aspect of socio-cognitive neuroscience. It is an interdisciplinary field spanning computer science, electrical engineering, psychology, and cognitive science. Physiological Computing is a category of technology in which electrophysiological data recorded directly from human activity are used to interface with a computing device. This technology becomes even more relevant when computing can be integrated pervasively in everyday life environments. Thus, Socio-Cognitive and Affective Computing systems should be able to adapt their behavior according to the Physiological Computing paradigm. This book integrates proposals from researchers who use signals from the brain and/or body to infer people's intentions and psychological state in smart computing systems. The design of this kind of systems combines knowledge and methods of ubiquitous and pervasive computing, as well as physiological data measurement and processing, with those of socio-cognitive and affective computing

The Multimodal Tutor: Adaptive Feedback from Multimodal Experiences

This doctoral thesis describes the journey of ideation, prototyping and empirical testing of the Multimodal Tutor, a system designed for providing digital feedback that supports psychomotor skills acquisition using learning and multimodal data capturing. The feedback is given in real-time with machine-driven assessment of the learner's task execution. The predictions are tailored by supervised machine learning models trained with human annotated samples. The main contributions of this thesis are: a literature survey on multimodal data for learning, a conceptual model (the Multimodal Learning Analytics Model), a technological framework (the Multimodal Pipeline), a data annotation tool (the Visual Inspection Tool) and a case study in Cardiopulmonary Resuscitation training (CPR Tutor). The CPR Tutor generates real-time, adaptive feedback using kinematic and myographic data and neural networks

Explainable machine learning for educational data

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonEducational repositories contain complex trajectories of students and university data. Being able to model this data would offer great value in being able to identify students’ trajectories, predicting their likely future performance, and identifying those who require appropriate intervention as early as possible. However, understanding the nature of the correlations and the dependencies among the educational attributes (which can be time-dependant non-linear relationships) is fundamental for the learning of robust predictive classifiers. When predicting academic performance, many machine learning algorithms make decisions based on data that can be imbalanced, badly sampled, or biased based on historical societal prejudices. In this thesis, I explore, implement, and evaluate temporal predictive classifiers that aim to overcome some of these issues. The approach combines time-series clustering in conjunction with probabilistic learning, resampling, feature subspace learning, and specialist deep learning methods to learn models that are simultaneously accurate and unbiased. A key technical objective in learning these classifiers is to incorporate different types of temporal performance data collected at different times (student admission to a higher education institution, and at Year 1 and 2 of a student’s studies), for the explicit modelling of cognitive styles. A resampling method is applied with bootstrap aggregating to address the issue of the imbalanced time-series educational datasets, which is related to miss-classifying the minority-class of the high-risk or failing students. The evaluation of an unsupervised subspace learning approach using an Autoassociative Neural Network (Autoencoder) is also made, to reconstruct the educational data by maximising variance for improved performance prediction. In addition, the issues of modelling bias are explored such that the types are identified and whether they are accounting for inflated predictive accuracies is established. A graphical learning approach with a BN, that is transparent in how they make decisions, is compared with three forms of Deep Multi-label Convolutional Neural Network (CNNs) to investigate whether deep learning classifiers can be learned that maximise accuracy and minimise bias. The evaluation of the experimental results reveals that identifying cognitive styles improves both explanation and accuracy; that rebalancing also improves accuracy, and that a combination of probabilistic modelling and deep 1D Multi-label CNN can successfully identify and eliminate many biases when predicting student’s academic performance.Saudi Arabian Cultural Bureau in the United Kingdo

Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief

Comprend des références bibliographiquesIn data mining and data analytics, tools and techniques once confined to research laboratories are being adopted by forward-looking industries to improve decision making. Higher education institutions are beginning to use analytics for improving the services they provide and for increasing student grades and retention. The U.S. Department of Education’s National Education Technology Plan, as one part of its model for learning powered by technology, envisions ways of using data from online learning systems to improve instruction. With analytics and data mining experiments in education starting to proliferate, sorting out fact from fiction and identifying research possibilities and practical applications are not easy. This issue brief is intended to help policymakers and administrators understand how analytics and data mining have been - and can be - applied for educational improvement while rigorously protecting student privacy

Modeling Learner Mood In Realtime Through Biosensors For Intelligent Tutoring Improvements

Computer-based instructors, just like their human counterparts, should monitor the emotional and cognitive states of their students in order to adapt instructional technique. Doing so requires a model of student state to be available at run time, but this has historically been difficult. Because people are different, generalized models have not been able to be validated. As a person’s cognitive and affective state vary over time of day and seasonally, individualized models have had differing difficulties. The simultaneous creation and execution of an individualized model, in real time, represents the last option for modeling such cognitive and affective states. This dissertation presents and evaluates four differing techniques for the creation of cognitive and affective models that are created on-line and in real time for each individual user as alternatives to generalized models. Each of these techniques involves making predictions and modifications to the model in real time, addressing the real time datastream problems of infinite length, detection of new concepts, and responding to how concepts change over time. Additionally, with the knowledge that a user is physically present, this work investigates the contribution that the occasional direct user query can add to the overall quality of such models. The research described in this dissertation finds that the creation of a reasonable quality affective model is possible with an infinitesimal amount of time and without “ground truth” knowledge of the user, which is shown across three different emotional states. Creation of a cognitive model in the same fashion, however, was not possible via direct AI modeling, even with all of the “ground truth” information available, which is shown across four different cognitive states