Assessing the Utility of Deep Learning: Using Learner-System Interaction Data from BioWorld

In recent years, deep learning (LeCun, Bengio, & Hinton, 2015) has drawn interest in many fields. As optimism for deep learning grows, a better understanding of the efficacy of deep learning is imperative, especially in analyzing and making sense of educational data. This study addresses this issue by establishing a benchmark for a common prediction task – student proficiency in diagnosing patient diseases in a system called BioWorld (Lajoie, 2009). To do so, we compared deep learning to existing solutions, including traditional machine learning algorithms that are commonly used in educational data mining. The dataset consists of log interaction data collected from 30 medical students solving 3 different cases. A 10-fold cross-validation method was used to evaluate the predictive accuracy of each model. Interestingly, our results indicate that deep learning does not outperform traditional machine learning algorithms in predicting diagnosis correctness. We discuss the implications in terms of understanding the proper conditions for its use in educational research

Mitigating Biases in Student Performance Prediction via Attention-Based Personalized Federated Learning

Traditional learning-based approaches to student modeling generalize poorly to underrepresented student groups due to biases in data availability. In this paper, we propose a methodology for predicting student performance from their online learning activities that optimizes inference accuracy over different demographic groups such as race and gender. Building upon recent foundations in federated learning, in our approach, personalized models for individual student subgroups are derived from a global model aggregated across all student models via meta-gradient updates that account for subgroup heterogeneity. To learn better representations of student activity, we augment our approach with a self-supervised behavioral pretraining methodology that leverages multiple modalities of student behavior (e.g., visits to lecture videos and participation on forums), and include a neural network attention mechanism in the model aggregation stage. Through experiments on three real-world datasets from online courses, we demonstrate that our approach obtains substantial improvements over existing student modeling baselines in predicting student learning outcomes for all subgroups. Visual analysis of the resulting student embeddings confirm that our personalization methodology indeed identifies different activity patterns within different subgroups, consistent with its stronger inference ability compared with the baselines.Comment: 10 pages, CIKM 202

An Intra-Subject Approach Based on the Application of HMM to Predict Concentration in Educational Contexts from Nonintrusive Physiological Signals in Real-World Situations

Previous research has proven the strong influence of emotions on student engagement and motivation. Therefore, emotion recognition is becoming very relevant in educational scenarios, but there is no standard method for predicting students’ affects. However, physiological signals have been widely used in educational contexts. Some physiological signals have shown a high accuracy in detecting emotions because they reflect spontaneous affect-related information, which is fresh and does not require additional control or interpretation. Most proposed works use measuring equipment for which applicability in real-world scenarios is limited because of its high cost and intrusiveness. To tackle this problem, in this work, we analyse the feasibility of developing low-cost and nonintrusive devices to obtain a high detection accuracy from easy-to-capture signals. By using both inter-subject and intra-subject models, we present an experimental study that aims to explore the potential application of Hidden Markov Models (HMM) to predict the concentration state from 4 commonly used physiological signals, namely heart rate, breath rate, skin conductance and skin temperature. We also study the effect of combining these four signals and analyse their potential use in an educational context in terms of intrusiveness, cost and accuracy. The results show that a high accuracy can be achieved with three of the signals when using HMM-based intra-subject models. However, inter-subject models, which are meant to obtain subject-independent approaches for affect detection, fail at the same task.This research was partly supported by Spanish Ministry of Science, Innovation and Universities through projects PGC2018-096463-B-I00 and PGC2018-102279-B-I00 (MCIU/AEI/FEDER, UE)

Análisis de bases de datos de expresiones faciales para la identificación automática de emociones centradas en el aprendizaje

This work presents the analysis of the state of the art of facial expressions databases for the automatic identification of learning-centered emotions. Obtaining data for automatic recognition processes in a specific context is essential for their success. Thus, this project begins by reviewing the information available to carry out the training and classification stages of emotions with the proposed computational techniques. The search activities of the databases of facial expressions that capture learning-centered emotions are described. These activities were part of the stages of the work methodology to recognize students' emotions while they carried out online learning activities. This allowed justifying the creation of the database, formalizing a protocol from its capture to its digitization.Este trabajo presenta el análisis del estado del arte de bases de datos de expresiones faciales para la identificación automática de emociones centradas en el aprendizaje. La obtención de datos para los procesos de reconocimiento automático en un contexto específico es esencial para su éxito. Así, este tipo de proyectos inician haciendo una revisión de la información disponible para llevar a cabo las etapas de entrenamiento y clasificación de las emociones con las técnicas computacionales que se propongan. Se describen las actividades de búsqueda de las bases de datos de expresiones faciales que capturan emociones centradas en el aprendizaje. Estas actividades formaron parte de las etapas de la metodología del trabajo para reconocer las emociones de estudiantes mientras realizaban actividades de aprendizaje en línea. Esto permitió justificar la creación de la base de datos desde la formalización de un protocolo para su captura hasta su digitalización

Deep Learning use in recomendation systems to reduce the desertion in Colombian High Education

El anteproyecto de monografía planteado aquí, busca encontrar cómo la implementación de una herramienta de Deep Learning puede ser empleada para mitigar de forma efectiva, la deserción en Instituciones de educación superior, al reducir el tiempo de detección y recomendación de apoyos necesarios para que los alumnos puedan continuar su proceso educativo. El gobierno colombiano es consciente de la problemática que envuelve la deserción en educación superior, y por esto, ha generado iniciativas que buscan bajar las cifras del problema, pero en la actualidad los esfuerzos no han tenido el resultado esperado. La investigación se centrará en determinar la viabilidad de implementación de la herramienta en las plataformas en la nube con reconocimiento como las de mayor capacidad y completitud de su servicio, y en determinar si las Instituciones de Educación Superior tienen la capacidad de generar la información que permita alimentarla y la posibilidad de mantener el costo de una herramienta implementada en una plataforma en la nube.The preliminary monograph proposed here seeks to find how the implementation of a Deep Learning tool can effectively mitigate attrition in Higher Education Institutions by reducing the time of detection and recommendation of supports necessary for students to can continue their educational process. The Colombian government is aware of the problems surrounding higher education dropouts. For this reason, it has generated initiatives that seek to lower the numbers of the problem, but at present, the efforts have not had the expected result. The research will focus on determining the feasibility of implementing the tool on cloud platforms recognized as those with the greatest capacity and completeness of their service and on determining whether Higher Education Institutions have the capacity to generate the information that allows power it and the possibility of maintaining the cost of a tool deployed on a cloud platform.Magíster en Inteligencia de Negocio

Inferência de estados afetivos em ambientes educacionais : proposta de um modelo híbrido baseado em informações cognitivas e físicas

Orientador: Prof. Dr. Andrey Ricardo PimentelDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 11/12/2018Inclui referências: p.119-127Área de concentração: Ciência da ComputaçãoResumo: Na comunidade científica é comum o entendimento de que os softwares educacionais precisam evoluir para garantir um suporte mais efetivo ao processo de aprendizagem. Uma limitação recorrente destes softwares refere-se à falta de funcionalidades de adaptação às reações afetivas dos estudantes. Esta limitação torna-se relevante pois as emoções têm influencia direta no processo de aprendizagem. Reconhecer as emoções dos estudantes é o primeiro passo em direção a construção de software educativos sensíveis ao afeto. Trabalhos correlatos reportam relativo sucesso na tarefa de reconhecimento automático das emoções dos estudantes. No entanto, grande parte dos trabalhos correlatos utiliza sensores pouco práticos, intrusivos e caros que normalmente monitoram apenas reações físicas. No contexto educacional o conjunto de emoções a ser considerado no processo de reconhecimento deve observar as singularidades deste domínio. Sendo assim, neste trabalho a inferência é realizada utilizando uma abordagem de quadrantes formadas pelas dimensões valência e ativação. Estes quadrantes representam situações relevantes para a aprendizagem e podem ser utilizados para embasar adaptações no ambiente computacional. Diante disto, esta pesquisa apresenta a proposta de um modelo híbrido de inferência de emoções de estudantes durante o uso softwares educacionais. Este modelo tem como principal característica a utilização simultânea de informações oriundas de reações físicas (expressões faciais) e cognitivas (eventos no software educacional). Esta abordagem fundamenta-se na perspectiva teórica de que as emoções humanas são fortemente relacionadas com reações físicas, mas também são influenciadas por processos racionais ou cognitivos. A combinação de expressões faciais e informações sobre os eventos do software educacional permite a construção de uma solução de baixo custo e intrusividade. Além disso, esta solução apresenta viabilidade de utilização em larga escala e em ambientes reais de ensino. Experimentos realizados com estudantes em um ambiente real de ensino demonstraram a viabilidade desta proposta. Este fato é importante, considerando-se que a abordagem proposta neste trabalho é pouco explorada na comunidade científica e requer a fusão de informações bastante distintas. Nestes experimentos, foram obtidas acurácia e índice Cohen Kappa próximas de 66% e 0,55, respectivamente, na tarefa de inferência de cinco classes de emoções. Embora esses resultados sejam promissores quando comparados a trabalhos correlatos, entende-se que eles podem ser aprimorados no futuro, incorporando-se novos dados ao modelo proposto. Palavras-chave: Inferência de Emoção, Emoção Relacionada à Aprendizagem, Tutoria Afetiva, Computação Afetiva.Abstract: In the scientific community there is a common understanding that educational software must evolve to ensure more effective support to the learning process. A common limitation of these software refers to the lack of adaptive features to students' affective reactions. This limitation becomes relevant because the emotions have a direct influence on the learning process. Recognizing students' emotions is the first step toward building affect-sensitive educational software. Related work reports relatively successful in the task of automatically recognize students' emotions. However, most studies use impractical, intrusive and expensive sensors that typically monitor only physical reactions. In the educational context the set of emotions to be considered in the recognition process must observe the singularities of this domain. Thus, in this work the inference is performed using a quadrant approach formed by the valence and activation dimensions. These quadrants represent situations relevant to learning and can be used to support adaptations in the computational environment. So, this research presents a proposal of a hybrid model to infer emotions of students while using educational software. This model has as its main feature the simultaneous use of information coming from physical reactions (facial expressions) and cognitive (events in the educational software). This approach is based on the theoretical perspective that human emotions are strongly related with physical reactions, but are also influenced by rational or cognitive processes. Combining facial expressions and information about the events of educational software allows the construction of a low-cost and intrusiveness solution. In addition, this solution presents feasibility for use in large scale in real learning environments. Experiments with students in a real classroom demonstrated the feasibility of this proposal. This is important, considering that the approach proposed in this work is little explored in the scientific community and requires the fusion of quite different information. In these experiments, accuracy and Cohen Kappa index close to 66% and 0,55, respectively, were obtained in the inference of five emotion classes. Although these results are promising when compared to related works, it is understood that they can be improved in the future by incorporating new data into the proposed model. Keywords: Emotion inference, Learning related emotion, Affective tutoring, Affective Computing

The Upstream Sources Of Bias: Investigating Theory, Design, And Methods Shaping Adaptive Learning Systems

Adaptive systems in education need to ensure population validity to meet the needs of all students for an equitable outcome. Recent research highlights how these systems encode societal biases leading to discriminatory behaviors towards specific student subpopulations. However, the focus has mostly been on investigating bias in predictive modeling, particularly its downstream stages like model development and evaluation. My dissertation work hypothesizes that the upstream sources (i.e., theory, design, training data collection method) in the development of adaptive systems also contribute to the bias in these systems, highlighting the need for a nuanced approach to conducting fairness research. By empirically analyzing student data previously collected from various virtual learning environments, I investigate demographic disparities in three cases representative of the aspects that shape technological advancements in education: 1) non-conformance of data to a widely-accepted theoretical model of emotion, 2) differing implications of technology design on student outcomes, and 3) varying effectiveness of methodological improvements in annotated data collection. In doing so, I challenge implicit assumptions of generalizability in theory, design, and methods and provide an evidence-based commentary on future research and design practices in adaptive and artificially intelligent educational systems surrounding how we consider diversity in our investigations