1,595 research outputs found
Leveraging Affect Transfer Learning for Behavior Prediction in an Intelligent Tutoring System
In the context of building an intelligent tutoring system (ITS), which
improves student learning outcomes by intervention, we set out to improve
prediction of student problem outcome. In essence, we want to predict the
outcome of a student answering a problem in an ITS from a video feed by
analyzing their face and gestures. For this, we present a novel transfer
learning facial affect representation and a user-personalized training scheme
that unlocks the potential of this representation. We model the temporal
structure of video sequences of students solving math problems using a
recurrent neural network architecture. Additionally, we extend the largest
dataset of student interactions with an intelligent online math tutor by a
factor of two. Our final model, coined ATL-BP (Affect Transfer Learning for
Behavior Prediction) achieves an increase in mean F-score over state-of-the-art
of 45% on this new dataset in the general case and 50% in a more challenging
leave-users-out experimental setting when we use a user-personalized training
scheme
ATL-BP: a student engagement dataset and model for affect transfer learning for behavior prediction
IIS-1551572 - National Science Foundationhttp://10.0.4.85/TBIOM.2022.3210479Published versio
A Review of Data Mining in Personalized Education: Current Trends and Future Prospects
Personalized education, tailored to individual student needs, leverages
educational technology and artificial intelligence (AI) in the digital age to
enhance learning effectiveness. The integration of AI in educational platforms
provides insights into academic performance, learning preferences, and
behaviors, optimizing the personal learning process. Driven by data mining
techniques, it not only benefits students but also provides educators and
institutions with tools to craft customized learning experiences. To offer a
comprehensive review of recent advancements in personalized educational data
mining, this paper focuses on four primary scenarios: educational
recommendation, cognitive diagnosis, knowledge tracing, and learning analysis.
This paper presents a structured taxonomy for each area, compiles commonly used
datasets, and identifies future research directions, emphasizing the role of
data mining in enhancing personalized education and paving the way for future
exploration and innovation.Comment: 25 pages, 5 figure
Personalized face and gesture analysis using hierarchical neural networks
The video-based computational analyses of human face and gesture signals encompass a myriad of challenging research problems involving computer vision, machine learning and human computer interaction. In this thesis, we focus on the following challenges: a) the classification of hand and body gestures along with the temporal localization of their occurrence in a continuous stream, b) the recognition of facial expressivity levels in people with Parkinson's Disease using multimodal feature representations, c) the prediction of student learning outcomes in intelligent tutoring systems using affect signals, and d) the personalization of machine learning models, which can adapt to subject and group-specific nuances in facial and gestural behavior. Specifically, we first conduct a quantitative comparison of two approaches to the problem of segmenting and classifying gestures on two benchmark gesture datasets: a method that simultaneously segments and classifies gestures versus a cascaded method that performs the tasks sequentially. Second, we introduce a framework that computationally predicts an accurate score for facial expressivity and validate it on a dataset of interview videos of people with Parkinson's disease. Third, based on a unique dataset of videos of students interacting with MathSpring, an intelligent tutoring system, collected by our collaborative research team, we build models to predict learning outcomes from their facial affect signals. Finally, we propose a novel solution to a relatively unexplored area in automatic face and gesture analysis research: personalization of models to individuals and groups. We develop hierarchical Bayesian neural networks to overcome the challenges posed by group or subject-specific variations in face and gesture signals. We successfully validate our formulation on the problems of personalized subject-specific gesture classification, context-specific facial expressivity recognition and student-specific learning outcome prediction. We demonstrate the flexibility of our hierarchical framework by validating the utility of both fully connected and recurrent neural architectures
Recommended from our members
The role of machine learning in personalised instructional sequencing for language learning
The origins of personalised instructional sequencing can be dated back to the times of the Ancient Greeks to the times of Alexander The Great's tutor, Aristotle. However, over the centuries the demand for education and growth of students has been disproportionately greater than the number of teachers in training. Therefore, there has been a longstanding interest in finding a way to scale education without negatively affecting learning outcomes. This interest was fuelled further with the advent of computers and artificial intelligence, where a plethora of systems and models were built to bring technology driven personalised instructional sequencing to the world. Unfortunately, results were far from groundbreaking and many challenges still remain.
In my thesis, I investigate three aspects of personalised instructional sequencing: the personalised instructional sequencing mechanism, the student knowledge representation, and human forgetting. While I do not cover the entirety of personalised instructional sequencing, I cover what I consider the foundational components. I link psychological theory to model selection and design in each of my systems and present experiments to illustrate their impact. I show how reinforcement learning can be used for vocabulary learning. I also present a model that uses neural collaborative filtering to learn student knowledge representations. Lastly, I present a state-of-the-art model to predict the probability of vocabulary word recall for students learning English as a second language. The system's novelty lies in the use of word complexity to adapt the forgetting curve as well as its incorporation of psychological theory to select an appropriate model
Deep Reinforcement Learning Approaches for Technology Enhanced Learning
Artificial Intelligence (AI) has advanced significantly in recent years, transforming various industries and domains. Its ability to extract patterns and insights from large volumes of data has revolutionised areas such as image recognition, natural language processing, and autonomous systems. As AI systems become increasingly integrated into daily human life, there is a growing need for meaningful collaboration and mutual engagement between humans and AI, known as Human-AI Collaboration. This collaboration involves combining AI with human workflows to achieve shared objectives.
In the current educational landscape, the integration of AI methods in Technology Enhanced Learning (TEL) has become crucial for providing high-quality education and facilitating lifelong learning. Human-AI Collaboration also plays a vital role in the field of Technology Enhanced Learning (TEL), particularly in Intelligent Tutoring Systems (ITS). The COVID-19 pandemic has further emphasised the need for effective educational technologies to support remote learning and bridge the gap between traditional classrooms and online platforms. To maximise the performance of ITS while minimising the input and interaction required from students, it is essential to design collaborative systems that effectively leverage the capabilities of AI and foster effective collaboration between students and ITS.
However, there are several challenges that need to be addressed in this context. One challenge is the lack of clear guidance on designing and building user-friendly systems that facilitate collaboration between humans and AI. This challenge is relevant not only to education researchers but also to Human-Computer Interaction (HCI) researchers and developers. Another challenge is the scarcity of interaction data in the early stages of ITS development, which hampers the accurate modelling of students' knowledge states and learning trajectories, known as the cold start problem. Moreover, the effectiveness of Intelligent Tutoring Systems (ITS) in delivering personalised instruction is hindered by the limitations of existing Knowledge Tracing (KT) models, which often struggle to provide accurate predictions. Therefore, addressing these challenges is crucial for enhancing the collaborative process between humans and AI in the development of ITS.
This thesis aims to address these challenges and improve the collaborative process between students and ITS in TEL. It proposes innovative approaches to generate simulated student behavioural data and enhance the performance of KT models. The thesis starts with a comprehensive survey of human-AI collaborative systems, identifying key challenges and opportunities. It then presents a structured framework for the student-ITS collaborative process, providing insights into designing user-friendly and efficient systems.
To overcome the challenge of data scarcity in ITS development, the thesis proposes two student modelling approaches: Sim-GAIL and SimStu. SimStu leverages a deep learning method, the Decision Transformer, to simulate student interactions and enhance ITS training. Sim-GAIL utilises a reinforcement learning method, Generative Adversarial Imitation Learning (GAIL), to generate high-fidelity and diverse simulated student behavioural data, addressing the cold start problem in ITS training.
Furthermore, the thesis focuses on improving the performance of KT models. It introduces the MLFBKT model, which integrates multiple features and mines latent relations in student interaction data, aiming to improve the accuracy and efficiency of KT models. Additionally, the thesis proposes the LBKT model, which combines the strengths of the BERT model and LSTM to process long sequence data in KT models effectively.
Overall, this thesis contributes to the field of Human-AI collaboration in TEL by addressing key challenges and proposing innovative approaches to enhance ITS training and KT model performance. The findings have the potential to improve the learning experiences and outcomes of students in educational settings
Challenging Social Media Threats using Collective Well-being Aware Recommendation Algorithms and an Educational Virtual Companion
Social media (SM) have become an integral part of our lives, expanding our
inter-linking capabilities to new levels. There is plenty to be said about
their positive effects. On the other hand however, some serious negative
implications of SM have repeatedly been highlighted in recent years, pointing
at various SM threats for society, and its teenagers in particular: from common
issues (e.g. digital addiction and polarization) and manipulative influences of
algorithms to teenager-specific issues (e.g. body stereotyping). The full
impact of current SM platform design -- both at an individual and societal
level -- asks for a comprehensive evaluation and conceptual improvement. We
extend measures of Collective Well-Being (CWB) to SM communities. As users'
relationships and interactions are a central component of CWB, education is
crucial to improve CWB. We thus propose a framework based on an adaptive
"social media virtual companion" for educating and supporting the entire
students' community to interact with SM. The virtual companion will be powered
by a Recommender System (CWB-RS) that will optimize a CWB metric instead of
engagement or platform profit, which currently largely drives recommender
systems thereby disregarding any societal collateral effect. CWB-RS will
optimize CWB both in the short term, by balancing the level of SM threat the
students are exposed to, as well as in the long term, by adopting an
Intelligent Tutor System role and enabling adaptive and personalized sequencing
of playful learning activities. This framework offers an initial step on
understanding how to design SM systems and embedded educational interventions
that favor a more healthy and positive society
Explicit Feedback Within Game-based Training: Examining The Influence Of Source Modality Effects On Interaction
This research aims to enhance Simulation-Based Training (SBT) applications to support training events in the absence of live instruction. The overarching purpose is to explore available tools for integrating intelligent tutoring communications in game-based learning platforms and to examine theory-based techniques for delivering explicit feedback in such environments. The primary tool influencing the design of this research was the Generalized Intelligent Framework for Tutoring (GIFT), a modular domain-independent architecture that provides the tools and methods to author, deliver, and evaluate intelligent tutoring technologies within any training platform. Influenced by research surrounding Social Cognitive Theory and Cognitive Load Theory, the resulting experiment tested varying approaches for utilizing an Embodied Pedagogical Agent (EPA) to function as a tutor during interaction in a game-based environment. Conditions were authored to assess the tradeoffs between embedding an EPA directly in a game, embedding an EPA in GIFT’s browser-based Tutor-User Interface (TUI), or using audio prompts alone with no social grounding. The resulting data supports the application of using an EPA embedded in GIFT’s TUI to provide explicit feedback during a game-based learning event. Analyses revealed conditions with an EPA situated in the TUI to be as effective as embedding the agent directly in the game environment. This inference is based on evidence showing reliable differences across conditions on the metrics of performance and self-reported mental demand and feedback usefulness items. This research provides source modality tradeoffs linked to tactics for relaying training relevant explicit information to a user based on real-time performance in a game
- …