Logistic Knowledge Tracing: A Constrained Framework for Learner Modeling

Adaptive learning technology solutions often use a learner model to trace learning and make pedagogical decisions. The present research introduces a formalized methodology for specifying learner models, Logistic Knowledge Tracing (LKT), that consolidates many extant learner modeling methods. The strength of LKT is the specification of a symbolic notation system for alternative logistic regression models that is powerful enough to specify many extant models in the literature and many new models. To demonstrate the generality of LKT, we fit 12 models, some variants of well-known models and some newly devised, to 6 learning technology datasets. The results indicated that no single learner model was best in all cases, further justifying a broad approach that considers multiple learner model features and the learning context. The models presented here avoid student-level fixed parameters to increase generalizability. We also introduce features to stand in for these intercepts. We argue that to be maximally applicable, a learner model needs to adapt to student differences, rather than needing to be pre-parameterized with the level of each student's ability

THE ROLE OF SIMULATION IN SUPPORTING LONGER-TERM LEARNING AND MENTORING WITH TECHNOLOGY

Mentoring is an important part of professional development and longer-term learning. The nature of longer-term mentoring contexts means that designing, developing, and testing adaptive learning sys-tems for use in this kind of context would be very costly as it would require substantial amounts of fi-nancial, human, and time resources. Simulation is a cheaper and quicker approach for evaluating the impact of various design and development decisions. Within the Artificial Intelligence in Education (AIED) research community, however, surprisingly little attention has been paid to how to design, de-velop, and use simulations in longer-term learning contexts. The central challenge is that adaptive learning system designers and educational practitioners have limited guidance on what steps to consider when designing simulations for supporting longer-term mentoring system design and development deci-sions. My research work takes as a starting point VanLehn et al.’s [1] introduction to applications of simulated students and Erickson et al.’s [2] suggested approach to creating simulated learning envi-ronments. My dissertation presents four research directions using a real-world longer-term mentoring context, a doctoral program, for illustrative purposes. The first direction outlines a framework for guid-ing system designers as to what factors to consider when building pedagogical simulations, fundamen-tally to answer the question: how can a system designer capture a representation of a target learning context in a pedagogical simulation model? To illustrate the feasibility of this framework, this disserta-tion describes how to build, the SimDoc model, a pedagogical model of a longer-term mentoring learn-ing environment – a doctoral program. The second direction builds on the first, and considers the issue of model fidelity, essentially to answer the question: how can a system designer determine a simulation model’s fidelity to the desired granularity level? This dissertation shows how data from a target learning environment, the research literature, and common sense are combined to achieve SimDoc’s medium fidelity model. The third research direction explores calibration and validation issues to answer the question: how many simulation runs does it take for a practitioner to have confidence in the simulation model’s output? This dissertation describes the steps taken to calibrate and validate the SimDoc model, so its output statistically matches data from the target doctoral program, the one at the university of Saskatchewan. The fourth direction is to demonstrate the applicability of the resulting pedagogical model. This dissertation presents two experiments using SimDoc to illustrate how to explore pedagogi-cal questions concerning personalization strategies and to determine the effectiveness of different men-toring strategies in a target learning context. Overall, this dissertation shows that simulation is an important tool in the AIED system design-ers’ toolkit as AIED moves towards designing, building, and evaluating AIED systems meant to support learners in longer-term learning and mentoring contexts. Simulation allows a system designer to exper-iment with various design and implementation decisions in a cost-effective and timely manner before committing to these decisions in the real world

How Simulation can Illuminate Pedagogical and System Design Issues in Dynamic Open Ended Learning Environments

A Dynamic Open-Ended Learning Environment (DOELE) is a collection of learners and learning objects (LOs) that could be constantly changing. In DOELEs, learners need the support of Advanced Learning Technology (ALT), but most ALT is not designed to run in such environments. An architecture for designing advanced learning technology that is compatible with DOELEs is the ecological approach (EA). This thesis looks at how to test and develop ALT based on the EA, and argues that this process would benefit from the use of simulation. The essential components of an EA-based simulation are: simulated learners, simulated LOs, and their simulated interactions. In this thesis the value of simulation is demonstrated with two experiments. The first experiment focuses on the pedagogical issue of peer impact, how learning is impacted by the performance of peers. By systematically varying the number and type of learners and LOs in a DOELE, the simulation uncovers behaviours that would otherwise go unseen. The second experiment shows how to validate and tune a new instructional planner built on the EA, the Collaborative Filtering based on Learning Sequences planner (CFLS). When the CFLS planner is configured appropriately, simulated learners achieve higher performance measurements that those learners using the baseline planners. Simulation results lead to predictions that ultimately need to be proven in the real world, but even without real world validation such predictions can be useful to researchers to inform the ALT system design process. This thesis work shows that it is not necessary to model all the details of the real world to come to a better understanding of a pedagogical issue such as peer impact. And, simulation allowed for the design of the first known instructional planner to be based on usage data, the CFLS planner. The use of simulation for the design of EA-based systems opens new possibilities for instructional planning without knowledge engineering. Such systems can find niche learning paths that may have never been thought of by a human designer. By exploring pedagogical and ALT system design issues for DOELEs, this thesis shows that simulation is a valuable addition to the toolkit for ALT researchers

A Closer Look into Recent Video-based Learning Research: A Comprehensive Review of Video Characteristics, Tools, Technologies, and Learning Effectiveness

People increasingly use videos on the Web as a source for learning. To support this way of learning, researchers and developers are continuously developing tools, proposing guidelines, analyzing data, and conducting experiments. However, it is still not clear what characteristics a video should have to be an effective learning medium. In this paper, we present a comprehensive review of 257 articles on video-based learning for the period from 2016 to 2021. One of the aims of the review is to identify the video characteristics that have been explored by previous work. Based on our analysis, we suggest a taxonomy which organizes the video characteristics and contextual aspects into eight categories: (1) audio features, (2) visual features, (3) textual features, (4) instructor behavior, (5) learners activities, (6) interactive features (quizzes, etc.), (7) production style, and (8) instructional design. Also, we identify four representative research directions: (1) proposals of tools to support video-based learning, (2) studies with controlled experiments, (3) data analysis studies, and (4) proposals of design guidelines for learning videos. We find that the most explored characteristics are textual features followed by visual features, learner activities, and interactive features. Text of transcripts, video frames, and images (figures and illustrations) are most frequently used by tools that support learning through videos. The learner activity is heavily explored through log files in data analysis studies, and interactive features have been frequently scrutinized in controlled experiments. We complement our review by contrasting research findings that investigate the impact of video characteristics on the learning effectiveness, report on tasks and technologies used to develop tools that support learning, and summarize trends of design guidelines to produce learning video

Electroencephalogram Signals for Detecting Confused Students in Online Education Platforms with Probability-Based Features

Article discusses how despite the advantages of online education, it lacks face-to-face settings, which makes it very difficult to analyze the students’ level of interaction, understanding, and confusion. This study proposes a novel engineering approach that uses probability-based features (PBF) for increasing the efficacy of machine learning models

Sequenciamento adaptativo de exercícios baseado na correspondência entre a dificuldade da solução e o desempenho dinâmico do aprendiz

Orientador : Prof. Dr. Alexandre I. DireneTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa: Curitiba, 21/10/2015Inclui referências : f. 67-74Resumo: A perícia do aprendiz geralmente e desenvolvida através da resolução de exercícios que requerem um conjunto de habilidades avaliadas, tanto no sistema educacional de sala de aula convencional quanto em sistemas de apredizagem baseados em computador tais como Sistemas Tutores Inteligentes. Esta pesquisa propôs uma formula de rating para avaliação automática do desempenho do aluno, partindo-se do princípio de que o grau de dificuldade das questões pode ser medido pela taxa de alunos que as acertam/erram, sendo essa informação usada no cálculo de sua nota. Neste trabalho, os aspectos motivacionais em aprendizagem são considerados relevantes, sendo importante propor atividades adequadas ao nível da expertise do estudante, pois a apresentação de exercícios com grau de dificuldade muito abaixo (ou acima) do nível cognitivo do aprendiz pode causar entediamento (ou frustração), ocasionando o abandono da atividade proposta. Nesse sentido, este estudo também desenvolveu um algoritmo de sequenciamento adaptivo de exercícios que se baseia nos graus de dificuldade das questões, em que o sequenciamento e guiado pela performance dinâmica do aprendiz. Foi realizado um estudo empírico a partir de dados coletados de alunos reais que demonstrou a validade da formula de rating. Os algoritmos para calculo do grau de dificuldades das questões e dos ratings dos alunos, bem como o algoritmo de sequenciamento adaptivo foram implementados efetuando-se alterações na ferramenta web de autoria de objetos de aprendizagem FARMA, gerando assim o ambiente ADAPTFARMA. Também foi realizada uma avaliação experimental da aprendizagem através de experimento estatístico comparativo entre diferentes modalidades de sequenciamento de exercícios usando como base um objeto de aprendizagem construído em ADAPTFARMA. Palavras-chave: calibragem de exercícios, rating, Sistemas Tutores Inteligentes.Abstract: The expertise of learners is usually developed by solving problems that require a set of assessed skills. This is done in both conventional education schools and by applying advanced learning technologies, such as Intelligent Tutoring Systems. This research proposed a formula for automatic assessment of students, assuming that the degree of difficulty of the questions can be measured by counting the students that are successful and those who failed. This information is used to calculate their grade as a particular rating scale. Besides, the motivational aspects of learning are considered in depth. In this sense, it is important to propose activities according to the student's level of expertise, which is achieved through presenting students with exercises that are compatible with the difficulty degree of their cognitive skills. In doing so, both boredom and frustation can be avoided, as much as can be the withdrawal of the proposed activities on the part of students. An empirical study based on existing students data partially influenced the development of the first version of an adaptive algorithm for exercises sequencing, based on the difficulty degree of the questions. The sequencing process is guided by the learner's dynamic performance. An experiment has also been carried out with four maths classes of a local public school. Data collected from students' performance gains demonstrated the suitability of the rating formula. The algorithms for calculating and matching the difficulty degree of the questions and the students' rating were implemented by extending the web authoring tool of learning objects named FARMA, thus generating the ADAPTFARMA environment. Finally, conclusions and future research directions are described. Keywords: exercises calibration, rating, Intelligent Tutoring Systems

Integrating knowledge tracing and item response theory: A tale of two frameworks

Traditionally, the assessment and learning science commu-nities rely on different paradigms to model student performance. The assessment community uses Item Response Theory which allows modeling different student abilities and problem difficulties, while the learning science community uses Knowledge Tracing, which captures skill acquisition. These two paradigms are complementary - IRT cannot be used to model student learning, while Knowledge Tracing assumes all students and problems are the same. Recently, two highly related models based on a principled synthesis of IRT and Knowledge Tracing were introduced. However, these two models were evaluated on different data sets, using different evaluation metrics and with different ways of splitting the data into training and testing sets. In this paper we reconcile the models' results by presenting a unified view of the two models, and by evaluating the models under a common evaluation metric. We find that both models are equivalent and only differ in their training procedure. Our results show that the combined IRT and Knowledge Tracing models offer the best of assessment and learning sciences - high prediction accuracy like the IRT model, and the ability to model student learning like Knowledge Tracing

Developing Computational Thinking Teaching Strategies to Model Pandemics and Containment Measures.

COVID-19 has been extremely difficult to control. The lack of understanding of key aspects of pandemics has affected virus transmission. On the other hand, there is a demand to incorporate computational thinking (CT) in the curricula with applications in STEM. However, there are still no exemplars in the curriculum that apply CT to real-world problems such as controlling a pandemic or other similar global crises. In this paper, we fill this gap by proposing exemplars of CT for modeling the pandemic. We designed exemplars following the three pillars of the framework for CT from the Inclusive Mathematics for Sustainability in a Digital Economy (InMside) project by Asia-Pacific Economic Cooperation (APEC): algorithmic thinking, computational modeling, and machine learning. For each pillar, we designed a progressive sequence of activities that covers from elementary to high school. In an experimental study with elementary and middle school students from 2 schools of high vulnerability, we found that the computational modeling exemplar can be implemented by teachers and correctly understood by students. We conclude that it is feasible to introduce the exemplars at all grade levels and that this is a powerful example of Science Technology, Engineering, and Mathematics (STEM) integration that helps reflect and tackle real-world and challenging public health problems of great impact for students and their families

Recognising Complex Mental States from Naturalistic Human-Computer Interactions

New advances in computer vision techniques will revolutionize the way we interact with computers, as they, together with other improvements, will help us build machines that understand us better. The face is the main non-verbal channel for human-human communication and contains valuable information about emotion, mood, and mental state. Affective computing researchers have investigated widely how facial expressions can be used for automatically recognizing affect and mental states. Nowadays, physiological signals can be measured by video-based techniques, which can also be utilised for emotion detection. Physiological signals, are an important indicator of internal feelings, and are more robust against social masking. This thesis focuses on computer vision techniques to detect facial expression and physiological changes for recognizing non-basic and natural emotions during human-computer interaction. It covers all stages of the research process from data acquisition, integration and application. Most previous studies focused on acquiring data from prototypic basic emotions acted out under laboratory conditions. To evaluate the proposed method under more practical conditions, two different scenarios were used for data collection. In the first scenario, a set of controlled stimulus was used to trigger the user’s emotion. The second scenario aimed at capturing more naturalistic emotions that might occur during a writing activity. In the second scenario, the engagement level of the participants with other affective states was the target of the system. For the first time this thesis explores how video-based physiological measures can be used in affect detection. Video-based measuring of physiological signals is a new technique that needs more improvement to be used in practical applications. A machine learning approach is proposed and evaluated to improve the accuracy of heart rate (HR) measurement using an ordinary camera during a naturalistic interaction with computer

Recognising Complex Mental States from Naturalistic Human-Computer Interactions

New advances in computer vision techniques will revolutionize the way we interact with computers, as they, together with other improvements, will help us build machines that understand us better. The face is the main non-verbal channel for human-human communication and contains valuable information about emotion, mood, and mental state. Affective computing researchers have investigated widely how facial expressions can be used for automatically recognizing affect and mental states. Nowadays, physiological signals can be measured by video-based techniques, which can also be utilised for emotion detection. Physiological signals, are an important indicator of internal feelings, and are more robust against social masking. This thesis focuses on computer vision techniques to detect facial expression and physiological changes for recognizing non-basic and natural emotions during human-computer interaction. It covers all stages of the research process from data acquisition, integration and application. Most previous studies focused on acquiring data from prototypic basic emotions acted out under laboratory conditions. To evaluate the proposed method under more practical conditions, two different scenarios were used for data collection. In the first scenario, a set of controlled stimulus was used to trigger the user’s emotion. The second scenario aimed at capturing more naturalistic emotions that might occur during a writing activity. In the second scenario, the engagement level of the participants with other affective states was the target of the system. For the first time this thesis explores how video-based physiological measures can be used in affect detection. Video-based measuring of physiological signals is a new technique that needs more improvement to be used in practical applications. A machine learning approach is proposed and evaluated to improve the accuracy of heart rate (HR) measurement using an ordinary camera during a naturalistic interaction with computer