When Easy Becomes Boring and Difficult Becomes Frustrating: Disentangling the Effects of Item Difficulty Level and Person Proficiency on Learning and Motivation.

The research on electronic learning environments has evolved towards creating adaptive learning environments. In this study, the focus is on adaptive curriculum sequencing, in particular, the efficacy of an adaptive curriculum sequencing algorithm based on matching the item difficulty level to the learner’s proficiency level. We therefore explored the effect of the relative difficulty level on learning outcome and motivation. Results indicate that, for learning environments consisting of questions focusing on just one dimension and with knowledge of correct response, it does not matter whether we present easy, moderate or difficult items or whether we present the items with a random mix of difficulty levels, regarding both learning and motivation

I’ve (Urn)ed This: An Application and Criterion-based Evaluation of the Urnings Algorithm

There is increased interest in personalized learning and making e-learning environments more adaptable. Some e-learning systems may use an Item Response Theory (IRT)-based assessment system. An important distinction between assessment and learning contexts is that learner proficiency is expected to remain constant across an assessment, while it is expected to change over time in a learning context. Constant learner proficiency during an assessment enables conventional approaches to estimating person and item parameters using IRT. These IRT-based systems could be abandoned for alternative approaches to modeling learners and system learning content, but assessments may provide more functions than adapting learning material to students. Thus, there is the question, how can e-learning systems with IRT-based assessment components more dynamically adapt their learning content? Is there a solution that leverages IRT for adapting the learning content of the system? A promising solution is the Urnings algorithm. Like other candidate algorithms, it is computationally light, but this algorithm has mechanisms for preventing variance inflation and is suitable for e-learning contexts. It also provides a measure of uncertainty around estimates. It has been studied both through simulations and applications to e-learning systems. Results are promising; however, there has not been an application of the Urnings algorithm to an e-learning context where there are conventionally estimated person parameters to compare the algorithm estimates to. This study addresses this gap by applying the Urnings algorithm to a K–8 reading and mathematics learning platform. In data from this platform, we have person parameter estimates across academic years from an in-system diagnostic assessment. Results from this study will help industry researchers understand the feasibility of the Urnings algorithm for large e-learning systems with IRT-based assessment components

Towards Designing AI-Enabled Adaptive Learning Systems

Paper I, III, IV and V are not available as a part of the dissertation due to the copyright.Among the many innovations driven by artificial intelligence (AI) are more advanced learning systems known as AI-enabled adaptive learning systems (AI-ALS). AI-ALS are platforms that adapt to the learning strategies of students by modifying the order and difficulty level of learning tasks based on the abilities of students. These systems support adaptive learning, which is the personalization of learning for students in a learning system, such that the system can deal with individual differences in aptitude. AI-ALS are gaining traction due to their ability to deliver learning content and adapt to individual student needs. While the potential and importance of such systems have been well documented, the actual implementation of AI-ALS and other AI-based learning systems in real-world teaching and learning settings has not reached the effectiveness envisaged on the level of theory. Moreover, AI-ALS lack transferable insights and codification of knowledge on their design and development. The reason for this is that many previous studies were experimental. Thus, this dissertation aims to narrow the gap between experimental research and field practice by providing practical design statements that can be implemented in effective AI-ALSs.publishedVersio

Adaptive and Re-adaptive Pedagogies in Higher Education: A Comparative, Longitudinal Study of Their Impact on Professional Competence Development across Diverse Curricula

This study addresses concerns that traditional, lecture-based teaching methods may not sufficiently develop the integrated competencies demanded by modern professional practice. A disconnect exists between conventional pedagogy and desired learning outcomes, prompting increased interest in innovative, student-centered instructional models tailored to competence growth. Despite this, nuanced differences in competence development across diverse university curricula remain underexplored, with research predominantly relying on students’ self-assessments. To address these gaps, this study employs longitudinal mixed-methods approaches with regard to theory triangulation and investigator triangulation to better understand how professional knowledge, skills, and dispositions evolve across varied curricula and contexts. This research emphasizes adaptive and re-adaptive teaching approaches incorporating technology, individualization, and experiential learning, which may uniquely integrate skill development with contextual conceptual learning. Specific attention is paid to professional education paths like design, media, and communications degrees, where contemporary competence models stress capabilities beyond core conceptual knowledge. Results from this study aim to guide reform efforts to optimize professional competence development across diverse academic areas

Computational Psychometrics for Item-based Computerized Adaptive Learning

With advances in computer technology and expanded access to educational data, psychometrics faces new opportunities and challenges for enhancing pattern discovery and decision-making in testing and learning. In this dissertation, I introduced three computational psychometrics studies for solving the technical problems in item-based computerized adaptive learning (CAL) systems related to dynamic measurement, diagnosis, and recommendation based on Bayesian item response theory (IRT). For the first study, I introduced a new knowledge tracing (KT) model, dynamic IRT (DIRT), which can iteratively update the posterior distribution of latent ability based on moment match approximation and capture the uncertainty of ability change during the learning process. For dynamic measurement, DIRT has advantages in interpretation, flexibility, computation cost, and implementability. For the second study, A new measurement model, named multilevel and multidimensional item response theory with Q matrix (MMIRT-Q), was proposed to provide fine-grained diagnostic feedback. I introduced sequential Monte Carlo (SMC) for online estimation of latent abilities. For the third study, I proposed the maximum expected ratio of posterior variance reduction criterion (MERPV) for testing purposes and the maximum expected improvement in posterior mean (MEIPM) criterion for learning purposes under the unified framework of IRT. With these computational psychometrics solutions, we can improve the students’ learning and testing experience with accurate psychometrics measurement, timely diagnosis feedback, and efficient item selection

Augmented Conversation and Cognitive Apprenticeship Metamodel Based Intelligent Learning Activity Builder System

This research focused on a formal (theory based) approach to designing Intelligent Tutoring System (ITS) authoring tool involving two specific conventional pedagogical theories—Conversation Theory (CT) and Cognitive Apprenticeship (CA). The research conceptualised an Augmented Conversation and Cognitive Apprenticeship Metamodel (ACCAM) based on apriori theoretical knowledge and assumptions of its underlying theories. ACCAM was implemented in an Intelligent Learning Activity Builder System (ILABS)—an ITS authoring tool. ACCAM’s implementation aims to facilitate formally designed tutoring systems, hence, ILABS―the practical implementation of ACCAM― constructs metamodels for Intelligent Learning Activity Tools (ILATs) in a numerical problem-solving context (focusing on the construction of procedural knowledge in applied numerical disciplines). Also, an Intelligent Learning Activity Management System (ILAMS), although not the focus of this research, was developed as a launchpad for ILATs constructed and to administer learning activities. Hence, ACCAM and ILABS constitute the conceptual and practical contributions that respectively flow from this research. ACCAM’s implementation was tested through the evaluation of ILABS and ILATs within an applied numerical domain―the accounting domain. The evaluation focused on the key constructs of ACCAM―cognitive visibility and conversation, implemented through a tutoring strategy employing Process Monitoring (PM). PM augments conversation within a cognitive apprenticeship framework; it aims to improve the visibility of the cognitive process of a learner and infers intelligence in tutoring systems. PM was implemented via an interface that attempts to bring learner’s thought process to the surface. This approach contrasted with previous studies that adopted standard Artificial Intelligence (AI) based inference techniques. The interface-based PM extends the existing CT and CA work. The strategy (i.e. interface-based PM) makes available a new tutoring approach that aimed fine-grain (or step-wise) feedbacks, unlike the goal-oriented feedbacks of model-tracing. The impact of PM—as a preventive strategy (or intervention) and to aid diagnosis of learners’ cognitive process—was investigated in relation to other constructs from the literature (such as detection of misconception, feedback generation and perceived learning effectiveness). Thus, the conceptualisation and implementation of PM via an interface also contributes to knowledge and practice. The evaluation of the ACCAM-based design approach and investigation of the above mentioned constructs were undertaken through users’ reaction/perception to ILABS and ILAT. This involved, principally, quantitative approach. However, a qualitative approach was also utilised to gain deeper insight. Findings from the evaluation supports the formal (theory based) design approach—the design of ILABS through interaction with ACCAM. Empirical data revealed the presence of conversation and cognitive visibility constructs in ILATs, which were determined through its behaviour during the learning process. This research identified some other theoretical elements (e.g. motivation, reflection, remediation, evaluation, etc.) that possibly play out in a learning process. This clarifies key conceptual variables that should be considered when constructing tutoring systems for applied numerical disciplines (e.g. accounting, engineering). Also, the research revealed that PM enhances the detection of a learner’s misconception and feedback generation. Nevertheless, qualitative data revealed that frequent feedbacks due to the implementation of PM could be obstructive to thought process at advance stage of learning. Thus, PM implementations should also include delayed diagnosis, especially for advance learners who prefer to have it on request. Despite that, current implementation allows users to turn PM off, thereby using alternative learning route. Overall, the research revealed that the implementation of interface-based PM (i.e. conversation and cognitive visibility) improved the visibility of learner’s cognitive process, and this in turn enhanced learning—as perceived

Adaptive intelligent tutoring for teaching modern standard Arabic

A thesis submitted to the University of Bedfordshire, in partial fulfilment of the requirements for the degree of Doctor of PhilosophyThe aim of this PhD thesis is to develop a framework for adaptive intelligent tutoring systems (ITS) in the domain of Modern Standard Arabic language. This framework will comprise of a new approach to using a fuzzy inference mechanism and generic rules in guiding the learning process. In addition, the framework will demonstrate another contribution in which the system can be adapted to be used in the teaching of different languages. A prototype system will be developed to demonstrate these features. This system is targeted at adult English-speaking casual learners with no pre-knowledge of the Arabic language. It will consist of two parts: an ITS for learners to use and a teachers‘ tool for configuring and customising the teaching rules and artificial intelligence components among other configuration operations. The system also provides a diverse teaching-strategies‘ environment based on multiple instructional strategies. This approach is based on general rules that provide means to a reconfigurable prediction. The ITS determines the learner‘s learning characteristics using multiple fuzzy inferences. It has a reconfigurable design that can be altered by the teacher at runtime via a teacher-interface. A framework for an independent domain (i.e. pluggable-domain) for foreign language tutoring systems is introduced in this research. This approach allows the system to adapt to the teaching of a different language with little changes required. Such a feature has the advantages of reducing the time and cost required for building intelligent language tutoring systems. To evaluate the proposed system, two experiments are conducted with two versions of the software: the ITS and a cut down version with no artificial intelligence components. The learners used the ITS had shown an increase in scores between the post-test and the pre-test with learning gain of 35% compared to 25% of the learners from the cut down version

A Guided Chatbot Learning Experience in the Science Classroom

This dissertation describes a practitioner’s design-based development of a prototype chatbot to guide students in learning biological concepts of genetic mutations and protein synthesis. This chatbot’s architecture provides learning activities, feedback, and support throughout a series of short, connected lessons. The chatbot is designed to scaffold learners through a predict, observe, explain model of inquiry learning. It utilizes real-world phenomena to lead students through biology core ideas, science and engineering practices, and crosscutting concepts. Results of prototype testing include survey results in support of the proof of concept among both students and teachers, as well as accuracy measurements of chatbot intents. Descriptive statistics and suggestions were collected from both groups to evaluate the relevancy, consistency, practicality, and effectiveness of the project as well as speak to improvements for future projects. The designer finds that the construction of chatbots as guided learning experiences holds untapped potential in science educational technology. Advisor: Guy Traini