Empowering Qualitative Research Methods in Education with Artificial Intelligence

Artificial Intelligence is one of the fastest growing disciplines, disrupting many sectors. Originally mainly for computer scientists and engineers, it has been expanding its horizons and empowering many other disciplines contributing to the development of many novel applications in many sectors. These include medicine and health care, business and finance, psychology and neuroscience, physics and biology to mention a few. However, one of the disciplines in which artificial intelligence has not been fully explored and exploited yet is education. In this discipline, many research methods are employed by scholars, lecturers and practitioners to investigate the impact of different instructional approaches on learning and to understand the ways skills and knowledge are acquired by learners. One of these is qualitative research, a scientific method grounded in observations that manipulates and analyses non-numerical data. It focuses on seeking answers to why and how a particular observed phenomenon occurs rather than on its occurrences. This study aims to explore and discuss the impact of artificial intelligence on qualitative research methods. In particular, it focuses on how artificial intelligence have empowered qualitative research methods so far, and how it can be used in education for enhancing teaching and learning

An Evaluation of the Reliability, Validity and Sensitivity of Three Human Mental Workload Measures Under Different Instructional Conditions in Third-Level Education

Although Cognitive Load Theory (CLT) has been researched for many years, it has been criticised for its theoretical clarity and its methodological approach. A crucial issue is the measurement of three types of cognitive load conceived in the theory, and the assessment of overall human cognitive load during learning tasks. This research study is motivated by these issues and it aims to investigate the reliability, validity and sensitivity of three existing self-reporting mental workload instruments, mainly used in Ergonomics, when applied to Education and in particular to the field of Teaching and Learning. A primary research study has been designed and performed in a typical third-level classroom in Computer Science, and the self-reporting mental workload instruments employed are the NASA Task Load Index, the Workload Profile and the Rating Scale Mental Effort. Three instructional design conditions have been designed and employed for the above purposes. The first design condition followed the traditional explicit instruction paradigm whereby a lecturer delivers instructional material mainly using a one-way approach with almost no interactions with students. The second design condition was inspired by the Cognitive Theory of Multimedia Learning whereby the same content, delivered under the first condition, was converted in a multimedia video by following a set of its design principles. The third design condition was an extension of the second condition whereby an inquiry activity was executed after the delivery of the second condition. The empirical evidence gathered in this study suggests that the three selected mental workload measures are highly reliable. Their moderate face validity is in line with the results obtained so far within Ergonomics emphasising and confirming the difficulty in creating optimally valid measures of mental workload. However, the sensitivity of these measures, as achieved in this study, is low, indicating how the three instructional design conditions, as conceived and implemented, do not impose significantly different mental workload levels on learners

A Novel Parabolic Model of Instructional Efficiency Grounded on Ideal Mental Workload and Performance

Instructional efficiency within education is a measurable concept and models have been proposed to assess it. The main assumption behind these models is that efficiency is the capacity to achieve established goals at the minimal expense of resources. This article challenges this assumption by contributing to the body of Knowledge with a novel model that is grounded on ideal mental workload and performance, namely the parabolic model of instructional efficiency. A comparative empirical investigation has been constructed to demonstrate the potential of this model for instructional design evaluation. Evidence demonstrated that this model achieved a good concurrent validity with the well-known likelihood model of instructional efficiency, treated as baseline, but a better discriminant validity for the evaluation of the training and learning phases. Additionally, the inferences produced by this novel model have led to a superior information gain when compared to the baseline

Analysing the Impact of Machine Learning to Model Subjective Mental Workload: A Case Study in Third-Level Education

Mental workload measurement is a complex multidisciplinary research area that includes both the theoretical and practical development of models. These models are aimed at aggregating those factors, believed to shape mental workload, and their interaction, for the purpose of human performance prediction. In the literature, models are mainly theory-driven: their distinct development has been influenced by the beliefs and intuitions of individual scholars in the disciplines of Psychology and Human Factors. This work presents a novel research that aims at reversing this tendency. Specifically, it employs a selection of learning techniques, borrowed from machine learning, to induce models of mental workload from data, with no theoretical assumption or hypothesis. These models are subsequently compared against two well-known subjective measures of mental workload, namely the NASA Task Load Index and the Workload Profile. Findings show how these data-driven models are convergently valid and can explain overall perception of mental workload with a lower error

Evaluating instructional designs with mental workload assessments in university classrooms

Cognitive cognitive load theory (CLT) has been conceived for improving instructional design practices. Although researched for many years, one open problem is a clear definition of its cognitive load types and their aggregation towards an index of overall cognitive load. In Ergonomics, the situation is different with plenty of research devoted to the development of robust constructs of mental workload (MWL). By drawing a parallel between CLT and MWL, as well as by integrating relevant theories and measurement techniques from these two fields, this paper is aimed at investigating the reliability, validity and sensitivity of three existing self-reporting mental workload measures when applied to long learning sessions, namely, the NASA Task Load index, the Workload Profile and the Rating Scale Mental Effort, in a typical university classroom. These measures were aimed at serving for the evaluation of two instructional conditions. Evidence suggests these selected measures are reliable and their moderate validity is in line with results obtained within Ergonomics. Additionally, an analysis of their sensitivity by employing the descriptive Harrell-Davis estimator suggests that the Workload Profile is more sensitive than the Nasa Task Load Index and the Rating Scale Mental Effort for long learning sessions

Adaptive and Re-adaptive Pedagogies in Higher Education: A Comparative, Longitudinal Study of Their Impact on Professional Competence Development across Diverse Curricula

This study addresses concerns that traditional, lecture-based teaching methods may not sufficiently develop the integrated competencies demanded by modern professional practice. A disconnect exists between conventional pedagogy and desired learning outcomes, prompting increased interest in innovative, student-centered instructional models tailored to competence growth. Despite this, nuanced differences in competence development across diverse university curricula remain underexplored, with research predominantly relying on students’ self-assessments. To address these gaps, this study employs longitudinal mixed-methods approaches with regard to theory triangulation and investigator triangulation to better understand how professional knowledge, skills, and dispositions evolve across varied curricula and contexts. This research emphasizes adaptive and re-adaptive teaching approaches incorporating technology, individualization, and experiential learning, which may uniquely integrate skill development with contextual conceptual learning. Specific attention is paid to professional education paths like design, media, and communications degrees, where contemporary competence models stress capabilities beyond core conceptual knowledge. Results from this study aim to guide reform efforts to optimize professional competence development across diverse academic areas

A Comparison of Instructional Efficiency Models in Third Level Education

This study investigates the validity and sensitivity of a novel model of instructional efficiency: the parabolic model. The novel model is compared against state-of-the-art models present in instructional design today; Likelihood model, Deviational model and Multidimensional model. This models is based on the assumption that optimal mental workload and high performance leads to high efficiency, while other models assume that low mental workload and high performance leads to high efficiency. The investigation makes use of two instructional design conditions: a direct instructions approach to learning and its extension with a collaborative activity. A control group received the former instructional design while an experimental group received the latter design. A performance score was extracted for evaluation. The models of efficiency compared were based upon both a unidimensional and a multidimensional measure of mental workload, which were acquired through self-reporting from the participants. These mental load measures in conjunction with the performance score contribute to the calculation of efficiency scores for each model. The aim of this study is to determine whether the novel model is able to better differentiate between the control and experimental groups based on the resulting efficiency when compared to the other models. The models were analysed and compared using various statistical tests and techniques. Empirical evidence partially supports the proposed hypothesis that parabolic model demonstrates validity, however lacks sufficient statistical evidence to suggest that the model has better sensitivity and its capacity to differentiate between the two groups

An Empirical Evaluation of the Inferential Capacity of Defeasible Argumentation, Non-monotonic Fuzzy Reasoning and Expert Systems

Several non-monotonic formalisms exist in the field of Artificial Intelligence for reasoning under uncertainty. Many of these are deductive and knowledge-driven, and also employ procedural and semi-declarative techniques for inferential purposes. Nonetheless, limited work exist for the comparison across distinct techniques and in particular the examination of their inferential capacity. Thus, this paper focuses on a comparison of three knowledge-driven approaches employed for non-monotonic reasoning, namely expert systems, fuzzy reasoning and defeasible argumentation. A knowledge-representation and reasoning problem has been selected: modelling and assessing mental workload. This is an ill-defined construct, and its formalisation can be seen as a reasoning activity under uncertainty. An experimental work was performed by exploiting three deductive knowledge bases produced with the aid of experts in the field. These were coded into models by employing the selected techniques and were subsequently elicited with data gathered from humans. The inferences produced by these models were in turn analysed according to common metrics of evaluation in the field of mental workload, in specific validity and sensitivity. Findings suggest that the variance of the inferences of expert systems and fuzzy reasoning models was higher, highlighting poor stability. Contrarily, that of argument-based models was lower, showing a superior stability of its inferences across knowledge bases and under different system configurations. The originality of this research lies in the quantification of the impact of defeasible argumentation. It contributes to the field of logic and non-monotonic reasoning by situating defeasible argumentation among similar approaches of non-monotonic reasoning under uncertainty through a novel empirical comparison

Human Mental Workload: A Survey and a Novel Inclusive Definition

Human mental workload is arguably the most invoked multidimensional construct in Human Factors and Ergonomics, getting momentum also in Neuroscience and Neuroergonomics. Uncertainties exist in its characterization, motivating the design and development of computational models, thus recently and actively receiving support from the discipline of Computer Science. However, its role in human performance prediction is assured. This work is aimed at providing a synthesis of the current state of the art in human mental workload assessment through considerations, definitions, measurement techniques as well as applications, Findings suggest that, despite an increasing number of associated research works, a single, reliable and generally applicable framework for mental workload research does not yet appear fully established. One reason for this gap is the existence of a wide swath of operational definitions, built upon different theoretical assumptions which are rarely examined collectively. A second reason is that the three main classes of measures, which are self-report, task performance, and physiological indices, have been used in isolation or in pairs, but more rarely in conjunction all together. Multiple definitions complement each another and we propose a novel inclusive definition of mental workload to support the next generation of empirical-based research. Similarly, by comprehensively employing physiological, task-performance, and self-report measures, more robust assessments of mental workload can be achieved

Discover Influential Mental Workload Attributes Impacting Learners Performance in Third-Level Education

Human Mental Workload is an intervening variable and a fundamental concept in the discipline of Ergonomics. It is deduced from variations in performance. High or low mental workload leads to hampering of performance. Mental workload in an educational setting has been extensively researched. It is applied in instructional design but it is obscure as to which factors are majorly driving mental workload in learners. This dissertation investigates the importance of the features used in the the NASA-Task Load Index mental workload assessment instrument and their impact on the performance of learners as assessed by multiple-choice tests conducted in classrooms of an MSc programme in a university. Model training is performed on these attributes using machine learning approaches including decision tree regression and linear regression. Montecarlo sampling was used in the training phase to ensure model stability. The identification of the importance of selected features is carried on using the permutation feature technique since it is adaptable and applicable across a variety of supervised learning methods. Empirical evidence emphasises the absence of more important features over the others tentatively suggesting their applicability in a multi-dimensional model