Multimodal Text-Powered Interactive E-Module for Enhancing English Structure Learning

This study aimed to develop an interactive e-module based on multimodal text for the Intermediate English Structure course in supporting digital learning at Universitas Riau, Indonesia. The research followed a systematic Research and Development (R&D) approach and applied the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) model, which integrated the initial research techniques with final evaluation methods to ensure the effectiveness of the e-module. This study utilized purposive sampling techniques which involved 42 students and experts in media and materials. The development started with a thorough analysis of students’ needs, obtaining a score of 3.05 which indicated high necessity. The subsequent step involved designing the desired product, which is an interactive e-module containing multimodal texts, that is, various types of texts which include written texts, images, animated videos and interactive quizzes. The e-module consists of six units; each containing sections such as video-based explanation, text-based descriptions, illustrative examples, interactive quizzes, and chapter summaries. Validation of the e-module was carried out by material and media experts, resulting in scores of 3.73 (indicating very valid) and 3.77 (indicating very valid) respectively. Then, when tested to students, the results revealed a 'clarity of language use in the learning material' indicator is 3.57 (indicating very good), while the 'ease of use' indicator stands at 3.60 (indicating very good). Thus, these results suggest that the developed e-module exhibits excellent quality and is well-suited to serve as a digital learning resource for the Intermediate Structure course in the English department at Universitas Riau, Indonesia. Furthermore, this e-module also has the potential to transform how students learn. With combination of different types of content, the researchers have created a tool that can set a new standard for digital learning at Universitas Riau and beyond

Temporal pathways to learning: how learning emerges in an open-ended collaborative activity

The learning process depends on the nature of the learning environment, particularly in the case of open-ended learning environments, where the learning process is considered to be non-linear. In this paper, we report on the findings of employing a multimodal Hidden Markov Model (HMM) based methodology to investigate the temporal learning processes of two types of learners that have learning gains and a type that does not have learning gains in an open-ended collaborative learning activity. Considering log data, speech behavior, affective states and gaze patterns, we find that all learners start from a similar state of non-productivity, but once out of it they are unlikely to fall back into that state, especially in the case of the learners that have learning gains. Those who have learning gains shift between two problem solving strategies, each characterized by both exploratory and reflective actions, as well as demonstrate speech and gaze patterns associated with these strategies, that differ from those who don't have learning gains. Further, the teams that have learning gains also differ between themselves in the manner in which they employ the problem solving strategies over the interaction, as well as in the manner they express negative emotions while exhibiting a particular strategy. These outcomes contribute to understanding the multiple pathways of learning in an open-ended collaborative learning environment, and provide actionable insights for designing effective interventions

Experimental Studies in Learning Technology and Child–Computer Interaction

This book is about the ways in which experiments can be employed in the context of research on learning technologies and child–computer interaction (CCI). It is directed at researchers, supporting them to employ experimental studies while increasing their quality and rigor. The book provides a complete and comprehensive description on how to design, implement, and report experiments, with a focus on and examples from CCI and learning technology research. The topics covered include an introduction to CCI and learning technologies as interdisciplinary fields of research, how to design educational interfaces and visualizations that support experimental studies, the advantages and disadvantages of a variety of experiments, methodological decisions in designing and conducting experiments (e.g. devising hypotheses and selecting measures), and the reporting of results. As well, a brief introduction on how contemporary advances in data science, artificial intelligence, and sensor data have impacted learning technology and CCI research is presented. The book details three important issues that a learning technology and CCI researcher needs to be aware of: the importance of the context, ethical considerations, and working with children. The motivation behind and emphasis of this book is helping prospective CCI and learning technology researchers (a) to evaluate the circumstances that favor (or do not favor) the use of experiments, (b) to make the necessary methodological decisions about the type and features of the experiment, (c) to design the necessary “artifacts” (e.g., prototype systems, interfaces, materials, and procedures), (d) to operationalize and conduct experimental procedures to minimize potential bias, and (e) to report the results of their studies for successful dissemination in top-tier venues (such as journals and conferences). This book is an open access publication

Experimental Studies in Learning Technology and Child–Computer Interaction

This book is about the ways in which experiments can be employed in the context of research on learning technologies and child–computer interaction (CCI). It is directed at researchers, supporting them to employ experimental studies while increasing their quality and rigor. The book provides a complete and comprehensive description on how to design, implement, and report experiments, with a focus on and examples from CCI and learning technology research. The topics covered include an introduction to CCI and learning technologies as interdisciplinary fields of research, how to design educational interfaces and visualizations that support experimental studies, the advantages and disadvantages of a variety of experiments, methodological decisions in designing and conducting experiments (e.g. devising hypotheses and selecting measures), and the reporting of results. As well, a brief introduction on how contemporary advances in data science, artificial intelligence, and sensor data have impacted learning technology and CCI research is presented. The book details three important issues that a learning technology and CCI researcher needs to be aware of: the importance of the context, ethical considerations, and working with children. The motivation behind and emphasis of this book is helping prospective CCI and learning technology researchers (a) to evaluate the circumstances that favor (or do not favor) the use of experiments, (b) to make the necessary methodological decisions about the type and features of the experiment, (c) to design the necessary “artifacts” (e.g., prototype systems, interfaces, materials, and procedures), (d) to operationalize and conduct experimental procedures to minimize potential bias, and (e) to report the results of their studies for successful dissemination in top-tier venues (such as journals and conferences). This book is an open access publication

Predicting learners' effortful behaviour in adaptive assessment using multimodal data

Many factors influence learners' performance on an activity beyond the knowledge required. Learners' on-task effort has been acknowledged for strongly relating to their educational outcomes, reflecting how actively they are engaged in that activity. However, effort is not directly observable. Multimodal data can provide additional insights into the learning processes and may allow for effort estimation. This paper presents an approach for the classification of effort in an adaptive assessment context. Specifically, the behaviour of 32 students was captured during an adaptive self-assessment activity, using logs and physiological data (i.e., eye-tracking, EEG, wristband and facial expressions). We applied k-means to the multimodal data to cluster students' behavioural patterns. Next, we predicted students' effort to complete the upcoming task, based on the discovered behavioural patterns using a combination of Hidden Markov Models (HMMs) and the Viterbi algorithm. We also compared the results with other state-of-the-art classification algorithms (SVM, Random Forest). Our findings provide evidence that HMMs can encode the relationship between effort and behaviour (captured by the multimodal data) in a more efficient way than the other methods. Foremost, a practical implication of the approach is that the derived HMMs also pinpoint the moments to provide preventive/prescriptive feedback to the learners in real-time, by building-upon the relationship between behavioural patterns and the effort the learners are putting in

Cognitive Decay And Memory Recall During Long Duration Spaceflight

This dissertation aims to advance the efficacy of Long-Duration Space Flight (LDSF) pre-flight and in-flight training programs, acknowledging existing knowledge gaps in NASA\u27s methodologies. The research\u27s objective is to optimize the cognitive workload of LDSF crew members, enhance their neurocognitive functionality, and provide more meaningful work experiences, particularly for Mars missions.The study addresses identified shortcomings in current training and learning strategies and simulation-based training systems, focusing on areas requiring quantitative measures for astronaut proficiency and training effectiveness assessment. The project centers on understanding cognitive decay and memory loss under LDSF-related stressors, seeking to establish when such cognitive decline exceeds acceptable performance levels throughout mission phases. The research acknowledges the limitations of creating a near-orbit environment due to resource constraints and the need to develop engaging tasks for test subjects. Nevertheless, it underscores the potential impact on future space mission training and other high-risk professions. The study further explores astronaut training complexities, the challenges encountered in LDSF missions, and the cognitive processes involved in such demanding environments. The research employs various cognitive and memory testing events, integrating neuroimaging techniques to understand cognition\u27s neural mechanisms and memory. It also explores Rasmussen\u27s S-R-K behaviors and Brain Network Theory’s (BNT) potential for measuring forgetting, cognition, and predicting training needs. The multidisciplinary approach of the study reinforces the importance of integrating insights from cognitive psychology, behavior analysis, and brain connectivity research. Research experiments were conducted at the University of North Dakota\u27s Integrated Lunar Mars Analog Habitat (ILMAH), gathering data from selected subjects via cognitive neuroscience tools and Electroencephalography (EEG) recordings to evaluate neurocognitive performance. The data analysis aimed to assess brain network activations during mentally demanding activities and compare EEG power spectra across various frequencies, latencies, and scalp locations. Despite facing certain challenges, including inadequacies of the current adapter boards leading to analysis failure, the study provides crucial lessons for future research endeavors. It highlights the need for swift adaptation, continual process refinement, and innovative solutions, like the redesign of adapter boards for high radio frequency noise environments, for the collection of high-quality EEG data. In conclusion, while the research did not reveal statistically significant differences between the experimental and control groups, it furnished valuable insights and underscored the need to optimize astronaut performance, well-being, and mission success. The study contributes to the ongoing evolution of training methodologies, with implications for future space exploration endeavors

Leveraging Multimodal Learning Analytics to Understand How Humans Learn with Emerging Technologies

Major education and training challenges are plaguing the United States in preparing the next generation of the future workforce to meet the demands of the 21st Century. Several calls have been released to improve education programs to ensure learners are acquiring 21st century knowledge, skills, and abilities (KSAs). As we embark on the digital and automation ages of the 21st century, it is essential that we move away from traditional education programs that define and measure KSAs as static constructs (e.g., standardized assessments) with little consideration of the actual real-time deployment of these processes, missing critical information on the degree to which learners are acquiring and applying 21st century KSAs. The objective of this dissertation is to use 1 book chapter and 2 journal articles to illustrate the value in leveraging emerging technologies and multimodal trace data to define and measure scientific thinking, reflection, and self-regulated learning--core 21st century skills, across contexts, domains, tasks, and populations (e.g., medical versus undergraduates versus middle-school students). Chapters 2-4 of this dissertation provide evidence of ways to leverage multimodal trace data guided by theoretical perspectives in cognitive and learning sciences, with a special focus in self-regulated learning, to assess the extent to which learners engaged in scientific thinking, reflection, and self-regulated learning during learning activities with emerging technologies. Overall, results from these chapters illustrate that it is necessary to utilize methods that capture learning processes as they unfold during learning activities that are guided by theoretical perspectives in self-regulated learning. Findings from this research hold significant broader impacts for addressing the education and training challenges in the United States by collecting multimodal trace data over the course of learning to not only detect and identify how learners are developing KSAs such as scientific thinking, reflection, and self-regulated learning, but where these data could be fed into an intelligent and adaptive system to repurpose it back to trainers, teachers, instructors, and learners for just-in-time interventions and individualized feedback. The intellectual merit of this dissertation focuses predominantly on the importance of utilizing rich streams of multimodal trace data that are mapped onto different theoretical perspectives on how humans self-regulate across tasks like clinical reasoning, scientific thinking, and reflection with emerging technologies such as a game-based learning environment called Crystal Island. Discussion is incorporated around ways to leverage multimodal trace data on undergraduate, middle-school, and medical student populations across a range of tasks including learning about microbiology to problem solving with a game-based learning environment called Crystal Island and clinically reasoning about diagnoses across emerging technologies