Situating multimodal learning analytics

The digital age has introduced a host of new challenges and opportunities for the learning sciences community. These challenges and opportunities are particularly abundant in multimodal learning analytics (MMLA), a research methodology that aims to extend work from Educational Data Mining (EDM) and Learning Analytics (LA) to multimodal learning environments by treating multimodal data. Recognizing the short-term opportunities and longterm challenges will help develop proof cases and identify grand challenges that will help propel the field forward. To support the field's growth, we use this paper to describe several ways that MMLA can potentially advance learning sciences research and touch upon key challenges that researchers who utilize MMLA have encountered over the past few years

The use of tools of data mining to decision making in engineering education—A systematic mapping study

In recent years, there has been an increasing amount of theoretical and applied research that has focused on educational data mining. The learning analytics is a discipline that uses techniques, methods, and algorithms that allow the user to discover and extract patterns in stored educational data, with the purpose of improving the teaching‐learning process. However, there are many requirements related to the use of new technologies in teaching‐learning processes that are practically unaddressed from the learning analytics. In an analysis of the literature, the existence of a systematic revision of the application of learning analytics in the field of engineering education is not evident. The study described in this article provides researchers with an overview of the progress made to date and identifies areas in which research is missing. To this end, a systematic mapping study has been carried out, oriented toward the classification of publications that focus on the type of research and the type of contribution. The results show a trend toward case study research that is mainly directed at software and computer science engineering. Furthermore, trends in the application of learning analytics are highlighted in the topics, such as student retention or dropout prediction, analysis of academic student data, student learning assessment and student behavior analysis. Although this systematic mapping study has focused on the application of learning analytics in engineering education, some of the results can also be applied to other educational areas

The role of learning theory in multimodal learning analytics

This study presents the outcomes of a semi-systematic literature review on the role of learning theory in multimodal learning analytics (MMLA) research. Based on previous systematic literature reviews in MMLA and an additional new search, 35MMLA works were identified that use theory. The results show that MMLA studies do not always discuss their findings within an established theoretical framework. Most of the theory-driven MMLA studies are positioned in the cognitive and affective domains, and the three most frequently used theories are embodied cognition, cognitive load theory and control–value theory of achievement emotions. Often, the theories are only used to inform the study design, but there is a relationship between the most frequently used theories and the data modalities used to operationalize those theories. Although studies such as these are rare, the findings indicate that MMLA affordances can, indeed, lead to theoretical contributions to learning sciences. In this work, we discuss methods of accelerating theory-driven MMLA research and how this acceleration can extend or even create new theoretical knowledge

Teaching Analytics: Towards Automatic Extraction of Orchestration Graphs Using Wearable Sensors

"Teaching analytics" is the application of learning analytics techniques to understand teaching and learning processes, and eventually enable supportive interventions. However, in the case of (often, half-improvised) teaching in face-to-face classrooms, such interventions would require first an understanding of what the teacher actually did, as the starting point for teacher reflection and inquiry. Currently, such teacher enactment characterization requires costly manual coding by researchers. This paper presents a case study exploring the potential of machine learning techniques to automatically extract teaching actions during classroom enactment, from five data sources collected using wearable sensors (eye-tracking, EEG, accelerometer, audio and video). Our results highlight the feasibility of this approach, with high levels of accuracy in determining the social plane of interaction (90%, k=0.8). The reliable detection of concrete teaching activity (e.g., explanation vs. questioning) accurately still remains challenging (67%, k=0.56), a fact that will prompt further research on multimodal features and models for teaching activity extraction, as well as the collection of a larger multimodal dataset to improve the accuracy and generalizability of these methods

Monitoring, Awareness and Reflection in Blended Technology Enhanced Learning: a Systematic Review

Education is experiencing a paradigm shift towards blended learning models in technology-enhanced learning (TEL). Despite the potential benefits of blended learning, it also entails additional complexity in terms of monitoring, awareness and reflection, as learning happens across different spaces and modalities. In recent years, literature on Learning Analytics (LA) and Educational Data Mining (EDM) has gained momentum and started to address the issue. To provide a clear picture of the current state of the research on the topic and to outline open research gaps, this paper presents a systematic literature review of the state-of-the-art of research in LA and EDM on monitoring, awareness and reflection in blended TEL scenarios. The search included six main academic databases in TEL that were enriched with the proceedings of the workshop on ’Awareness and Reflection in TEL’ (ARTEL), resulting in 1089 papers out of which 40 papers were included in the final analysis

Monitoring, Awareness and Reflection in Blended Technology Enhanced Learning: a Systematic Review

Education is experiencing a paradigm shift towards blended learning models in technology-enhanced learning (TEL). Despite the potential benefits of blended learning, it also entails additional complexity in terms of monitoring, awareness and reflection, as learning happens across different spaces and modalities. In recent years, literature on Learning Analytics (LA) and Educational Data Mining (EDM) has gained momentum and started to address the issue. To provide a clear picture of the current state of the research on the topic and to outline open research gaps, this paper presents a systematic literature review of the state-of-the-art of research in LA and EDM on monitoring, awareness and reflection in blended TEL scenarios. The search included six main academic databases in TEL that were enriched with the proceedings of the workshop on ’Awareness and Reflection in TEL’ (ARTEL), resulting in 1089 papers out of which 40 papers were included in the final analysis

Computer detection of spatial visualization in a location-based task

An untapped area of productivity gains hinges on automatic detection of user cognitive characteristics. One such characteristic, spatial visualization ability, relates to users’ computer performance. In this dissertation, we describe a novel, behavior-based, spatial visualization detection technique. The technique does not depend on sensors or knowledge of the environment and can be adopted on generic computers. In a Census Bureau location-based address verification task, detection rates exceeded 80% and approached 90%

Learning and Transfer from an Engineering Design Task: The Roles of Goals, Contrasting Cases, and Focusing on Deep Structure

As maker spaces, engineering design curricula, and other hands-on active learning tasks become more popular in science classrooms, it is important to consider what students are intended to take away from these tasks. Many teachers use engineering design tasks as a means of teaching students more general science principles. However, few studies have explored exactly how the design of these activities can support more generalized student learning and transfer. Specifically, research has yet to sufficiently investigate the effects of task design components on the learning and transfer processes that can occur during these kinds of tasks. This dissertation explores how various task manipulations and focusing processes affect how well students can learn and transfers science concepts from an engineering design task. I hypothesized that learning goals that focus students on the deep structure of the problem, and contrasting cases that help students notice that deep structure, would aid learning and transfer. In two experimental studies, students were given an engineering design task. The first study was a 2x2 between subjects design where goal where goal (outcome or learning) and reflection (on contrasting cases or the engineering design process) were manipulated. A subsequent second study then gave all students contrasting cases to reflect on, and only the goal manipulation was manipulated. Results showed that learning goals improved student performance on a transfer task that required students to apply the deep structure to a different engineering design task. In the second study, learning goals improved student performance on a transfer test. Transfer performance in both studies was predicted by the ability to notice the deep structure during the reflection on contrasting cases, even though noticing this structure did not differ by goal condition. Students with a learning goal valued the learning resources they were given more during the engineering design activity, and this perceived value of resources was linked to greater learning. A qualitative case study analysis was then conducted using video data from the second study. This case study investigated noticing processes during the building process, partner dialogue, and resource use. This analysis showed how high transfer pairs were better able to focus on the deep structure of the problem. Results suggest that what students noticed didn’t differ much between the various pairs. However, high transfer pairs were better able to focus on the deep structure through establishing a joint understanding of the deep structure, sustaining concentration on that deep structure during the cases reflection, referencing resources to identify features to test, and then systematically testing those features to identify their relevance. These processes are discussed in relation to how they differ in low transfer pairs. This dissertation consists of four chapters: an intro, two standalone journal articles, and a conclusion. The first chapter provides a conceptual framing for the two journal articles, and discusses the findings from these articles in conversation. The second chapter describes the two empirical studies investigating how task goals and contrasting cases affect learning, and transfer from an engineering design task. The third chapter describes the comparative case study of how mechanisms of focusing on the deep structure differ between high and low transfer pairs. Finally, the fourth conclusion chapter discusses the implications of the work from both of these papers