A LEARNER INTERACTION STUDY OF DIFFERENT ACHIEVEMENT GROUPS IN MPOCS WITH LEARNING ANALYTICS TECHNIQUES

The purpose of this study was to conduct data-driven research by employing learning analytics methodology and Big Data in learning management systems (LMSs), and then to identify and compare learners’ interaction patterns in different achievement groups through different course processes in Massive Private Online Courses (MPOCs). Learner interaction is the foundation of a successful online learning experience. However, the uncertainties about the temporal and sequential patterns of online interaction and the lack of knowledge about using dynamic interaction traces in LMSs have prevented research on ways to improve interactive qualities and learning effectiveness in online learning. Also, most research focuses on the most popular online learning organization form, Massive Open Online Courses (MOOCs), and little online learning research has been conducted to investigate learners’ interaction behaviors in another important online learning organization form: MPOCs. To fill these needs, the study pays attention to investigate the frequent and effective interaction patterns in different achievement groups as well as in different course processes, and attaches importance to LMS trace data (log data) in better serving learners and instructors in online learning. Further, the learning analytics methodology and techniques are introduced here into online interaction research. I assume that learners with different achievements express different interaction characteristics. Therefore, the hypotheses in this study are: 1) the interaction activity patterns of the high-achievement group and the low-achievement group are different; 2) in both groups, interaction activity patterns evolve through different course processes (such as the learning process and the exam process). The final purpose is to find interaction activity patterns that characterize the different achievement groups in specific MPOCs courses. Some learning analytics approaches, including Hidden Markov models (HMMs) and other related measures, are taken into account to identify frequently occurring interaction activity sequence patterns of High/Low achievement groups in the Learning/Exam processes under MPOCs settings. The results demonstrate that High-achievement learners especially focused on content learning, assignments, and quizzes to consolidate their knowledge construction in both Learning and Exam processes, while Low-achievement learners significantly did not perform the same. Further, High-achievement learners adjusted their learning strategies based on the goals of different course processes; Low-achievement learners were inactive in the learning process and opportunistic in the exam process. In addition, despite achievements or course processes, all learners were most interested in checking their performance statements, but they engaged little in forum discussion and group learning. In sum, the comparative analysis implies that certain interaction patterns may distinguish the High-achievement learners from the Low-achievement ones, and learners change their patterns more or less based on different course processes. This study provides an attempt to conduct learner interaction research by employing learning analytics techniques. In the short term, the results will give in-depth knowledge of the dynamic interaction patterns of MPOCs learners. In the long term, the results will help learners to gain insight into and evaluate their learning, help instructors identify at-risk learners and adjust instructional strategies, help developers and administrators to build recommendation systems based on objective and comprehensive information, all of which in turn will help to improve the achievements of all learner groups in specific MPOC courses

Interaction patterns in exploratory learning environments for mathematics: a sequential analysis of feedback and external representations in Chinese schools

Feedback in exploratory learning systems has been depicted as an important contributor to encourage exploration. However, few studies have explored learners’ interaction patterns associated with feedback and the use of external representations in exploratory learning environments. This study used Fractions Lab, an exploratory learning environment for mathematics, to facilitate children’s conceptual understanding of fractions in three Chinese schools. Students (n = 189) from six different classes were invited to use Fractions Lab, and 260,000 event logs were collected. Beyond demonstrating the overall efficacy of the approach, lag sequential analysis supported us in approaching a deeper understanding of patterns of interaction. The findings highlight that the design of three-levels of feedback (Socratic, guidance, and didactic-procedural feedback) played different roles in supporting students to use external representations to perform mathematical tasks in an exploratory learning environment. This study sheds light on how these interaction patterns might be applied to the Fractions Lab system in order to provide increasingly tailored support, based on cultural differences, to enhance students’ technology-mediated learning experiences

How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility

Recommendation systems are ubiquitous and impact many domains; they have the potential to influence product consumption, individuals' perceptions of the world, and life-altering decisions. These systems are often evaluated or trained with data from users already exposed to algorithmic recommendations; this creates a pernicious feedback loop. Using simulations, we demonstrate how using data confounded in this way homogenizes user behavior without increasing utility

Mathematical modelling and Problem Solving in Engineering Education

BackgroundInnovation and technology have made the 21st century engineering workplace problems more diverse and challenging. Mathematical modelling is being increasingly used as the primary form of engineering design and is fundamental in everyday engineering problem solving. The demand on engineering education is to prepare graduates to meet the challenges and needs of this rapid changing society. One of the keys to do so is to develop skills to be able to integrate knowledge from multiple domains and adapt to solving novel, complex problems in workplace. This means engineering education needs to create learning environments that are effective and mindful of the authentic practices of engineers. PurposeThe goal of this thesis is to contribute to research efforts of improving engineering education, focusing on developing students’ ability to solve mathematical modelling problems. In pursuit of this goal, this thesis examines an alternative learning design in a mathematical modelling and problem-solving course for engineers and understand how the learning design contributes to students’ learning.Scope/MethodThe two empirical studies presented in this thesis employed a qualitative case study methodology. The case under investigation is a course in mathematical modelling and problem solving offered to undergraduate engineering students at Chalmers University of Technology. The first study aimed to understand how engineering students approach mathematical modelling problems early in the course and how the course impacts their learning. The second study aimed to contribute to the knowledge base of authentic learning by examining students’ perceptions of different elements of authentic learning in the course. FindingsThe results show that students had little experience of mathematical modelling and solving realistic problems that lead to experiencing challenges early in the course. Many were unaware of the importance of understanding the problem and exploring alternatives which related to their lack of self-regulation or metacognitive skills and was impeded by different types of beliefs, attitude and expectations shaped by their prior experiences. The most important impact of the course was on students’ metacognitive development. In the analysis of students’ perception of this alternative learning environment, the results showed that students experienced elements of authentic learning in the course. Even though the tasks were not entirely ‘real’, the student experience them authentic and ‘bought in’. Students engaged in deep reflective thinking in the course and presented several mechanisms of learning that linked elements of authentic learning and the course.ConclusionThe findings in this thesis demonstrates the importance of self-regulation and beliefs in developing students’ mathematical problem-solving abilities and exemplifies how the learning environment in the course contributed to developing students’ mathematical modelling and realistic problem-solving skills as well as metacognitive skills. Furthermore, the thesis presents interesting outlook on students’ perception of authenticity in the course’s learning environment contributing to the knowledge base of authentic learning in engineering education. Finally, we recommend expanding this course’s concept to other engineering programs and offer pointers to design courses that intend to provide authentic learning experience

Distributed Load Testing by Modeling and Simulating User Behavior

Modern human-machine systems such as microservices rely upon agile engineering practices which require changes to be tested and released more frequently than classically engineered systems. A critical step in the testing of such systems is the generation of realistic workloads or load testing. Generated workload emulates the expected behaviors of users and machines within a system under test in order to find potentially unknown failure states. Typical testing tools rely on static testing artifacts to generate realistic workload conditions. Such artifacts can be cumbersome and costly to maintain; however, even model-based alternatives can prevent adaptation to changes in a system or its usage. Lack of adaptation can prevent the integration of load testing into system quality assurance, leading to an incomplete evaluation of system quality. The goal of this research is to improve the state of software engineering by addressing open challenges in load testing of human-machine systems with a novel process that a) models and classifies user behavior from streaming and aggregated log data, b) adapts to changes in system and user behavior, and c) generates distributed workload by realistically simulating user behavior. This research contributes a Learning, Online, Distributed Engine for Simulation and Testing based on the Operational Norms of Entities within a system (LODESTONE): a novel process to distributed load testing by modeling and simulating user behavior. We specify LODESTONE within the context of a human-machine system to illustrate distributed adaptation and execution in load testing processes. LODESTONE uses log data to generate and update user behavior models, cluster them into similar behavior profiles, and instantiate distributed workload on software systems. We analyze user behavioral data having differing characteristics to replicate human-machine interactions in a modern microservice environment. We discuss tools, algorithms, software design, and implementation in two different computational environments: client-server and cloud-based microservices. We illustrate the advantages of LODESTONE through a qualitative comparison of key feature parameters and experimentation based on shared data and models. LODESTONE continuously adapts to changes in the system to be tested which allows for the integration of load testing into the quality assurance process for cloud-based microservices

The student-produced electronic portfolio in craft education

The authors studied primary school students’ experiences of using an electronic portfolio in their craft education over four years. A stimulated recall interview was applied to collect user experiences and qualitative content analysis to analyse the collected data. The results indicate that the electronic portfolio was experienced as a multipurpose tool to support learning. It makes the learning process visible and in that way helps focus on and improves the quality of learning. © ISLS.Peer reviewe

Real-time Assessment, Prediction, and Scaffolding of Middle School Students’ Data Collection Skills within Physical Science Simulations

Despite widespread recognition by science educators, researchers and K-12 frameworks that scientific inquiry should be an essential part of science education, typical classrooms and assessments still emphasize rote vocabulary, facts, and formulas. One of several reasons for this is that the rigorous assessment of complex inquiry skills is still in its infancy. Though progress has been made, there are still many challenges that hinder inquiry from being assessed in a meaningful, scalable, reliable and timely manner. To address some of these challenges and to realize the possibility of formative assessment of inquiry, we describe a novel approach for evaluating, tracking, and scaffolding inquiry process skills. These skills are demonstrated as students experiment with computer-based simulations. In this work, we focus on two skills related to data collection, designing controlled experiments and testing stated hypotheses. Central to this approach is the use and extension of techniques developed in the Intelligent Tutoring Systems and Educational Data Mining communities to handle the variety of ways in which students can demonstrate skills. To evaluate students\u27 skills, we iteratively developed data-mined models (detectors) that can discern when students test their articulated hypotheses and design controlled experiments. To aggregate and track students\u27 developing latent skill across activities, we use and extend the Bayesian Knowledge-Tracing framework (Corbett & Anderson, 1995). As part of this work, we directly address the scalability and reliability of these models\u27 predictions because we tested how well they predict for student data not used to build them. When doing so, we found that these models demonstrate the potential to scale because they can correctly evaluate and track students\u27 inquiry skills. The ability to evaluate students\u27 inquiry also enables the system to provide automated, individualized feedback to students as they experiment. As part of this work, we also describe an approach to provide such scaffolding to students. We also tested the efficacy of these scaffolds by conducting a study to determine how scaffolding impacts acquisition and transfer of skill across science topics. When doing so, we found that students who received scaffolding versus students who did not were better able to acquire skills in the topic in which they practiced, and also transfer skills to a second topic when was scaffolding removed. Our overall findings suggest that computer-based simulations augmented with real-time feedback can be used to reliably measure the inquiry skills of interest and can help students learn how to demonstrate these skills. As such, our assessment approach and system as a whole shows promise as a way to formatively assess students\u27 inquiry

A Data Mining Toolbox for Collaborative Writing Processes

Collaborative writing (CW) is an essential skill in academia and industry. Providing support during the process of CW can be useful not only for achieving better quality documents, but also for improving the CW skills of the writers. In order to properly support collaborative writing, it is essential to understand how ideas and concepts are developed during the writing process, which consists of a series of steps of writing activities. These steps can be considered as sequence patterns comprising both time events and the semantics of the changes made during those steps. Two techniques can be combined to examine those patterns: process mining, which focuses on extracting process-related knowledge from event logs recorded by an information system; and semantic analysis, which focuses on extracting knowledge about what the student wrote or edited. This thesis contributes (i) techniques to automatically extract process models of collaborative writing processes and (ii) visualisations to describe aspects of collaborative writing. These two techniques form a data mining toolbox for collaborative writing by using process mining, probabilistic graphical models, and text mining. First, I created a framework, WriteProc, for investigating collaborative writing processes, integrated with the existing cloud computing writing tools in Google Docs. Secondly, I created new heuristic to extract the semantic nature of text edits that occur in the document revisions and automatically identify the corresponding writing activities. Thirdly, based on sequences of writing activities, I propose methods to discover the writing process models and transitional state diagrams using a process mining algorithm, Heuristics Miner, and Hidden Markov Models, respectively. Finally, I designed three types of visualisations and made contributions to their underlying techniques for analysing writing processes. All components of the toolbox are validated against annotated writing activities of real documents and a synthetic dataset. I also illustrate how the automatically discovered process models and visualisations are used in the process analysis with real documents written by groups of graduate students. I discuss how the analyses can be used to gain further insight into how students work and create their collaborative documents