MOOCs Meet Measurement Theory: A Topic-Modelling Approach

This paper adapts topic models to the psychometric testing of MOOC students based on their online forum postings. Measurement theory from education and psychology provides statistical models for quantifying a person's attainment of intangible attributes such as attitudes, abilities or intelligence. Such models infer latent skill levels by relating them to individuals' observed responses on a series of items such as quiz questions. The set of items can be used to measure a latent skill if individuals' responses on them conform to a Guttman scale. Such well-scaled items differentiate between individuals and inferred levels span the entire range from most basic to the advanced. In practice, education researchers manually devise items (quiz questions) while optimising well-scaled conformance. Due to the costly nature and expert requirements of this process, psychometric testing has found limited use in everyday teaching. We aim to develop usable measurement models for highly-instrumented MOOC delivery platforms, by using participation in automatically-extracted online forum topics as items. The challenge is to formalise the Guttman scale educational constraint and incorporate it into topic models. To favour topics that automatically conform to a Guttman scale, we introduce a novel regularisation into non-negative matrix factorisation-based topic modelling. We demonstrate the suitability of our approach with both quantitative experiments on three Coursera MOOCs, and with a qualitative survey of topic interpretability on two MOOCs by domain expert interviews.Comment: 12 pages, 9 figures; accepted into AAAI'201

A Probabilistic Approach to Modeling Socio-Behavioral Interactions

In our ever-increasingly connected world, it is essential to build computational models that represent, reason, and model the underlying characteristics of real-world networks. Data generated from these networks are often heterogeneous, interlinked, and exhibit rich multi-relational graph structures having unobserved latent characteristics. My work focuses on building computational models for representing and reasoning about rich, heterogeneous, interlinked graph data. In my research, I model socio-behavioral interactions and predict user behavior patterns in two important online interaction platforms: online courses and online professional networks. Structured data from these interaction platforms contain rich behavioral and interaction data, and provide an opportunity to design machine learning methods for understanding and interpreting user behavior. The data also contains unstructured data, such as natural language text from forum posts and other online discussions. My research aims at constructing a family of probabilistic models for modeling social interactions involving both structured and unstructured data. In the early part of this thesis, I present a family of probabilistic models for online courses for: 1) modeling student engagement, 2) predicting student completion and dropouts, 3) modeling student sentiment toward various course aspects (e.g., content vs. logistics), 4) detecting coarse and fine-grained course aspects (e.g., grading, video, content), and 5) modeling evolution of topics in repeated offerings of online courses. These methods have the potential to improve student experience and focus limited instructor resources in ways that will have the most impact. In the latter part of this thesis, I present methods to model multi-relational influence in online professional networks. I test the effectiveness of this model via experimentation on the professional network, LinkedIn. My models can potentially be adapted to address a wide range of problems in real-world networks including predicting user interests, user retention, personalization, and making recommendations

Indicators for enhancing learners’ engagement in massive open online courses: A systematic review

Massive open online courses (MOOCs) have paved a new learning path for the 21st-century world. The potential to reach a massive geographically dispersed audience is one of the major advantages of MOOCs. Moreover, they can be offered on a self-paced and self-regulated basis and have become an integral part of lifelong learning, especially in workplaces. However, one persistent problem is the lack of learners’ engagement. A harmonisation of studies providing a holistic view into aggregating indicators for enhancing learners’ engagement in MOOCs is lacking. The coronavirus pandemic has accelerated MOOC adoption, and learners’ engagement in MOOCs has become even more essential for the success of this educational innovation. We examine the existing literature to derive indicators important for enhancing learners’ engagement in MOOC learning environments. Using a systematic approach, 83 empirical studies were examined, and 10 indicators were identified as important considerations for enhancing learners’ engagement while designing MOOCs—from initiatives for individual learners to platform and instructional design perspectives. We also present a table describing these indicators and offer a structured discussion on each one. We believe the results provide guidelines for MOOC designers and instructors, educational policymakers, higher education institutions, and MOOC engagement researchers.Peer reviewe

Student Engagement in Aviation Moocs: Identifying Subgroups and Their Differences

The purpose of this study was to expand the current understanding of learner engagement in aviation-related Massive Open Online Courses (MOOCs) through cluster analysis. MOOCs, regarded for their low- or no-cost educational content, often attract thousands of students who are free to engage with the provided content to the extent of their choosing. As online training for pilots, flight attendants, mechanics, and small unmanned aerial system operators continues to expand, understanding how learners engage in optional aviation-focused, online course material may help inform course design and instruction in the aviation industry. In this study, Moore’s theory of transactional distance, which posits psychological or communicative distance can impede learning and success, was used as a descriptive framework for analysis. Archived learning analytics datasets from two 2018 iterations of the same small unmanned aerial systems MOOC were cluster-analyzed (N = 1,032 and N = 4,037). The enrolled students included individuals worldwide; some were affiliated with the host institution, but most were not. The data sets were cluster analyzed separately to categorize participants into common subpopulations based on discussion post pages viewed and posts written, video pages viewed, and quiz grades. Subgroup differences were examined in days of activity and record of completion. Pre- and postcourse survey data provided additional variables for analysis of subgroup differences in demographics (age, geographic location, education level, employment in the aviation industry) and learning goals. Analysis of engagement variables revealed three significantly different subgroups for each MOOC. Engagement patterns were similar between MOOCs for the most and least engaged groups, but differences were noted in the middle groups; MOOC 1’s middle group had a broader interest in optional content (both in discussions and videos); whereas MOOC 2’s middle group had a narrower interest in optional discussions. Mandatory items (Mandatory Discussion or Quizzes) were the best predictors in classifying subgroups for both MOOCs. Significant associations were found between subgroups and education levels, days of activity, and total quiz scores. This study addressed two known problems: a lack of information on student engagement in aviation-related MOOCs, and more broadly, a growing imperative to examine learners who utilize MOOCs but do not complete them. This study served as an important first step for course developers and instructors who aim to meet the diverse needs of the aviation-education community

Augmenting Learning Activities with Contextual Information Scent

Students often have information needs while carrying out a multitude of learning activities at universities. When information is needed for investigating a problem, the student may interrupt the work and switch to an information seeking task. As Internet connectivity becomes ubiquitous, searching information has been routinized and integrated in the learning experience. However, information needs are not always fully recognized, or they can not be well articulated. A MOOC student may perceive a video to be difficult, but fails to express what information can be helpful. Sometimes it is improper to interrupt the learning task for searching information, especially when social factors are concerned, e.g. in a seminar talk. These situations create research potentials for making ambient information cues, hereafter referred to as contextual information scent (CIS), available to address students' situational information needs in learning activities. The CIS is designed to combine context-awareness with information seeking, ambient interaction as well as serendipitous encounter. In this thesis, we investigate the CIS mainly in collaborative learning activities. We explore three different contexts: conversation, groupware interaction and video content for MOOC learning. RaindropSearch investigates capturing conversational words as CIS for building search queries, while the TileSearch triggers Web searches based on group discussions and retrieved image and Wikipedia results as CIS for serendipitous interactions. These two explorations both focus on conversation context and provide initial insights into the CIS design practice. Next, we present MeetHub Search, which includes three CIS components based on text interactions in a groupware. Our last prototype, the BOOC Player employs textbook pages as CIS and links them to MOOC videos during the course of collaborative video viewing. All prototypes show how we manipulated design parameters to reduce distraction, increase relevance and ensure timeliness. The studies also exhibit the influence of group dynamics on the use of CIS. We finally extend our research scope to individual MOOC learning and summarize the design insights obtained from MOOC analytics. The contributions of this thesis are summarized as (1) a dedicated research framework derived from both research literature and requirement analysis for recognizing the design challenges, design principles and design space of CIS. The framework lays the foundation for us to explore different contexts in this thesis, where we generated (2) design implications that identify the key attributes of CIS. Last but not least, we employed (3) a variety of evaluation methodologies in this thesis for assessing the usability as well as the benefit and appeal of CIS

AI in Learning: Designing the Future

AI (Artificial Intelligence) is predicted to radically change teaching and learning in both schools and industry causing radical disruption of work. AI can support well-being initiatives and lifelong learning but educational institutions and companies need to take the changing technology into account. Moving towards AI supported by digital tools requires a dramatic shift in the concept of learning, expertise and the businesses built off of it. Based on the latest research on AI and how it is changing learning and education, this book will focus on the enormous opportunities to expand educational settings with AI for learning in and beyond the traditional classroom. This open access book also introduces ethical challenges related to learning and education, while connecting human learning and machine learning. This book will be of use to a variety of readers, including researchers, AI users, companies and policy makers