Predicting Learners' Success in a Self-paced MOOC Through Sequence Patterns of Self-regulated Learning

Proceeding of: 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3-5, 2018.In the past years, predictive models in Massive Open Online Courses (MOOCs) have focused on forecasting learners' success through their grades. The prediction of these grades is useful to identify problems that might lead to dropouts. However, most models in prior work predict categorical and continuous variables using low-level data. This paper contributes to extend current predictive models in the literature by considering coarse-grained variables related to Self-Regulated Learning (SRL). That is, using learners' self-reported SRL strategies and MOOC activity sequence patterns as predictors. Lineal and logistic regression modelling were used as a first approach of prediction with data collected from N = 2,035 learners who took a self-paced MOOC in Coursera. We identified two groups of learners: (1) Comprehensive, who follow the course path designed by the teacher; and (2) Targeting, who seek for the information required to pass assessments. For both type of learners, we found a group of variables as the most predictive: (1) the self-reported SRL strategies 'goal setting', 'strategic planning', 'elaboration' and 'help seeking'; (2) the activity sequences patterns 'only assessment', 'complete a video-lecture and try an assessment', 'explore the content' and 'try an assessment followed by a video-lecture'; and (3) learners' prior experience, together with the self-reported interest in course assessments, and the number of active days and time spent in the platform. These results show how to predict with more accuracy when students reach a certain status taking in to consideration not only low-level data, but complex data such as their SRL strategies.This work was supported by FONDECYT (Chile) under project initiation grant No.11150231, the MOOC-Maker Project (561533-EPP-1-2015-1-ES-EPPKA2-CBHE-JP), the LALA Project (586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP), and CONICYT/DOCTORADO NACIONAL 2016/21160081, the Spanish Ministry of Education, Culture and Sport, under an FPU fellowship (FPU016/00526) and the Spanish Ministry of Economy and Competiveness (Smartlet project, grant number TIN2017-85179-C3-1-R) funded by the Agencia Estatal de Investigación (AEI) and Fondo Europeo de Desarrollo Regional (FEDER).Publicad

Analisi di tassi di completamento e abbandono nei MOOC di EduOpen

The completion rate of massive open online courses (MOOCs) is generally less than 10% of participants. This is due to several factors, many of which cannot be eliminated: spontaneous enrolment, participants’ extreme heterogeneity, self-regulated processes and differences in motivational and cultural profiles. One of the factors that can affect the rate of completing a MOOC is the modality of delivery. The active presence of theteacher and of other support figures in MOOCs, even where criticality is linked to the number of students and the management of the dynamics present in the online learning environment, can qualitatively and quantitatively affect both the levels of interaction and participation of the users and the completion percentages of the course itself. The MOOCs published on the EduOpen Portal provide two specific methods of use: selfpaced and tutoring. The choice of modality, which is defined in the design phase, “impacts” the structure and timing of the course itself, its learning objectives and the types of teaching resources. Consequently, the levels of interaction and evaluation processes are also “calibrated” in relation to the “presence or absence” of support figures in the online environment. The contribution, starting from the first data generated by the Learning Analytics system of the Portal, focuses on analysis of the percentage of the completion/ dropout rate recorded for the entire group of MOOCs published in relation to the delivery methods defined in the design phase of the various courses. In July 2019 there were 247 courses in the catalogue with more than 55,000 users. The final objective of the analysis is to include in the guidelines for the design of a MOOC the results of this first study.Il tasso di completamento di MOOCs e generalmente inferiore al 10% degli iscritti. Questo a causa di diversi fattori, molti non eliminabili, quali: reclutamento spontaneo, estrema eterogeneità degli iscritti, processi di autoregolazione, differenze nei profili motivazionali e culturali. Uno dei fattori che può incidere sul tasso di completamento di un MOOC e rappresentato dalla modalità di erogazione. La presenza attiva del docente e di altre figure di supporto in corsi MOOCs, se pur con le evidenti criticità legate alla numerosità degli studenti e alla gestione delle dinamiche presenti dall’ambiente di apprendimento online può incidere (qualitativamente e quantitativamente) sia sui livelli di interazione e partecipazione degli utenti sia sulle percentuali di completamento del corso stesso. I MOOCs pubblicati sul Portale EduOpen prevedono nello specifico due modalità di fruizione: autoapprendimento e tutorata. La scelta della modalità - definita in fase progettuale - “impatta” sulla struttura e sulle tempistiche stesse del corso, sugli obiettivi di apprendimento e sulla tipologia delle risorse didattiche. Di conseguenza, i livelli di interazione e i processi di valutazione sono “calibrati” anche in relazione “alla presenza o all’assenza” di figure di supporto nell’ambiente online. Il contributo, a partire dai primi dati generati dal sistema di Learning Analytics del portale, si focalizza sull’analisi delle percentuali di completamento/tasso di abbandono registrate sull’intero insieme di MOOCs pubblicati in relazione alle modalità di erogazione definite nella fase di progettazione dei vari corsi. A luglio 2019 i corsi presenti nel catalogo sono 247 con un numero di utenti superiore a 55000 utenti. L’obiettivo finale dell’analisi e quello di includere nelle linee guida alla progettazione dei MOOCs i risultati emersi da questa prima ricerca

From procrastination to engagement? An experimental exploration of the effects of an adaptive virtual assistant on self-regulation in online learning

Compared to traditional classroom learning, success in online learning tends to depend more on the learner’s skill to self-regulate. Self-regulation is a complex meta-cognitive skill set that can be acquired. This study explores the effectiveness of a virtual learning assistant in terms of (a) developmental, (b) general compensatory, and (c) differential compensatory effects on learners’ self-regulatory skills in a sample of N = 157 online learners using an experimental intervention-control group design. Methods employed include behavioural trace data as well as self-reporting measures. Participants provided demographic information and responded to a 24-item self-regulation questionnaire and a 20-item personality trait questionnaire. Results indicate that the adaptive assistance did not lead to substantial developmental shifts as captured in learners’ perceived levels of self-regulation. However, various patterns of behavioural changes emerged in response to the intervention. This suggests that the virtual learning assistant has the potential to help online learners effectively compensate for deficits (in contrast to developmental shifts) in self-regulatory skills that might not yet have been developed

Analysing self-regulated learning strategies of MOOC learners through self-reported data

Massive open online courses (MOOCs) require registered learners to be autonomous in their learning. Nevertheless, prior research studies showed that many learners lack the necessary self-regulated learning (SRL) skills to succeed in MOOCs. This research study aimed to gain insights into the relationships that exist between SRL and background information from MOOC learners. To this end, a series of three MOOCs on computer programming offered through edX were used to collect self-reported data from learners using an adaptation of the Motivated Strategies for Learning Questionnaire. Results show significant differences in general learning strategies and motivation by continent, prior computing experience and percentage of completed MOOCs. Men reported higher motivation than women, whereas pre-university learners needed further guidance to improve their learning strategies.This work was supported in part by the FEDER/Ministerio de Ciencia, Innovación y Universidades-Agencia Estatal de Investigación, through the Smartlet Project under Grant TIN2017-85179-C3-1-R, and in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS-4307, a project which is co-funded by the European Structural Funds (FSE and FEDER). Partial support has also been received from the European Commission through Erasmus+ Capacity Building in the Field of Higher Education projects, more specifically through projects LALA, InnovaT and PROF-XXI (586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP), (598758-EPP-1-2018-1-AT-EPPKA2-CBHE-JP), (609767-EPP-1-2019-1-ES-EPPKA2-CBHE-JP). This publication reflects the views only of the authors, and funders cannot be held responsible for any use which may be made of the information contained therein

Generalizing predictive models of admission test success based on online interactions

This article belongs to the Special Issue Sustainability of Learning AnalyticsTo start medical or dentistry studies in Flanders, prospective students need to pass a central admission test. A blended program with four Small Private Online Courses (SPOCs) was designed to support those students. The logs from the platform provide an opportunity to delve into the learners' interactions and to develop predictive models to forecast success in the test. Moreover, the use of different courses allows analyzing how models can generalize across courses. This article has the following objectives: (1) to develop and analyze predictive models to forecast who will pass the admission test, (2) to discover which variables have more effect on success in different courses, (3) to analyze to what extent models can be generalized to other courses and subsequent cohorts, and (4) to discuss the conditions to achieve generalizability. The results show that the average grade in SPOC exercises using only first attempts is the best predictor and that it is possible to transfer predictive models with enough reliability when some context-related conditions are met. The best performance is achieved when transferring within the same cohort to other SPOCs in a similar context. The performance is still acceptable in a consecutive edition of a course. These findings support the sustainability of predictive models.This work was partially funded by the LALA project (grant no. 586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP). The LALA project has been funded with support from the European Commission. In addition, this work has been partially funded by FEDER/Ministerio de Ciencia, Innovación y Universidades—Agencia Estatal de Investigación/project Smartlet (TIN2017-85179-C3-1-R) and by the Madrid Regional Government through the project e-Madrid-CM (S2018/TCS-4307). The latter is also cofinanced by the Structural Funds (FSE and FEDER). It has also been supported by the Spanish Ministry of Science, Innovation, and Universities, under an FPU fellowship (FPU016/00526

학습자 동기 부여 지원을 위한 MOOCs 인터페이스 개발

학위논문(석사) -- 서울대학교대학원 : 사범대학 교육학과(교육공학전공), 2022.2. Young Hoan Cho.With the development of information and communication technology (ICT), many people get educated not only in traditional classrooms but also online nowadays. As one way of online education, the market of MOOCs (Massive Open Online Courses) has been growing continuously since the first platform was opened to the public in 2012. Today in 2021, the number of MOOCs learners has reached up to 220 million, and MOOCs are playing an irreplaceable role in higher education, lifelong education, corporate education, etc. Although we can expect that MOOCs will become increasingly important in the field of education, with its rapid growth during the last decade, some issues of it have been exposed. One of the issues is the low completion rate. Compared to traditional education, MOOCs learners are reported more likely to drop out, which leads to the average completion rate at around 10%. According to previous studies, one of the reasons that causes this phenomenon is lacking motivation. As a part that interacts directly with users of an application, interface is crucial because it offers affordance and determines the way users use the application. And the interface becomes even more important when it comes to E-learning because motivators, which can affect learners' motivation, can be designed in the user interface. However, studies have shown that the current interface design of MOOCs lacks motivation factors and fails to facilitate interactive communication among MOOCs learners. Therefore, in this research, a MOOCs interface that focuses on improving learners' motivation was designed. To achieve this goal, the research questions considered were: 1) What are the interface design guidelines and interface functions to motivate MOOCs learners to sustain their learning? 2) What is the interface to motivate MOOCs learners to sustain their learning? and 3) What are the learners' responses to the interface? To answer the research questions, the type 1 design and development methodology proposed by Richey and Klein was followed. First, MOOCs interface design guidelines were derived by literature review and followed by 2 rounds of expert review conducted by 4 experts to ensure the internal validity. Second, a prototype of MOOCs interface was designed based on the guidelines by using prototyping tool Figma. Third, the prototype was given to 5 learners along with a series of tasks for learner response tests to ensure the external validity of the design guidelines, and based on the result, both the prototype and the guidelines were revised. The final version of the MOOCs interface design guidelines consists of 3 motivational design principles （Autonomy, Competence, and Relatedness）, 12 motivational design guidelines (5 autonomy-supported, 4 competence-supported, and 3 relatedness-supported) along with 34 design guidelines developed for MOOCs interface. Based on these design guidelines, the functions of the MOOCs interface in this research were designed. Based on the autonomy-supported guidelines, functions such as learning mode selection (self-paced, scheduled, premiere), learning group, learning activity, goal setting, dashboard, reminder, recommendation, and feedback were designed. And based on the competence-supported guidelines, functions such as account register, course enrollment, learning path, team activity support, dashboard, and goal setting were designed. Meanwhile, based on the relatedness-supported guidelines, functions such as dashboard, feedback, keyword checklist, learning group, chatting window, group/team activity, course evaluation, team assignment, mind map, and note were designed. The participating learners were satisfied with the design. The survey data showed that learners’ general perceptions of the MOOCs interface reached 4.44, perceived autonomy reached 4.40, perceived competence reached 4.52, and perceived relatedness reached 4.66 (5 points Likert scale). The in-depth interview data was open coded into three categories: 1) Advantages of the MOOCs interface, 2) Problems with the MOOCs interface, and 3) Suggestions for improvement. The advantages include providing choices for autonomy support, providing scaffolding and adaptive learning for competence support, providing interactive learning for relatedness support, and providing novel meanwhile helpful functions that existing platforms don’t have. The problems include lacking tutorials for novel functions, inconsistent icons and choice of words, and improper positioning and interaction. The suggestions for improvement include adding the wiki function, adding the reminder function, and visualizing the timetable. The significance of this research can be summarized as follows: 1) proposed an intrinsic motivation oriented MOOCs interface. 2) introduced three learning modes to the MOOCs learning environment. 3) introduced the learning group and learning team to the MOOCs environment to facilitate learners’ interaction. 4) provided an example of the dashboard for the context of MOOCs. And 5) provided insight into how to help learners achieve personalized learning in the MOOCs environment.정보통신기술(ICT)의 발달로 오늘날 많은 사람이 전통적인 교실에서뿐만 아니라 온라인으로도 교육을 받고 있다. 온라인 교육의 한 방법으로 MOOC(Massive Open Online Courses)의 시장은 2012년 첫 플랫폼이 공개된 이후 지속적으로 성장하고 있으며, 교육학자에게 많은 관심을 받고 있다. 특히 MOOC는 고등교육, 평생교육, 기업교육 등에서 대체할 수 없는 역할을 한다는 연구 결과가 속속히 나오고 있다. 교육 분야에서 MOOC가 점점 더 중요해질 것이라고 예상되지만, 지난 10년 동안 MOOC의 급속한 성장과 함께 MOOC의 문제점들이 일부 발견되었다. 그중 하나는 낮은 이수율로서, 기존 교육에 비해 MOOC 학습자는 평균 이수율이 약 10%에 머무를 정도로 중도 탈락할 가능성이 높은 것으로 보고되었다. 선행 연구에 따르면 이러한 현상을 일으키는 원인 중 하나는 부족한 동기부여 때문이다. 사용자 인터페이스는 애플리케이션에서 사용자와 직접 상호 작용하는 부분으로서 어포던스를 제공하고, 사용자가 애플리케이션을 사용하는 방식을 결정하기 때문에 매우 중요하다고 말할 수 있다. 특히, E-learning 상황에서는 인터페이스가 더욱 중요한데, 이는 동기 부여에 영향을 미칠 수 있는 동기 요소들이 사용자 인터페이스에 설계될 수 있기 때문이다. 그러나 선행 연구에 따르면 현재의 MOOC 인터페이스 디자인은 동기 요소가 부족하고 MOOC 학습자 간의 상호 작용을 촉진하지 못하고 있다. 이에 따라서 본 연구에서는 학습자의 동기부여에 초점을 맞추어 MOOC 인터페이스를 설계하였고, 이를 위해 다음의 연구 문제들을 고려하였다. 1) MOOC 학습자가 학습을 지속하도록 동기를 부여하는 인터페이스 디자인 가이드라인은 무엇인가? 2) 인터페이스 디자인 가이드라인에 따른 예시적인 인터페이스는 무엇인가? 3) 인터페이스에 대한 학습자의 반응은 무엇인가? 연구 질문에 답하기 위해 Richey와 Klein이 제안한 설계ㆍ개발 방법론 중 유형 1을 따랐다. 먼저 문헌검토를 통해 MOOC 인터페이스 디자인 가이드라인을 도출하고, 내적 타당성을 확보하기 위해 4명의 전문가가 2차례에 걸쳐 전문가검토를 실시하였다. 다음으로, 프로토타이핑 도구인 Figma를 사용하여 가이드라인에 따라 MOOC 인터페이스의 프로토타입을 개발했다. 마지막으로, 디자인 가이드라인의 외적 타당성을 확보하기 위해 2차례에 걸쳐 사용성 평가를 하고 학습자의 반응을 측정했다. 일련의 과제와 함께 프로토타입을 5명의 학습자에게 제공하고, 그 결과를 바탕으로 프로토타입과 가이드라인을 모두 수정했다. MOOC 인터페이스 디자인 가이드라인의 최종 버전은 3가지 동기 부여 디자인 원리 (자율성, 유능성 및 관계성), 12가지 동기 부여 디자인 가이드라인 (5개 자율성 지원, 4개 유능성 지원, 3개 관계성 지원)과 34개의 MOOC 인터페이스 디자인 가이드라인으로 구성되었다. 본 연구에서 개발한 MOOC 인터페이스는 이러한 MOOC 인터페이스 디자인 가이드라인에 따른 기능을 포함하고 있다. 자율성 지원 가이드라인을 반영하는 기능으로 학습 모드 선택 (self-paced, scheduled, premiere), 학습 그룹, 학습활동 선택, 학습목표 설정, 대시보드, 리마인더, 추천, 피드백 등 기능이 있었고, 유능성 지원 가이드라인을 반영하는 기능으로 레지스터, 수업 등록, 학습 경로, 팀 활동 지원, 대시보드, 학습목표 설정 등 기능이 있었으며, 관계성 지원 가이드라인을 반영하는 기능으로 대시보드, 피드백, 키워드 체크리스트, 학습 그룹, 채팅창, 그룹/팀 활동, 수업 평가, 팀 과제, 마인드맵, 노트 등 기능이 있었다. 사용성 평가에 참여한 학습자들은 개발한 MOOCs 인터페이스에 만족했으며, 5점 척도 구성한 설문 문항으로 조사한 결과, 인터페이스 전반에 대한 인식 4.44점, 자율성에 대한 인식 4.40점, 유능성에 대한 인식 4.52점, 관계성에 대한 인식 4.66점이었다. 그리고 심층 인터뷰에서 취득한 질적 데이터를 오픈 코딩을 통해서 정리한 결과, 개발한 MOOCs 인터페이스의 장점, 문제점, 개선점을 도출했다. 장점으로 자율성 지원을 위한 선택 제공, 유능성 지원을 위한 스캐폴딩과 적응형 학습, 관계성 지원을 위해서 제공하는 상호작용, 그리고 기존 플랫폼과의 의미있는 차이점 등이 있었다. 문제점으로 기존 플랫폼들이 제공하지 않은 새로운 기능에 대한 가이드가 필요하다는 점, 불일치한 아이콘과 용어, 부적절한 포지셔닝과 인터랙션 등이 있었다. 개선점으로 위키 기능, 알림 메시지 기능을 추가하고 시간표를 시각화 등이 있었다. 본 연구의 의의는 다음과 같이 요약될 수 있다. 1) 내재적 동기부여 지향한 MOOC 인터페이스를 제안하였다. 2) MOOC 학습 환경에 3가지 학습 모드를 도입했다. 3) 학습자의 상호 작용을 촉진하기 위해 학습 그룹과 학습 팀을 MOOC 학습 환경에 도입했다. 4) MOOC 상황에 대시보드의 예시를 제공했다. 그리고 5) 학습자가 MOOC 환경에서 맞춤형 학습을 달성하도록 돕는 방법에 대한 인사이트를 제공했다.Chapter 1. Introduction 1 1.1. Problem Statement 1 1.2. Purpose of the Research 4 1.3. Research Questions 8 1.4. Definition of Terms 9 1.4.1. MOOCs 9 1.4.2. Motivation 11 1.4.3. User Interface 12 Chapter 2. Literature Review 14 2.1. MOOCs 14 2.1.1. Characteristics and Meaning 14 2.1.2. History of MOOCs 16 2.1.3. MOOCs Platforms and MOOCs Learners 20 2.1.4. The Critiques and Drop Out Phenomenon 26 2.2. Motivation 35 2.2.1. Motivation in Learning 35 2.2.2. Motivation Theories 37 2.3.1. User Interface Design & Interface Design for Education 45 2.3.2 Motivation Supported User Interface Design 48 Chapter 3. Research Method 54 3.1. Design and Development Methodology 54 3.2. Research Participants 58 3.3. Research Tools 60 3.3.1. Internal Validation Tools 60 3.3.2. Prototyping Tool 60 3.3.3. External Validation Tools 62 3.4. Data Collection and Analysis 65 3.4.1. Expert Review 65 3.4.2. Learners' Responses 66 Chapter 4. Findings 68 4.1. The MOOCs Interface Design Guidelines 68 4.1.1. The Final Version of the MOOCs Interface Design Guidelines 69 4.1.2. The Results of the Expert Review 80 4.2. Prototype of the MOOCs Interface 85 4.3. Learners' Responses to the MOOCs Interface 118 4.3.1. Learners' Response to the MOOCs Interface (First Usability Test) 119 4.3.2. Learners' Responses to the Revised Interface (Second Usability Test) 143 Chapter 5. Discussion and Conclusion 145 5.1. Discussion 146 5.2. Conclusion 151 Reference 154 APPENDIX 1 175 APPENDIX 2 189 APPENDIX 3 197 APPENDIX 4 206 APPENDIX 5 213 APPENDIX 6 220 국문초록 224석

An Experimental Study of Self-regulated Learning Strategies Application in Moocs

Online learning has been widely adopted in higher education to reach students who typically would not have a chance to complete accredited courses (Kentnor, 2015). Massive open online courses (MOOC), which is a type of online learning, makes it easier for people to take university courses with internet access and a fraction of cost compared to traditional residential programs (Reich, 2020). MOOCs also become popular for those who want to increase their professional profile or advance their academic career (Pheatt, 2017). However, online learning has long been criticized for its universally low completion rates, high dropout rate and poor learning performance (Almeda et al., 2018). This phenomenon is more exacerbated in MOOC environments. Historical studies have attempted to support learner self-regulated learning (SRL) activities in order to enhance completion rates and academic outcomes. Prior studies have conducted pre-course questionnaires as inexpensive SRL interventions to prompt learners as SRL support(Kizilcec et al., 2017, Kizilcec & Cohen, 2017; Kizilcec et al., 2020; Yeomans & Reich, 2017). Yet, these one-time-only, short-term interventions only yield limited or no effects. This study implemented and evaluated the effectiveness of an alternative intervention, the self-regulated learning user interface (SRLUI), to support students' self-regulated learning (SRL) strategies in a MOOC environment. SRLUI is based on Zimmerman’s (2000) SRL model and develops learner’s SRL skills through longitudinal, recurring practice of multiple SRL dimensions activities (i.e., goal setting, self-evaluation, task planning, setting reminders) with content-specific information. The study utilized a randomized experimental design and implemented SRLUI in eight MOOCs with a total of 808 participants. The results indicated a higher usage rate of SRL support compared to the historical findings, which may be owing to the SRL support embedded into the learning activities throughout the course. Also, the study showed improved learning outcomes for a subgroup of participants, but there was no reduction in the number of dropouts. Based on the findings of this study, it is recommended that a personalized SRL tool featuring content-specific information should be embedded in online courses. The research design also recorded direct cognitive records of learners' SRL activities, which yield stronger validity compared to trace and survey data. The result suggested SRLUI might only benefit a subgroup of learners with passing grades. Thus, it is recommended that future research identify various subgroups of learner profiles in MOOC environments and to consider how to reach and support learners in different subgroups

Open World Learning

This book provides state-of-the-art contemporary research insights into key applications and processes in open world learning. Open world learning seeks to understand access to education, structures, and the presence of dialogue and support systems. It explores how the application of open world and educational technologies can be used to create opportunities for open and high-quality education. Presenting ground-breaking research from an award winning Leverhulme doctoral training programme, the book provides several integrated and cohesive perspectives of the affordances and limitations of open world learning. The chapters feature a wide range of open world learning topics, ranging from theoretical and methodological discussions to empirical demonstrations of how open world learning can be effectively implemented, evaluated, and used to inform theory and practice. The book brings together a range of innovative uses of technology and practice in open world learning from 387,134 learners and educators learning and working in 136 unique learning contexts across the globe and considers the enablers and disablers of openness in learning, ethical and privacy implications, and how open world learning can be used to foster inclusive approaches to learning across educational sectors, disciplines and countries. The book is unique in exploring the complex, contradictory and multi-disciplinary nature of open world learning at an international level and will be of great interest to academics, researchers, professionals, and policy makers in the field of education technology, e-learning and digital education