THE DESIGN OF A LEARNING ANALYTICS DASHBOARD: EDUOPEN MOOC PLATFORM REDEFINITION PROCEDURES

The current EduOpen dashboard is not capable of monitoring performances and trends over the medium to long term both for the students as for the instructors; summarising and synthesising the adequate information; allowing implementation of any sort of predictive actions and functions (learning prediction). The article aims to expose the process of innovation and redefinition of a learning analytics dashboard in the EduOpen MOOC platform in order to define a model to design it accurately in terms of productivity for all users (teachers and students above all). From the literature analysis, main MOOC platform comparisons and the insights from the round tables a time spent variable is identified as at the basis of the entire user experience in online training paths. A concrete experimentation, through the design of a learning timeline and a constructive feedback system of an upcoming course in the EduOpen catalogue, is designed and explained relaying on the hypothesis of the existence of a correlation between the “time spent” (time value) and the final performance of the student

학습자 동기 부여 지원을 위한 MOOCs 인터페이스 개발

학위논문(석사) -- 서울대학교대학원 : 사범대학 교육학과(교육공학전공), 2022.2. Young Hoan Cho.With the development of information and communication technology (ICT), many people get educated not only in traditional classrooms but also online nowadays. As one way of online education, the market of MOOCs (Massive Open Online Courses) has been growing continuously since the first platform was opened to the public in 2012. Today in 2021, the number of MOOCs learners has reached up to 220 million, and MOOCs are playing an irreplaceable role in higher education, lifelong education, corporate education, etc. Although we can expect that MOOCs will become increasingly important in the field of education, with its rapid growth during the last decade, some issues of it have been exposed. One of the issues is the low completion rate. Compared to traditional education, MOOCs learners are reported more likely to drop out, which leads to the average completion rate at around 10%. According to previous studies, one of the reasons that causes this phenomenon is lacking motivation. As a part that interacts directly with users of an application, interface is crucial because it offers affordance and determines the way users use the application. And the interface becomes even more important when it comes to E-learning because motivators, which can affect learners' motivation, can be designed in the user interface. However, studies have shown that the current interface design of MOOCs lacks motivation factors and fails to facilitate interactive communication among MOOCs learners. Therefore, in this research, a MOOCs interface that focuses on improving learners' motivation was designed. To achieve this goal, the research questions considered were: 1) What are the interface design guidelines and interface functions to motivate MOOCs learners to sustain their learning? 2) What is the interface to motivate MOOCs learners to sustain their learning? and 3) What are the learners' responses to the interface? To answer the research questions, the type 1 design and development methodology proposed by Richey and Klein was followed. First, MOOCs interface design guidelines were derived by literature review and followed by 2 rounds of expert review conducted by 4 experts to ensure the internal validity. Second, a prototype of MOOCs interface was designed based on the guidelines by using prototyping tool Figma. Third, the prototype was given to 5 learners along with a series of tasks for learner response tests to ensure the external validity of the design guidelines, and based on the result, both the prototype and the guidelines were revised. The final version of the MOOCs interface design guidelines consists of 3 motivational design principles （Autonomy, Competence, and Relatedness）, 12 motivational design guidelines (5 autonomy-supported, 4 competence-supported, and 3 relatedness-supported) along with 34 design guidelines developed for MOOCs interface. Based on these design guidelines, the functions of the MOOCs interface in this research were designed. Based on the autonomy-supported guidelines, functions such as learning mode selection (self-paced, scheduled, premiere), learning group, learning activity, goal setting, dashboard, reminder, recommendation, and feedback were designed. And based on the competence-supported guidelines, functions such as account register, course enrollment, learning path, team activity support, dashboard, and goal setting were designed. Meanwhile, based on the relatedness-supported guidelines, functions such as dashboard, feedback, keyword checklist, learning group, chatting window, group/team activity, course evaluation, team assignment, mind map, and note were designed. The participating learners were satisfied with the design. The survey data showed that learners’ general perceptions of the MOOCs interface reached 4.44, perceived autonomy reached 4.40, perceived competence reached 4.52, and perceived relatedness reached 4.66 (5 points Likert scale). The in-depth interview data was open coded into three categories: 1) Advantages of the MOOCs interface, 2) Problems with the MOOCs interface, and 3) Suggestions for improvement. The advantages include providing choices for autonomy support, providing scaffolding and adaptive learning for competence support, providing interactive learning for relatedness support, and providing novel meanwhile helpful functions that existing platforms don’t have. The problems include lacking tutorials for novel functions, inconsistent icons and choice of words, and improper positioning and interaction. The suggestions for improvement include adding the wiki function, adding the reminder function, and visualizing the timetable. The significance of this research can be summarized as follows: 1) proposed an intrinsic motivation oriented MOOCs interface. 2) introduced three learning modes to the MOOCs learning environment. 3) introduced the learning group and learning team to the MOOCs environment to facilitate learners’ interaction. 4) provided an example of the dashboard for the context of MOOCs. And 5) provided insight into how to help learners achieve personalized learning in the MOOCs environment.정보통신기술(ICT)의 발달로 오늘날 많은 사람이 전통적인 교실에서뿐만 아니라 온라인으로도 교육을 받고 있다. 온라인 교육의 한 방법으로 MOOC(Massive Open Online Courses)의 시장은 2012년 첫 플랫폼이 공개된 이후 지속적으로 성장하고 있으며, 교육학자에게 많은 관심을 받고 있다. 특히 MOOC는 고등교육, 평생교육, 기업교육 등에서 대체할 수 없는 역할을 한다는 연구 결과가 속속히 나오고 있다. 교육 분야에서 MOOC가 점점 더 중요해질 것이라고 예상되지만, 지난 10년 동안 MOOC의 급속한 성장과 함께 MOOC의 문제점들이 일부 발견되었다. 그중 하나는 낮은 이수율로서, 기존 교육에 비해 MOOC 학습자는 평균 이수율이 약 10%에 머무를 정도로 중도 탈락할 가능성이 높은 것으로 보고되었다. 선행 연구에 따르면 이러한 현상을 일으키는 원인 중 하나는 부족한 동기부여 때문이다. 사용자 인터페이스는 애플리케이션에서 사용자와 직접 상호 작용하는 부분으로서 어포던스를 제공하고, 사용자가 애플리케이션을 사용하는 방식을 결정하기 때문에 매우 중요하다고 말할 수 있다. 특히, E-learning 상황에서는 인터페이스가 더욱 중요한데, 이는 동기 부여에 영향을 미칠 수 있는 동기 요소들이 사용자 인터페이스에 설계될 수 있기 때문이다. 그러나 선행 연구에 따르면 현재의 MOOC 인터페이스 디자인은 동기 요소가 부족하고 MOOC 학습자 간의 상호 작용을 촉진하지 못하고 있다. 이에 따라서 본 연구에서는 학습자의 동기부여에 초점을 맞추어 MOOC 인터페이스를 설계하였고, 이를 위해 다음의 연구 문제들을 고려하였다. 1) MOOC 학습자가 학습을 지속하도록 동기를 부여하는 인터페이스 디자인 가이드라인은 무엇인가? 2) 인터페이스 디자인 가이드라인에 따른 예시적인 인터페이스는 무엇인가? 3) 인터페이스에 대한 학습자의 반응은 무엇인가? 연구 질문에 답하기 위해 Richey와 Klein이 제안한 설계ㆍ개발 방법론 중 유형 1을 따랐다. 먼저 문헌검토를 통해 MOOC 인터페이스 디자인 가이드라인을 도출하고, 내적 타당성을 확보하기 위해 4명의 전문가가 2차례에 걸쳐 전문가검토를 실시하였다. 다음으로, 프로토타이핑 도구인 Figma를 사용하여 가이드라인에 따라 MOOC 인터페이스의 프로토타입을 개발했다. 마지막으로, 디자인 가이드라인의 외적 타당성을 확보하기 위해 2차례에 걸쳐 사용성 평가를 하고 학습자의 반응을 측정했다. 일련의 과제와 함께 프로토타입을 5명의 학습자에게 제공하고, 그 결과를 바탕으로 프로토타입과 가이드라인을 모두 수정했다. MOOC 인터페이스 디자인 가이드라인의 최종 버전은 3가지 동기 부여 디자인 원리 (자율성, 유능성 및 관계성), 12가지 동기 부여 디자인 가이드라인 (5개 자율성 지원, 4개 유능성 지원, 3개 관계성 지원)과 34개의 MOOC 인터페이스 디자인 가이드라인으로 구성되었다. 본 연구에서 개발한 MOOC 인터페이스는 이러한 MOOC 인터페이스 디자인 가이드라인에 따른 기능을 포함하고 있다. 자율성 지원 가이드라인을 반영하는 기능으로 학습 모드 선택 (self-paced, scheduled, premiere), 학습 그룹, 학습활동 선택, 학습목표 설정, 대시보드, 리마인더, 추천, 피드백 등 기능이 있었고, 유능성 지원 가이드라인을 반영하는 기능으로 레지스터, 수업 등록, 학습 경로, 팀 활동 지원, 대시보드, 학습목표 설정 등 기능이 있었으며, 관계성 지원 가이드라인을 반영하는 기능으로 대시보드, 피드백, 키워드 체크리스트, 학습 그룹, 채팅창, 그룹/팀 활동, 수업 평가, 팀 과제, 마인드맵, 노트 등 기능이 있었다. 사용성 평가에 참여한 학습자들은 개발한 MOOCs 인터페이스에 만족했으며, 5점 척도 구성한 설문 문항으로 조사한 결과, 인터페이스 전반에 대한 인식 4.44점, 자율성에 대한 인식 4.40점, 유능성에 대한 인식 4.52점, 관계성에 대한 인식 4.66점이었다. 그리고 심층 인터뷰에서 취득한 질적 데이터를 오픈 코딩을 통해서 정리한 결과, 개발한 MOOCs 인터페이스의 장점, 문제점, 개선점을 도출했다. 장점으로 자율성 지원을 위한 선택 제공, 유능성 지원을 위한 스캐폴딩과 적응형 학습, 관계성 지원을 위해서 제공하는 상호작용, 그리고 기존 플랫폼과의 의미있는 차이점 등이 있었다. 문제점으로 기존 플랫폼들이 제공하지 않은 새로운 기능에 대한 가이드가 필요하다는 점, 불일치한 아이콘과 용어, 부적절한 포지셔닝과 인터랙션 등이 있었다. 개선점으로 위키 기능, 알림 메시지 기능을 추가하고 시간표를 시각화 등이 있었다. 본 연구의 의의는 다음과 같이 요약될 수 있다. 1) 내재적 동기부여 지향한 MOOC 인터페이스를 제안하였다. 2) MOOC 학습 환경에 3가지 학습 모드를 도입했다. 3) 학습자의 상호 작용을 촉진하기 위해 학습 그룹과 학습 팀을 MOOC 학습 환경에 도입했다. 4) MOOC 상황에 대시보드의 예시를 제공했다. 그리고 5) 학습자가 MOOC 환경에서 맞춤형 학습을 달성하도록 돕는 방법에 대한 인사이트를 제공했다.Chapter 1. Introduction 1 1.1. Problem Statement 1 1.2. Purpose of the Research 4 1.3. Research Questions 8 1.4. Definition of Terms 9 1.4.1. MOOCs 9 1.4.2. Motivation 11 1.4.3. User Interface 12 Chapter 2. Literature Review 14 2.1. MOOCs 14 2.1.1. Characteristics and Meaning 14 2.1.2. History of MOOCs 16 2.1.3. MOOCs Platforms and MOOCs Learners 20 2.1.4. The Critiques and Drop Out Phenomenon 26 2.2. Motivation 35 2.2.1. Motivation in Learning 35 2.2.2. Motivation Theories 37 2.3.1. User Interface Design & Interface Design for Education 45 2.3.2 Motivation Supported User Interface Design 48 Chapter 3. Research Method 54 3.1. Design and Development Methodology 54 3.2. Research Participants 58 3.3. Research Tools 60 3.3.1. Internal Validation Tools 60 3.3.2. Prototyping Tool 60 3.3.3. External Validation Tools 62 3.4. Data Collection and Analysis 65 3.4.1. Expert Review 65 3.4.2. Learners' Responses 66 Chapter 4. Findings 68 4.1. The MOOCs Interface Design Guidelines 68 4.1.1. The Final Version of the MOOCs Interface Design Guidelines 69 4.1.2. The Results of the Expert Review 80 4.2. Prototype of the MOOCs Interface 85 4.3. Learners' Responses to the MOOCs Interface 118 4.3.1. Learners' Response to the MOOCs Interface (First Usability Test) 119 4.3.2. Learners' Responses to the Revised Interface (Second Usability Test) 143 Chapter 5. Discussion and Conclusion 145 5.1. Discussion 146 5.2. Conclusion 151 Reference 154 APPENDIX 1 175 APPENDIX 2 189 APPENDIX 3 197 APPENDIX 4 206 APPENDIX 5 213 APPENDIX 6 220 국문초록 224석

eduGraph: A Dashboard for Personalised Feedback in Massive Open Online Courses

Learning Analytics is concerned with the design and implementation of tools and processes for collecting, analysing, and communicating information about teaching and learning. It is enabled by data, but not driven by it, rather it tries to empower human judgements by presenting meaningful facts. This thesis explores the data generated in Open edX courses to understand how it can be analysed and used to impact learners' motivation in online courses. It is carried out using Design Science, a research methodology aiming to produce artefacts that can improve the interaction with problems. In this thesis I present the eduGraph dashboard, a dashboard that uses Learning Analytics to present meaningful insights about learners' learning process in Massive Open Online Courses (MOOCs). Results indicate that learners perceive the dashboard as useful and effective at motivating them to take part in online courses, and that it enables them to keep track of their progress in the courses. I posit that the biggest problem facing Learning Analytics today are the lack of accessible data, and that it is possible for reasearchers to create more accurate learner models by using Learning Anaytics theories and methods in combination with the iterative and technical process of Information Systems development.Masteroppgave i informasjonsvitenskapINFO390MASV-INF

Data Visualization in Online Educational Research

This chapter presents a general and practical guideline that is intended to introduce the traditional visualization methods (word clouds), and the advanced visualization methods including interactive visualization (heatmap matrix) and dynamic visualization (dashboard), which can be applied in quantitative, qualitative, and mixed-methods research. This chapter also presents the potentials of each visualization method for assisting researchers in choosing the most appropriate one in the web-based research study. Graduate students, educational researchers, and practitioners can contribute to take strengths from each visual analytical method to enhance the reach of significant research findings into the public sphere. By leveraging the novel visualization techniques used in the web-based research study, while staying true to the analytical methods of research design, graduate students, educational researchers, and practitioners will gain a broader understanding of big data and analytics for data use and representation in the field of education

Delving into instructor‐led feedback interventions informed by learning analytics in massive open online courses

Producción CientíficaBackground:Providing feedback in massive open online courses (MOOCs) is chal-lenging due to the massiveness and heterogeneity of learners' population. Learninganalytics (LA) solutions aim at scaling up feedback interventions and supportinginstructors in this endeavour.Paper Objectives:This paper focuses on instructor-led feedback mediated by LAtools in MOOCs. Our goal is to answer how, to what extent data-driven feedback isprovided to learners, and what its impact is.Methods:We conducted a systematic literature review on the state-of-the-art LA-informed instructor-led feedback in MOOCs. From a pool of 227 publications, weselected 38 articles that address the topic of LA-informed feedback in MOOCs medi-ated by instructors. We applied etic content analysis to the collected data.Results and Conclusions:The results revealed a lack of empirical studies exploring LA todeliver feedback, and limited attention on pedagogy to inform feedback practices. Our find-ings suggest the need for systematization and evaluation of feedback. Additionally, there isa need for conceptual tools to guide instructors' in the design of LA-based feedback.Takeaways:We point out the need for systematization and evaluation of feedback. Weenvision that this research can support the design of LA-based feedback, thus contribut-ing to bridge the gap between pedagogy and data-driven practice in MOOCs.Consejo de Investigación de Estonia (PSG286)Ministerio de Ciencia e Innovación - Fondo Europeo de Desarrollo Regional y la Agencia Nacional de Investigación (grant PID2020-112584RB-C32) and (grant TIN2017-85179-C3-2-R)Junta de Castilla y León - Fondo Social Europeo y el Consejo Regional de Educación (grant E-47-2018-0108488

Utilizing Online Activity Data to Improve Face-to-Face Collaborative Learning in Technology-Enhanced Learning Environments

학위논문 (박사)-- 서울대학교 대학원 : 융합과학기술대학원 융합과학부(디지털정보융합전공), 2019. 2. Rhee, Wonjong .We live in a flood of information and face more and more complex problems that are difficult to be solved by a single individual. Collaboration with others is necessary to solve these problems. In educational practice, this leads to more attention on collaborative learning. Collaborative learning is a problem-solving process where students learn and work together with other peers to accomplish shared tasks. Through this group-based learning, students can develop collaborative problem-solving skills and improve the core competencies such as communication skills. However, there are many issues for collaborative learning to succeed, especially in a face-to-face learning environment. For example, group formation, the first step to design successful collaborative learning, requires a lot of time and effort. In addition, it is difficult for a small number of instructors to manage a large number of student groups when trying to monitor and support their learning process. These issues can amount hindrance to the effectiveness of face-to-face collaborative learning. The purpose of this dissertation is to enhance the effectiveness of face-to-face collaborative learning with online activity data. First, online activity data is explored to find whether it can capture relevant student characteristics for group formation. If meaningful characteristics can be captured from the data, the entire group formation process can be performed more efficiently because the task can be automated. Second, learning analytics dashboards are implemented to provide adaptive support during a class. The dashboards system would monitor each group's collaboration status by utilizing online activity data that is collected during class in real-time, and provide adaptive feedback according to the status. Lastly, a predictive model is built to detect at-risk groups by utilizing the online activity data. The model is trained based on various features that represent important learning behaviors of a collaboration group. The results reveal that online activity data can be utilized to address some of the issues we have in face-to-face collaborative learning. Student characteristics captured from the online activity data determined important group characteristics that significantly influenced group achievement. This indicates that student groups can be formed efficiently by utilizing the online activity data. In addition, the adaptive support provided by learning analytics dashboards significantly improved group process as well as achievement. Because the data allowed the dashboards system to monitor current learning status, appropriate feedback could be provided accordingly. This led to an improvement of both learning process and outcome. Finally, the predictive model could detect at-risk groups with high accuracy during the class. The random forest algorithm revealed important learning behaviors of a collaboration group that instructors should pay more attention to. The findings indicate that the online activity data can be utilized to address practical issues of face-to-face collaborative learning and to improve the group-based learning where the data is available. Based on the investigation results, this dissertation makes contributions to learning analytics research and face-to-face collaborative learning in technology-enhanced learning environments. First, it can provide a concrete case study and a guide for future research that may take a learning analytics approach and utilize student activity data. Second, it adds a research endeavor to address challenges in face-to-face collaborative learning, which can lead to substantial enhancement of learning in educational practice. Third, it suggests interdisciplinary problem-solving approaches that can be applied to the real classroom context where online activity data is increasingly available with advanced technologies.Abstract i Chapter 1. Introduction １ 1.1. Motivation １ 1.2. Research questions ４ 1.3. Organization ６ Chapter 2. Background ８ 2.1. Learning analytics ８ 2.2. Collaborative learning ２２ 2.3. Technology-enhanced learning environment ２７ Chapter 3. Heterogeneous group formation with online activity data ３５ 3.1. Student characteristics for heterogeneous group formation ３６ 3.2. Method ４１ 3.3. Results ５１ 3.4. Discussion ５９ 3.5. Summary ６４ Chapter 4. Real-time dashboard for adaptive feedback in face-to-face CSCL ６７ 4.1. Theoretical background ７０ 4.2. Dashboard characteristics ８１ 4.3. Evaluation of the dashboard ９４ 4.4. Discussion １０７ 4.5. Summary １１４ Chapter 5. Real-time detection of at-risk groups in face-to-face CSCL １１８ 5.1. Important learning behaviors of group in collaborative argumentation １１８ 5.2. Method １２０ 5.3. Model performance and influential features １２５ 5.4. Discussion １２９ 5.5. Summary １３２ Chapter 6. Conclusion １３４ Bibliography １４０Docto

The Use of Learning Analytics Interactive Dashboards in Serious Games: A Review of the Literature

The learning analytics in serious games, corresponds to a subject in increasing demand in the educational field. In this context, there is a need to study how data visualizations found in the literature are adopted in learning analytics in serious games. This paper presents a Systematic Literature Review (SLR) on how the evolution of studies associated with the use of learning analytics interactive dashboards in serious games is processed, seeking to investigate the characteristics of using dashboards for viewing educational data. A bibliometric analysis was carried out in which 75 relevant studies were selected from the Scopus, Web of Science, and IEEExplore databases. From the data analysis, it was observed that in the current literature there is a reduced number of studies containing the main actors in the learning process, as follows: teachers/instructors, students/participants, game developers/designers, and managers/researchers. In the vast majority of investigated studies, data visualization algorithms are used, where the main focus takes into account only actors, such as teachers/instructors and students/participants

Sistema informática de apoyo a las analíticas para el aprendizaje (learning analytics) para entornos educativos on-line

Los cursos MOOC (Massive Open Online Courses) son una herramienta de aprendizaje en constante crecimiento en oferta (número de cursos ofertados), demanda (número de estudiantes matriculados) y relevancia en la disciplina del aprendizaje en línea. En este contexto, se han detectado potenciales factores relacionados con la insuficiente interacción estudiante-profesor y el aislamiento que sienten los estudiantes que pueden afectarles negativamente y que precisan de un amplio estudio para ser analizados, comprendidos y, si se requiere, mitigados o solucionados. Si este objetivo se cumple, se habrá conseguido dar un gran paso para hacer del aprendizaje con este tipo de recursos un proceso eficiente y útil para cualquier estudiante que desee utilizarlos. Además, los MOOC recogen gran cantidad de información en relación con la interactividad del estudiante con sus recursos, con lo que son una gran fuente de datos en este campo. La disciplina que facilita poder afrontar el problema planteado es la Analítica del Aprendizaje o Learning Analytics. El presente Trabajo de Fin de Grado busca avanzar dentro de la comprensión de este tipo de sensaciones. Para ello, se propone el desarrollo, implementación y explotación de una plataforma escalable (edX-LIMS: Learning Intervention Monitoring Service for edX MOOCs) que permita realizar un proceso de acompañamiento de los estudiantes de un MOOC. Dicha plataforma proporciona periódicamente a los estudiantes de un MOOC información visual en un Dashboard o Panel de aprendizaje en la Web, mostrándoles su progreso y participación en el MOOC. Esta información proporcionada es parte de una estrategia de intervención sobre el aprendizaje de estos estudiantes. El sistema ofrece también a los instructores de MOOC acceso a un Dashboard o Panel de Instructores en la Web que muestra el interés en este servicio por parte de los estudiantes y, por lo tanto, facilita la evaluación del éxito o el fracaso de la estrategia de intervención

A Systematic Review of Teacher-Facing Dashboards for Collaborative Learning Activities and Tools in Online Higher Education

Dashboard for online higher education support monitoring and evaluation of students’ interactions, but mostly limited to interaction occurring within learning management systems. In this study, we sought to find which collaborative learning activities and tools in online higher education are included in teaching dashboards. By following Kitchenham’s procedure for systematic reviews, 36 papers were identified according to this focus and analysed. The results identify dashboards supporting collaborative tools, both synchronous and asynchronous, along categories such as learning management systems, communication tools, social media, computer programming code management platforms, project management platforms, and collaborative writing tools. Dashboard support was also found for collaborative activities, grouped under four categories of forum discussion activities, three categories of communication activities and four categories of collaborative editing/sharing activities, though most of the analysed dashboards only provide support for no more than two or three collaborative tools. This represents a need for further research on how to develop dashboards that combine data from a more diverse set of collaborative activities and tools.This work was supported by the TRIO project funded by the European Union’s Erasmus+ KA220-ADU – Cooperation partnerships in adult education programme under grant agreement no. KA220-ADU-1B9975F8.info:eu-repo/semantics/publishedVersio