Exploring engagement profiling in MOOCs through Learning Analytics: The Open edX Case

The enormous amount of data being generated daily, requires effective and efficient ways of processing and analysing in order to extract useful information and form meaningful conclusions. Learning Analytics is a set of methodologies and practices that uncover such information from educational data. The research in this thesis explores the addition of a Learning Analytics feature to the context of a Learning Analytics tool that aids instructors using the online Massive Open Online Course (MOOC) platform, Open edX. This is done through the development and evaluation of a working artefact that supports profiling of students according to their activity throughout the course, alongside the visualizations, which represent said activity. As a result, the thoroughly demonstrated process of the artefact creation and feedback collection from the instructors shows the potential of Learning Analytics methods when applied to Open edX tracking data. Several practical features for creating different engagement groups, together with the visualizations, are conceptualized, implemented and evaluated, and are positively assessed by the target group of instructors. In addition, the challenges that were encountered in the period of the development, are presented, together with the suggestions to overcome them. Finally, a few extra features are outlined for future work, which could expand the existing functionality even more and bring additional knowledge to this research area.Master's Thesis in Information ScienceINFO390MASV-INF

Evaluation of a Learning Analytics Application for Open edX Platform

Massive open online courses (MOOCs) have recently emerged as a revolution in education. Due to the huge amount of users, it is difficult for teachers to provide personalized instruction. Learning analytics computer applications have emerged as a solution. At present, MOOC platforms provide low support for learning analytics visualizations, and a challenge is to provide useful and effective visualization applications about the learning process. At this paper we review the learning analytics functionality of Open edX and make an overview of our learning analytics application ANALYSE. We present a usability and effectiveness evaluation of ANALYSE tool with 40 students taking a Design of Telematics Applications course. The survey obtained very positive results in a system usability scale (SUS) questionnaire (78.44/100) in terms of the usefulness of visualizations (3.68/5) and the effectiveness ratio (92/100) of the actions required for the respondents. Therefore, we can conclude that the implemented learning analytics application is usable and effective.Acknowledgements: This work has been supported by the "eMadrid" project (Regional Government of Madrid) under grant S2013/ICE-2715, the "RESET" project (Ministry of Economy and Competiveness) under grant RESET TIN2014-53199-C3-1-R and the European Erasmus+ SHEILA project under grant 562080-EPP-1-2015-BE-EPPKA3-PI-FORWARD

Learning analytics visualizations of student-activity time distribution for the open Edx platform

MOOCs are one of the current trending topics in educational technology. They surged with the vision of a democratization in education worldwide by removing some access barriers. As every technology, MOOCs have promoters and detractors but truth is, they are an invaluable source of data related to student interaction with courses and their resources as has been available never before. This data is susceptible to shed light on the learning process in this online environment and potentially in uence in a positive way the learning outcomes. Students can be presented with visual, friendly information that enable them to re ect on their performance and gain awareness of their own learning style based on data beyond intuition. Teachers can be given the same metrics augmented with student aggregates for their courses. Thus, they can tune their pedagogical approach and resource quality for the better. In this context, Open edX is one of the most prominent MOOC platforms. However, its learning analytics support is low at present. This project extends the learning analytics support of the Open edX platform by adding new six visualizations related to time on video and problem modules, namely: 1) video time watched, 2) video and 3) problem time distributions, 4) video repetition pro le, 5) daily time on video and problem and 6) distribution of video events. The main technologies used have been Python, Django, MySQL, JavaScript, Google Charts and MongoDBLos MOOCs están de moda en lo que se refiere a tecnología educativa. Surgieron con la visión de remover algunas barreras de acceso en aras de la democratización de la educación en cada rincón del mundo. Como toda tecnología, tienen sus promotores y detractores, pero lo cierto es que constituyen una valiosa fuente de datos como no ha habido antes en lo que respecta a la interacción de los estudiantes con estos cursos y sus recursos. Estos datos pueden ayudarnos a entender el proceso de aprendizaje en estos entornos. Tienen además el potencial de in uir positivamente en los resultados del aprendizaje. Se puede presentar a los estudiantes una información visual fácil de entender, que les permita re exionar sobre su rendimiento y ganar conciencia de su estilo de aprendizaje a partir de los datos, más allá de lo que les pueda indicar la intuición. Las mismas métricas se pueden poner a disponibilidad de los profesores, en conjunto con valores agregados de la clase. De esta manera, los profesores pueden ajustar el enfoque pedagógico del curso y mejorar la calidad de los recursos. En este contexto, Open edX es una de las plataformas proveedoras de MOOCs más prominentes. Sin embargo, tiene todavía poco soporte para analitica del aprendizaje. Este proyecto extiende ese soporte al incorporar seis visualizaciones nuevas sobre tiempo en vídeos y problemas, especícamente: 1) tiempo visto de vídeos, distribución de tiempo en 2) vídeos y 3) problemas, 4) peril de repetición de vídeo, 5) tiempo diario en vídeos y problemas y 6) distribuci on de eventos de vídeo. Las principales tecnologías usadas son: Python, Django, MySQL, JavaScript, Google Charts y MongoDB.Ingeniería de Telecomunicació

Data Visualization in Online Educational Research

This chapter presents a general and practical guideline that is intended to introduce the traditional visualization methods (word clouds), and the advanced visualization methods including interactive visualization (heatmap matrix) and dynamic visualization (dashboard), which can be applied in quantitative, qualitative, and mixed-methods research. This chapter also presents the potentials of each visualization method for assisting researchers in choosing the most appropriate one in the web-based research study. Graduate students, educational researchers, and practitioners can contribute to take strengths from each visual analytical method to enhance the reach of significant research findings into the public sphere. By leveraging the novel visualization techniques used in the web-based research study, while staying true to the analytical methods of research design, graduate students, educational researchers, and practitioners will gain a broader understanding of big data and analytics for data use and representation in the field of education

Analyzing the behavior of students regarding learning activities, badges, and academic dishonesty in MOOC environment

Mención Internacional en el título de doctorThe ‘big data’ scene has brought new improvement opportunities to most products and services, including education. Web-based learning has become very widespread over the last decade, which in conjunction with the Massive Open Online Course (MOOC) phenomenon, it has enabled the collection of large and rich data samples regarding the interaction of students with these educational online environments. We have detected different areas in the literature that still need improvement and more research studies. Particularly, in the context of MOOCs and Small Private Online Courses (SPOCs), where we focus our data analysis on the platforms Khan Academy, Open edX and Coursera. More specifically, we are going to work towards learning analytics visualization dashboards, carrying out an evaluation of these visual analytics tools. Additionally, we will delve into the activity and behavior of students with regular and optional activities, badges and their online academically dishonest conduct. The analysis of activity and behavior of students is divided first in exploratory analysis providing descriptive and inferential statistics, like correlations and group comparisons, as well as numerous visualizations that facilitate conveying understandable information. Second, we apply clustering analysis to find different profiles of students for different purposes e.g., to analyze potential adaptation of learning experiences and pedagogical implications. Third, we also provide three machine learning models, two of them to predict learning outcomes (learning gains and certificate accomplishment) and one to classify submissions as illicit or not. We also use these models to discuss about the importance of variables. Finally, we discuss our results in terms of the motivation of students, student profiling, instructional design, potential actuators and the evaluation of visual analytics dashboards providing different recommendations to improve future educational experiments.Las novedades en torno al ‘big data’ han traído nuevas oportunidades de mejorar la mayoría de productos y servicios, incluyendo la educación. El aprendizaje mediante tecnologías web se ha extendido mucho durante la última década, que conjuntamente con el fenómeno de los cursos abiertos masivos en línea (MOOCs), ha permitido que se recojan grandes y ricas muestras de datos sobre la interacción de los estudiantes con estos entornos virtuales de aprendizaje. Nosotros hemos detectado diferentes áreas en la literatura que aún necesitan de mejoras y del desarrollo de más estudios, específicamente en el contexto de MOOCs y cursos privados pequeños en línea (SPOCs). En la tesis nos hemos enfocado en el análisis de datos en las plataformas Khan Academy, Open edX y Coursera. Más específicamente, vamos a trabajar en interfaces de visualizaciones de analítica de aprendizaje, llevando a cabo la evaluación de estas herramientas de analítica visual. Además, profundizaremos en la actividad y el comportamiento de los estudiantes con actividades comunes y opcionales, medallas y sus conductas en torno a la deshonestidad académica. Este análisis de actividad y comportamiento comienza primero con análisis exploratorio proporcionando variables descriptivas y de inferencia estadística, como correlaciones y comparaciones entre grupos, así como numerosas visualizaciones que facilitan la transmisión de información inteligible. En segundo lugar aplicaremos técnicas de agrupamiento para encontrar distintos perfiles de estudiantes con diferentes propósitos, como por ejemplo para analizar posibles adaptaciones de experiencias educativas y sus implicaciones pedagógicas. También proporcionamos tres modelos de aprendizaje máquina, dos de ellos que predicen resultados finales de aprendizaje (ganancias de aprendizaje y la consecución de certificados de terminación) y uno para clasificar que ejercicios han sido entregados de forma deshonesta. También usaremos estos tres modelos para analizar la importancia de las variables. Finalmente, discutimos todos los resultados en términos de la motivación de los estudiantes, diferentes perfiles de estudiante, diseño instruccional, posibles sistemas actuadores, así como la evaluación de interfaces de analítica visual, proporcionando recomendaciones que pueden ayudar a mejorar futuras experiencias educacionales.Programa Oficial de Doctorado en Ingeniería TelemáticaPresidente: Davinia Hernández Leo.- Secretario: Luis Sánchez Fernández.- Vocal: Adolfo Ruiz Callej

An algorithm and a tool for the automatic grading of MOOC learners from their contributions in the discussion forum

MOOCs (massive open online courses) have a built-in forum where learners can share experiences as well as ask questions and get answers. Nevertheless, the work of the learners in the MOOC forum is usually not taken into account when calculating their grade in the course, due to the difficulty of automating the calculation of that grade in a context with a very large number of learners. In some situations, discussion forums might even be the only available evidence to grade learners. In other situations, forum interactions could serve as a complement for calculating the grade in addition to traditional summative assessment activities. This paper proposes an algorithm to automatically calculate learners' grades in the MOOC forum, considering both the quantitative dimension and the relevance in their contributions. In addition, the algorithm has been implemented within a web application, providing instructors with a visual and a numerical representation of the grade for each learner. An exploratory analysis is carried out to assess the algorithm and the tool with a MOOC on programming, obtaining a moderate positive correlation between the forum grades provided by the algorithm and the grades obtained through the summative assessment activities. Nevertheless, the complementary analysis conducted indicates that this correlation may not be enough to use the forum grades as predictors of the grades obtained through summative assessment activities.This work was supported in part by the FEDER/Ministerio de Ciencia, Innovación y Universidades;Agencia Estatal de Investigación, through the Smartlet Project under Grant TIN2017-85179-C3-1-R, and in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS-4307, a project which is co-funded by the European Structural Funds (FSE and FEDER). Partial support has also been received from the European Commission through Erasmus+ Capacity Building in the Field of Higher Education projects, more specifically through projects LALA (586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP), InnovaT (598758-EPP-1-2018-1-AT-EPPKA2-CBHE-JP), and PROF-XXI (609767-EPP-1-2019-1-ES-EPPKA2-CBHE-JP)