A course agnostic approach to predicting student success from VLE log data using recurrent neural networks

We describe a method of improving the accuracy of a learning analytics system through the application of a Recurrent Neural Network over all students in a University, regardless of course. Our target is to discover how well a student will do in a class given their interaction with a virtual learning environment. We show how this method performs well when we want to predict how well students will do, even if we do not have a model trained based on their specific course

An Investigation Into Machine Learning Solutions Involving Time Series Across Different Problem Domains

In this thesis we will examine architectures and models for machine learning in three problem domains each of which are based around the use of time series data in time series applications. We set out to examine whether the architecture and model solutions in different problem domains will converge when optimised towards a similar solution or not. Stated clearly, our central research question is “That problem-solving in diverse problem domains using Machine Learning applied to time series data requires diverse models in order to achieve the best performance” . To investigate this research hypothesis we use a case study methodology. We will investigate three separate and diverse problem domains, and compare their results and best solutions. The first problem domain is in the field of educational analytics, the second is in the field of agri-analytics and the third is in the field of environmental science

Accurate, timely and portable: course-agnostic early prediction of student performance from LMS logs

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Data ScienceLearning management systems are essential intermediaries between students and educational content in the digital era. Among other factors, the institutional adoption of such systems is meant to foster student engagement and lead to better educational outcomes in a scalable manner. However, a significant challenge facing educators and institutions is the timely identification of students who may require special attention and feedback. Early identification of students allows educators to provide necessary feedback and adopt suitable corrective measures. Therefore, a significant body of research has been dedicated to developing early warning systems with clickstream data. However, comprehensive studies that attempt prediction on multiple courses are few and far between. Moreover, most predictive models require sophisticated domain knowledge, data skills and computational power that may not be available in practice. In this work, we used an academic year’s worth of data collected from all courses at a Portuguese information management school to perform two main experiments on two binary classification problems: the first being students at risk vs students not at risk and the second being high-performing students vs not high-performing students. In the first experiment, we compared the performances obtained with traditional machine learning classifiers against majority class classifiers at multiple stages of course completion (more specifically, the 10%, 25%, 33%, 50% and 100% course completion thresholds). For both classification problems, performances on all metrics peaked when using all of the data collected throughout the course – 88.6% accuracy and 92.3% Area Under the Receiver Operating Characteristic (AUROC) using Random Forest (RF) for students at risk and 78.2% accuracy and 79.6% AUROC using ExtraTrees for high-performing students. Concerning early prediction, acceptable performances for classifying at-risk students are achieved as early as the 25% course duration threshold (72.8% AUROC using RF). Performances for high-performing students were generally lower, with AUROC at earlier stages peaking at the courses’ midway point (64.4% AUROC using RF). Our second experiment deployed long-short term memory units (LSTM) trained with a time-dependent representation of a single feature (number of total clicks). While this approach achieved inferior performances, we argue that the more straightforward data pre-processing of this approach may represent a worthwhile tradeoff against relatively small losses in model performance, especially at earlier moments of prediction. We found the best tradeoff at 33% course duration – 64% AUROC against 74% AUROC using RF to predict at-risk students. To predict high-performing students, we found the best tradeoff to occur at 25% course duration (56% AUROC against 61% using RF). Results obtained using a different set of logs validate the portability of our approach when it comes to static aggregate models. However, our deep learning approach did not generalize well on this data, which suggests that portability between courses using this approach may only be possible in specific instances

Predicting Academic Performance of Students from VLE Big Data using Deep Learning Models

The abundance of accessible educational data, supported by the technology-enhanced learning platforms, provides opportunities to mine learning behavior of students, addressing their issues, optimizing the educational environment, and enabling data-driven decision making. Virtual learning environments complement the learning analytics paradigm by effectively providing datasets for analysing and reporting the learning process of students and its reflection and contribution in their respective performances. This study deploys a deep artificial neural network on a set of unique handcrafted features, extracted from the virtual learning environments clickstream data, to predict at-risk students providing measures for early intervention of such cases. The results show the proposed model to achieve a classification accuracy of 84%-93%. We show that a deep artificial neural network outperforms the baseline logistic regression and support vector machine models. While logistic regression achieves an accuracy of 79.82% - 85.60%, the support vector machine achieves 79.95% - 89.14%. Aligned with the existing studies - our findings demonstrate the inclusion of legacy data and assessment-related data to impact the model significantly. Students interested in accessing the content of the previous lectures are observed to demonstrate better performance. The study intends to assist institutes in formulating a necessary framework for pedagogical support, facilitating higher education decision-making process towards sustainable education

Novel Datasets, User Interfaces and Learner Models to Improve Learner Engagement Prediction on Educational Videos

With the emergence of Open Education Resources (OERs), educational content creation has rapidly scaled up, making a large collection of new materials made available. Among these, we find educational videos, the most popular modality for transferring knowledge in the technology-enhanced learning paradigm. Rapid creation of learning resources opens up opportunities in facilitating sustainable education, as the potential to personalise and recommend specific materials that align with individual users’ interests, goals, knowledge level, language and stylistic preferences increases. However, the quality and topical coverage of these materials could vary significantly, posing significant challenges in managing this large collection, including the risk of negative user experience and engagement with these materials. The scarcity of support resources such as public datasets is another challenge that slows down the development of tools in this research area. This thesis develops a set of novel tools that improve the recommendation of educational videos. Two novel datasets and an e-learning platform with a novel user interface are developed to support the offline and online testing of recommendation models for educational videos. Furthermore, a set of learner models that accounts for the learner interests, knowledge, novelty and popularity of content is developed through this thesis. The different models are integrated together to propose a novel learner model that accounts for the different factors simultaneously. The user studies conducted on the novel user interface show that the new interface encourages users to explore the topical content more rigorously before making relevance judgements about educational videos. Offline experiments on the newly constructed datasets show that the newly proposed learner models outperform their relevant baselines significantly

Latent deep sequential learning of behavioural sequences

The growing use of asynchronous online education (MOOCs and e-courses) in recent years has resulted in increased economic and scientific productivity, which has worsened during the coronavirus epidemic. The widespread usage of OLEs has increased enrolment, including previously excluded students, resulting in a far higher dropout rate than in conventional classrooms. Dropouts are a significant problem, especially considering the rising proliferation of online courses, from individual MOOCs to whole academic programmes due to the pandemic. Increased efficiency in dropout prevention techniques is vital for institutions, students, and faculty members and must be prioritised. In response to the resurgence of interest in the student dropout prediction (SDP) issue, there has been a significant rise in contributions to the literature on this topic. An in-depth review of the current state of the art literature on SDP is provided, with a special emphasis on Machine Learning prediction approaches; however, this is not the only focus of the thesis. We propose a complete hierarchical categorisation of the current literature that correlates to the process of design decisions in the SDP, and we demonstrate how it may be implemented. In order to enable comparative analysis, we develop a formal notation for universally defining the multiple dropout models examined by scholars in the area, including online degrees and their attributes. We look at several other important factors that have received less attention in the literature, such as evaluation metrics, acquired data, and privacy concerns. We emphasise deep sequential machine learning approaches and are considered to be one of the most successful solutions available in this field of study. Most importantly, we present a novel technique - namely GRU-AE - for tackling the SDP problem using hidden spatial information and time-related data from student trajectories. Our method is capable of dealing with data imbalances and time-series sparsity challenges. The proposed technique outperforms current methods in various situations, including the complex scenario of full-length courses (such as online degrees). This situation was thought to be less common before the outbreak, but it is now deemed important. Finally, we extend our findings to different contexts with a similar characterisation (temporal sequences of behavioural labels). Specifically, we show that our technique can be used in real-world circumstances where the unbalanced nature of the data can be mitigated by using class balancement technique (i.e. ADASYN), e.g., survival prediction in critical care telehealth systems where balancement technique alleviates the problem of inter-activity reliance and sparsity, resulting in an overall improvement in performance

Big data analytics and organisational change. The case of learning analytics

Much of the Information Systems (IS) literature on Big Data Analytics (BDA) assumes a straightforward relationship between human activity and data, and between data and analytical insights that can be used to steer operations (e.g. Chen, Preston and Swink, 2015; Brynjolfsson, Geva and Reichman, 2016; Yahav, Shmueli and Mani, 2016). On the other hand, researchers also try to understand the role of big data within organisations, the contributions of analytics to strategy and decision-making, and the value of big data and its organisational consequences (Constantiou and Kallinikos, 2015; Abbasi, Sarker and Chiang, 2016; Günther et al., 2017). At the same time, more critical scholars have suggested that the implications of BDA can go beyond decision-making, sometimes twisting or even undermining managerial efforts (Newell and Marabelli, 2015; Galliers et al., 2017; Markus, 2017). This research investigates how BDA systems change organisations that implement them and aims to uncover the resulting organisational transformations. In line with the Transformational Model of Social Activity (Archer and Bhaskar, 1998; Faulkner and Runde, 2013), it is argued that BDA systems as technological objects change how work is done, and these changes lead to the reproduction or transformation of organisations as social structures. In order to uncover this reproduction or transformation, the concepts of encoding, aggregation and correlation (Alaimo and Kallinikos, 2017) are deployed to analyse how data is produced, and the theory of reactivity (Espeland and Sauder, 2007), originally developed to study university rankings, is adapted to trace the mechanisms and effects of organisational transformation in a case study. The study provides an answer to the question of how organisations are transformed, in unintended ways, through the implementation of BDA systems. The concept of the analytical cage is proposed as a new form of organising emerging from BDA within organisations

Low-Resource Unsupervised NMT:Diagnosing the Problem and Providing a Linguistically Motivated Solution

Unsupervised Machine Translation hasbeen advancing our ability to translatewithout parallel data, but state-of-the-artmethods assume an abundance of mono-lingual data. This paper investigates thescenario where monolingual data is lim-ited as well, finding that current unsuper-vised methods suffer in performance un-der this stricter setting. We find that theperformance loss originates from the poorquality of the pretrained monolingual em-beddings, and we propose using linguis-tic information in the embedding train-ing scheme. To support this, we look attwo linguistic features that may help im-prove alignment quality: dependency in-formation and sub-word information. Us-ing dependency-based embeddings resultsin a complementary word representationwhich offers a boost in performance ofaround 1.5 BLEU points compared to stan-dardWORD2VECwhen monolingual datais limited to 1 million sentences per lan-guage. We also find that the inclusion ofsub-word information is crucial to improv-ing the quality of the embedding