Beyond A/B Testing: Sequential Randomization for Developing Interventions in Scaled Digital Learning Environments

Randomized experiments ensure robust causal inference that are critical to effective learning analytics research and practice. However, traditional randomized experiments, like A/B tests, are limiting in large scale digital learning environments. While traditional experiments can accurately compare two treatment options, they are less able to inform how to adapt interventions to continually meet learners' diverse needs. In this work, we introduce a trial design for developing adaptive interventions in scaled digital learning environments -- the sequential randomized trial (SRT). With the goal of improving learner experience and developing interventions that benefit all learners at all times, SRTs inform how to sequence, time, and personalize interventions. In this paper, we provide an overview of SRTs, and we illustrate the advantages they hold compared to traditional experiments. We describe a novel SRT run in a large scale data science MOOC. The trial results contextualize how learner engagement can be addressed through inclusive culturally targeted reminder emails. We also provide practical advice for researchers who aim to run their own SRTs to develop adaptive interventions in scaled digital learning environments

A literature synthesis of personalised technology-enhanced learning: what works and why

Personalised learning, having seen both surges and declines in popularity over the past few decades, is once again enjoying a resurgence. Examples include digital resources tailored to a particular learner’s needs, or individual feedback on a student’s assessed work. In addition, personalised technology-enhanced learning (TEL) now seems to be attracting interest from philanthropists and venture capitalists indicating a new level of enthusiasm for the area and a potential growth industry. However, these industries may be driven by profit rather than pedagogy, and hence it is vital these new developments are informed by relevant, evidence-based research. For many people, personalised learning is an ambiguous and even loaded term that promises much but does not always deliver. This paper provides an in-depth and critical review and synthesis of how personalisation has been represented in the literature since 2000, with a particular focus on TEL. We examine the reasons why personalised learning can be beneficial and examine how TEL can contribute to this. We also unpack how personalisation can contribute to more effective learning. Lastly, we examine the limitations of personalised learning and discuss the potential impacts on wider stakeholders

Devices, Information, and People: Abstracting the Internet of Things for End-User Personalization

Nowadays, end users can take advantage of end-user development platforms to personalize the Internet of Things. These platforms typically adopt a vendor-centric abstraction, by letting users to customize each of their smart device and/or online service through different trigger-action rules. Despite the popularity of such an approach, several research challenges in this domain are still underexplored. Which "things" would users personalize, and in which contexts? Are there any other effective abstractions besides the vendor-centric one? Would users adopt different abstractions in different contexts? To answer these questions, we report on the results of a 1-week-long diary study during which 24 participants noted down trigger-action rules arising during their daily activities. Results show that users would adopt multiple abstractions by personalizing devices, information, and people-related behaviors where the individual is at the center of the interaction. We found, in particular, that the adopted abstraction may depend on different factors, ranging from the user profile to the context in which the personalization is introduced. While users are inclined to personalize physical objects in the home, for example, they often go "beyond devices" in the city, where they are more interested in the underlying information. Our findings identify new design opportunities in HCI to improve the relationship between the Internet of Things, personalization paradigms, and users

Enhanced information retrieval using domain-specific recommender models

The objective of an information retrieval (IR) system is to retrieve relevant items which meet a user information need. There is currently significant interest in personalized IR which seeks to improve IR effectiveness by incorporating a model of the user’s interests. However, in some situations there may be no opportunity to learn about the interests of a specific user on a certain topic. In our work, we propose an IR approach which combines a recommender algorithm with IR methods to improve retrieval for domains where the system has no opportunity to learn prior information about the user’s knowledge of a domain for which they have not previously entered a query. We use search data from other previous users interested in the same topic to build a recommender model for this topic. When a user enters a query on a topic, new to this user, an appropriate recommender model is selected and used to predict a ranking which the user may find interesting based on the behaviour of previous users with similar queries. The recommender output is integrated with a standard IR method in a weighted linear combination to provide a final result for the user. Experiments using the INEX 2009 data collection with a simulated recommender training set show that our approach can improve on a baseline IR system

Wearable and mobile devices

Information and Communication Technologies, known as ICT, have undergone dramatic changes in the last 25 years. The 1980s was the decade of the Personal Computer (PC), which brought computing into the home and, in an educational setting, into the classroom. The 1990s gave us the World Wide Web (the Web), building on the infrastructure of the Internet, which has revolutionized the availability and delivery of information. In the midst of this information revolution, we are now confronted with a third wave of novel technologies (i.e., mobile and wearable computing), where computing devices already are becoming small enough so that we can carry them around at all times, and, in addition, they have the ability to interact with devices embedded in the environment. The development of wearable technology is perhaps a logical product of the convergence between the miniaturization of microchips (nanotechnology) and an increasing interest in pervasive computing, where mobility is the main objective. The miniaturization of computers is largely due to the decreasing size of semiconductors and switches; molecular manufacturing will allow for “not only molecular-scale switches but also nanoscale motors, pumps, pipes, machinery that could mimic skin” (Page, 2003, p. 2). This shift in the size of computers has obvious implications for the human-computer interaction introducing the next generation of interfaces. Neil Gershenfeld, the director of the Media Lab’s Physics and Media Group, argues, “The world is becoming the interface. Computers as distinguishable devices will disappear as the objects themselves become the means we use to interact with both the physical and the virtual worlds” (Page, 2003, p. 3). Ultimately, this will lead to a move away from desktop user interfaces and toward mobile interfaces and pervasive computing

The simplicity project: easing the burden of using complex and heterogeneous ICT devices and services

As of today, to exploit the variety of different "services", users need to configure each of their devices by using different procedures and need to explicitly select among heterogeneous access technologies and protocols. In addition to that, users are authenticated and charged by different means. The lack of implicit human computer interaction, context-awareness and standardisation places an enormous burden of complexity on the shoulders of the final users. The IST-Simplicity project aims at leveraging such problems by: i) automatically creating and customizing a user communication space; ii) adapting services to user terminal characteristics and to users preferences; iii) orchestrating network capabilities. The aim of this paper is to present the technical framework of the IST-Simplicity project. This paper is a thorough analysis and qualitative evaluation of the different technologies, standards and works presented in the literature related to the Simplicity system to be developed

ALBERT-Based Personalized Educational Recommender System: Enhancing Students’ Learning Outcomes in Online Learning

Online learners must navigate vast educational resources to find materials that meet their needs. This study introduces an ALBERT-based personalized educational recommender system to improve student learning. ALBERT (A Lite BERT), an optimized variant of the BERT algorithm, captures contextualized word representations and understands the semantic meaning of learning resources, student profiles, and interactions. This study evaluates the ALBERT-based recommender system’s personalized learning recommendations. To assess learning outcomes, a diverse group of students from different educational domains is evaluated. Before and after the recommender system, academic performance, knowledge retention, and engagement are assessed. User satisfaction surveys assess recommendation quality, relevance, and user experience. The recommender system uses ALBERT’s model optimization to improve recommendation accuracy, learner engagement, and personalized learning. The evaluation shows the ALBERT-based personalized recommender system improves online learning outcomes. System-generated recommendations boost student engagement, knowledge retention, and academic performance. User satisfaction surveys show that the ALBERT-based system meets learners’ needs by providing relevant and high-quality recommendations. This research shows how advanced deep learning algorithms like ALBERT can improve personalized online learning. ALBERT’s optimized training and inference speeds up the recommender system’s scalability. This empowers learners to access tailored and high-quality educational resources, maximizing their learning outcomes and potential in online learning