Beyond A/B Testing: Sequential Randomization for Developing Interventions in Scaled Digital Learning Environments

Randomized experiments ensure robust causal inference that are critical to effective learning analytics research and practice. However, traditional randomized experiments, like A/B tests, are limiting in large scale digital learning environments. While traditional experiments can accurately compare two treatment options, they are less able to inform how to adapt interventions to continually meet learners' diverse needs. In this work, we introduce a trial design for developing adaptive interventions in scaled digital learning environments -- the sequential randomized trial (SRT). With the goal of improving learner experience and developing interventions that benefit all learners at all times, SRTs inform how to sequence, time, and personalize interventions. In this paper, we provide an overview of SRTs, and we illustrate the advantages they hold compared to traditional experiments. We describe a novel SRT run in a large scale data science MOOC. The trial results contextualize how learner engagement can be addressed through inclusive culturally targeted reminder emails. We also provide practical advice for researchers who aim to run their own SRTs to develop adaptive interventions in scaled digital learning environments

Portinari: A Data Exploration Tool to Personalize Cervical Cancer Screening

Socio-technical systems play an important role in public health screening programs to prevent cancer. Cervical cancer incidence has significantly decreased in countries that developed systems for organized screening engaging medical practitioners, laboratories and patients. The system automatically identifies individuals at risk of developing the disease and invites them for a screening exam or a follow-up exam conducted by medical professionals. A triage algorithm in the system aims to reduce unnecessary screening exams for individuals at low-risk while detecting and treating individuals at high-risk. Despite the general success of screening, the triage algorithm is a one-size-fits all approach that is not personalized to a patient. This can easily be observed in historical data from screening exams. Often patients rely on personal factors to determine that they are either at high risk or not at risk at all and take action at their own discretion. Can exploring patient trajectories help hypothesize personal factors leading to their decisions? We present Portinari, a data exploration tool to query and visualize future trajectories of patients who have undergone a specific sequence of screening exams. The web-based tool contains (a) a visual query interface (b) a backend graph database of events in patients' lives (c) trajectory visualization using sankey diagrams. We use Portinari to explore diverse trajectories of patients following the Norwegian triage algorithm. The trajectories demonstrated variable degrees of adherence to the triage algorithm and allowed epidemiologists to hypothesize about the possible causes.Comment: Conference paper published at ICSE 2017 Buenos Aires, at the Software Engineering in Society Track. 10 pages, 5 figure

From Personalization to Adaptivity: Creating Immersive Visits through Interactive Digital Storytelling at the Acropolis Museum

Storytelling has recently become a popular way to guide museum visitors, replacing traditional exhibit-centric descriptions by story-centric cohesive narrations with references to the exhibits and multimedia content. This work presents the fundamental elements of the CHESS project approach, the goal of which is to provide adaptive, personalized, interactive storytelling for museum visits. We shortly present the CHESS project and its background, we detail the proposed storytelling and user models, we describe the provided functionality and we outline the main tools and mechanisms employed. Finally, we present the preliminary results of a recent evaluation study that are informing several directions for future work

Theory-based user modeling for personalized interactive information retrieval

In an effort to improve users’ search experiences during their information seeking process, providing a personalized information retrieval system is proposed to be one of the effective approaches. To personalize the search systems requires a good understanding of the users. User modeling has been approved to be a good method for learning and representing users. Therefore many user modeling studies have been carried out and some user models have been developed. The majority of the user modeling studies applies inductive approach, and only small number of studies employs deductive approach. In this paper, an EISE (Extended Information goal, Search strategy and Evaluation threshold) user model is proposed, which uses the deductive approach based on psychology theories and an existing user model. Ten users’ interactive search log obtained from the real search engine is applied to validate the proposed user model. The preliminary validation results show that the EISE model can be applied to identify different types of users. The search preferences of the different user types can be applied to inform interactive search system design and development

Perspectives on Bayesian Optimization for HCI

In this position paper we discuss optimization in the HCI domain based on our experiences with Bayesian methods for modeling and optimization of audio systems, including challenges related to evaluating, designing, and optimizing such interfaces. We outline and demonstrate how a combined Bayesian modeling and optimization approach provides a flexible framework for integrating various user and content attributes, while also supporting model-based optimization of HCI systems. Finally, we discuss current and future research direction and applications, such as inferring user needs and optimizing interfaces for computer assisted teaching