Optimization of Annealed Importance Sampling Hyperparameters

Annealed Importance Sampling (AIS) is a popular algorithm used to estimates the intractable marginal likelihood of deep generative models. Although AIS is guaranteed to provide unbiased estimate for any set of hyperparameters, the common implementations rely on simple heuristics such as the geometric average bridging distributions between initial and the target distribution which affect the estimation performance when the computation budget is limited. In order to reduce the number of sampling iterations, we present a parameteric AIS process with flexible intermediary distributions defined by a residual density with respect to the geometric mean path. Our method allows parameter sharing between annealing distributions, the use of fix linear schedule for discretization and amortization of hyperparameter selection in latent variable models. We assess the performance of Optimized-Path AIS for marginal likelihood estimation of deep generative models and compare it to compare it to more computationally intensive AIS

Exploring Instructor Contributions to Discussions in Massive Open Online Courses (MOOCs)

OOCs as a new form of online education have attracted the attention of researchers; however, little research has examined MOOC instructors’ practices particularly in delivering the courses. Therefore, this study set out to explore what instructors do in MOOC discussion areas and how learners react to them. Drawing on an extended mixed design, this research investigated the level (frequency) and type of instructors’ contributions to discussion areas, and the ways and extent to which learners engage with them. First, the content of 818 learner-instructor conversations of three FutureLearn MOOCs were analysed based on the Community of Inquiry (CoI) framework. Instructors’ contributions were then studied for learners’ explicit (responding) and implicit (liking) engagement. In addition, the changes to instructors’ contributions and learners’ engagement over the duration of courses were examined to explore the impact of time on instructors’ and learners’ discussion activities. Finally, in-depth qualitative interviews were conducted with instructors to understand the role of their contributions in learning. The findings revealed that social postings are the clear majority of instructor contributions, whilst postings related to teaching and cognitive presences constitute a smaller proportion. This indicates that instructors do not focus on all contribution types equally and that there is an imbalance between the social and content-related support that learners receive. More specifically, the results showed that instructors’ teaching contributions focus on facilitating the learning discourse and less on providing direct instruction. This suggests that instructors take a facilitative rather than a directive or leading role in FutureLearn MOOCs. The predominance of instructors’ social contributions, on the other hand, signifies the social emphasis of instructors’ discussion activities. Furthermore, the analysis showed that learners engaged with 42% of instructors’ contributions by responding to or liking them or a combination of both. Most learner engagement was evident when instructors’ contributions were focused on teaching presence. The most engaging combination appeared to be a high level of direct instruction and facilitating discourse in a contribution and the lowest level of affective responses. Considering the level of instructors’ contributions, more than half of contributions occurred at the beginning of MOOCs, and this proportion had more than halved by the middle and reached its lowest level at the end of MOOCs. Despite the decrease in all contribution types over time, the relative importance of each type changed. This study also showed that although the Community of Inquiry framework required re-operationalisation and re-conceptualisation of some indicators and the introduction of three new ones to describe the dynamics of learner-instructor interactions in MOOCs, it provided a powerful lens to explore MOOC instructor discussion activities. While this study has resulted in an enhanced understanding of instructors’ contributions to the MOOC discussions, and offered new insights into learners’ engagement with instructors, it revisited the CoI framework in a MOOC context. Thus, the significance of this study also lies in proposing a revised model that can inform future research into learning and teaching in MOOCs or other open, scaled and informal educational contexts

Anchor Data Augmentation

We propose a novel algorithm for data augmentation in nonlinear over-parametrized regression. Our data augmentation algorithm borrows from the literature on causality and extends the recently proposed Anchor regression (AR) method for data augmentation, which is in contrast to the current state-of-the-art domain-agnostic solutions that rely on the Mixup literature. Our Anchor Data Augmentation (ADA) uses several replicas of the modified samples in AR to provide more training examples, leading to more robust regression predictions. We apply ADA to linear and nonlinear regression problems using neural networks. ADA is competitive with state-of-the-art C-Mixup solutions

Adaptive Annealed Importance Sampling with Constant Rate Progress

Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution given its unnormalized density function. This algorithm relies on a sequence of interpolating distributions bridging the target to an initial tractable distribution such as the well-known geometric mean path of unnormalized distributions which is assumed to be suboptimal in general. In this paper, we prove that the geometric annealing corresponds to the distribution path that minimizes the KL divergence between the current particle distribution and the desired target when the feasible change in the particle distribution is constrained. Following this observation, we derive the constant rate discretization schedule for this annealing sequence, which adjusts the schedule to the difficulty of moving samples between the initial and the target distributions. We further extend our results to $f$-divergences and present the respective dynamics of annealing sequences based on which we propose the Constant Rate AIS (CR-AIS) algorithm and its efficient implementation for $\alpha$-divergences. We empirically show that CR-AIS performs well on multiple benchmark distributions while avoiding the computationally expensive tuning loop in existing Adaptive AIS

Look who’s talking: exploring instructors’ contributions to Massive Open Online Courses

© 2019 British Educational Research Association Previous research on xMOOC pedagogy has established that learner interactions in discussion forums play a fundamental role in learning. However, little is known about the extent to which MOOC instructors engage with learner conversations and the impact this has on learner engagement. Adopting a novel design, this study went beyond self-reported methods, and combined transcript analysis and in-depth interviews to examine the dynamics of learner-instructor interactions and to revisit the use of the Community of Inquiry framework (CoI) in MOOC context. The findings revealed that the majority of instructors’ contributions to learner conversations are social, followed by teaching and cognitive contributions. While all contribution types decreased over the duration of the MOOC, the relative importance of each type did not necessarily change. Furthermore, the analysis showed that learners engaged with 42% of instructor contributions by responding to or liking them or both. Considering the application of the CoI to massive and open online learning contexts, this study demonstrates that while the framework can unfold educational transactions in MOOCs, reoperationalisation and reconceptualisation of some indicators along with the introduction of new indicators are essential. The implications of this for theory and practice are discussed