Efficient Bayesian Nonparametric Modelling of Structured Point Processes

This paper presents a Bayesian generative model for dependent Cox point processes, alongside an efficient inference scheme which scales as if the point processes were modelled independently. We can handle missing data naturally, infer latent structure, and cope with large numbers of observed processes. A further novel contribution enables the model to work effectively in higher dimensional spaces. Using this method, we achieve vastly improved predictive performance on both 2D and 1D real data, validating our structured approach.Comment: Presented at UAI 2014. Bibtex: @inproceedings{structcoxpp14_UAI, Author = {Tom Gunter and Chris Lloyd and Michael A. Osborne and Stephen J. Roberts}, Title = {Efficient Bayesian Nonparametric Modelling of Structured Point Processes}, Booktitle = {Uncertainty in Artificial Intelligence (UAI)}, Year = {2014}

How to circumvent the size limitation of liquid metal batteries due to the Tayler instability

Recently, a new type of battery has been proposed that relies on the principle of self-assembling of a liquid metalloid positive electrode, a liquid electrolyte, and a liquid metal negative electrode. While this configuration has been claimed to allow arbitrary up-scaling, there is a size limitation of such a system due to a current-driven kink-type instability that is known as the Tayler instability. We characterize this instability in large-scale self-assembled liquid metal batteries and discuss various technical means how it can be avoided.Comment: 15 pages, 5 figure

Large Language Model-guided Document Selection

Large Language Model (LLM) pre-training exhausts an ever growing compute budget, yet recent research has demonstrated that careful document selection enables comparable model quality with only a fraction of the FLOPs. Inspired by efforts suggesting that domain-specific training document selection is in fact an interpretable process [Gunasekar et al., 2023], as well as research showing that instruction-finetuned LLMs are adept zero-shot data labelers [Gilardi et al.,2023], we explore a promising direction for scalable general-domain document selection; employing a prompted LLM as a document grader, we distill quality labels into a classifier model, which is applied at scale to a large, and already heavily-filtered, web-crawl-derived corpus autonomously. Following the guidance of this classifier, we drop 75% of the corpus and train LLMs on the remaining data. Results across multiple benchmarks show that: 1. Filtering allows us to quality-match a model trained on the full corpus across diverse benchmarks with at most 70% of the FLOPs, 2. More capable LLM labelers and classifier models lead to better results that are less sensitive to the labeler's prompt, 3. In-context learning helps to boost the performance of less-capable labeling models. In all cases we use open-source datasets, models, recipes, and evaluation frameworks, so that results can be reproduced by the community.Comment: 9 page

Factors affecting faculty use of learning technologies: Implications for models of technology adoption

This study examines factors associated with the use of learning technologies by higher education faculty. In an online survey in a UK university, 114 faculty respondents completed a measure of Internet self-efficacy, and reported on their use of learning technologies along with barriers to their adoption. Principal components analysis suggested two main barriers to adoption: structural constraints within the University and perceived usefulness of the tools. Regression analyses indicated both these variables, along with Internet self-efficacy, were associated with use of online learning technology. These findings are more consistent with models of technology engagement that recognize facilitating or inhibiting conditions (unified theory of acceptance and use of technology; decomposed theory of planned behavior) than the classic technology acceptance model (TAM). Practical implications for higher education institutions are that while faculty training and digital literacy initiatives may have roles to play, structural factors (e.g., provision of resources and technical support) must also be addressed for optimal uptake of learning technologies

Self Supervision Does Not Help Natural Language Supervision at Scale

Self supervision and natural language supervision have emerged as two exciting ways to train general purpose image encoders which excel at a variety of downstream tasks. Recent works such as M3AE and SLIP have suggested that these approaches can be effectively combined, but most notably their results use small pre-training datasets (<50M samples) and don't effectively reflect the large-scale regime (>100M examples) that is commonly used for these approaches. Here we investigate whether a similar approach can be effective when trained with a much larger amount of data. We find that a combination of two state of the art approaches: masked auto-encoders, MAE and contrastive language image pre-training, CLIP provides a benefit over CLIP when trained on a corpus of 11.3M image-text pairs, but little to no benefit (as evaluated on a suite of common vision tasks) over CLIP when trained on a large corpus of 1.4B images. Our work provides some much needed clarity into the effectiveness (or lack thereof) of self supervision for large-scale image-text training

Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature

We propose a novel sampling framework for inference in probabilistic models: an active learning approach that converges more quickly (in wall-clock time) than Markov chain Monte Carlo (MCMC) benchmarks. The central challenge in probabilistic inference is numerical integration, to average over ensembles of models or unknown (hyper-)parameters (for example to compute the marginal likelihood or a partition function). MCMC has provided approaches to numerical integration that deliver state-of-the-art inference, but can suffer from sample inefficiency and poor convergence diagnostics. Bayesian quadrature techniques offer a model-based solution to such problems, but their uptake has been hindered by prohibitive computation costs. We introduce a warped model for probabilistic integrands (likelihoods) that are known to be non-negative, permitting a cheap active learning scheme to optimally select sample locations. Our algorithm is demonstrated to offer faster convergence (in seconds) relative to simple Monte Carlo and annealed importance sampling on both synthetic and real-world examples

Mapping engineering ethics education

Writing a handbook implies describing the fundamental information needed by those teaching or researching in a field. This is a challenging task when the field is still maturing. This handbook grew from the idea that we could ‘collaboratively write’ engineering ethics education. In this introduction, we define the field in three dimensions: engineering, ethics, and education. The latter dimension is divided into three parts: the subjects that shape the teaching of engineering ethics education, the understanding of learners and learning that informs the pedagogical choices made, and the fusion of these two in the pedagogical methods used in teaching engineering ethics. Mapping these three dimensions implies taking an interdisciplinary approach, writing in a multi-perspectival way, and striving to critically reflect on – some in the editorial team would say ‘decolonize’ – the discourses that have traditionally dominated engineering ethics education. Since our starting point was to write engineering ethics education collaboratively, our process in editing this handbook meant we tried to be open to a wide diversity of voices while at the same time working to create coherence within chapters, within thematic sections, and across the book as a whole – without losing the possibility to say something significant about where engineering ethics education is today as a research area and as a topic for teaching. Working with 108 authors, the result is a collaborative, rich, multi-perspectival text