Parameter inference and model comparison using theoretical predictions from noisy simulations

When inferring unknown parameters or comparing different models, data must be compared to underlying theory. Even if a model has no closed-form solution to derive summary statistics, it is often still possible to simulate mock data in order to generate theoretical predictions. For realistic simulations of noisy data, this is identical to drawing realizations of the data from a likelihood distribution. Though the estimated summary statistic from simulated data vectors may be unbiased, the estimator has variance which should be accounted for. We show how to correct the likelihood in the presence of an estimated summary statistic by marginalizing over the true summary statistic in the framework of a Bayesian hierarchical model. For Gaussian likelihoods where the covariance must also be estimated from simulations, we present an alteration to the Sellentin-Heavens corrected likelihood. We show that excluding the proposed correction leads to an incorrect estimate of the Bayesian evidence with JLA data. The correction is highly relevant for cosmological inference that relies on simulated data for theory (e.g. weak lensing peak statistics and simulated power spectra) and can reduce the number of simulations required.Comment: 9 pages, 6 figures, published by MNRAS. Changes: matches published version, added Bayesian hierarchical interpretation and probabilistic graphical mode

Cosmology with dark matter maps

Physics is experiencing an exciting period of exploration into the nature of dark energy, dark matter, and gravitation. With 95% of the mass-energy of the Universe still unexplained, the answers to many further fundamental questions of astro-, theoretical- and particle-physics are being hampered. In the coming years, DES, HSC, KiDS, Euclid and LSST will image billions of galaxies, aiming to use observational data from the late Universe to infer cosmological parameters and compare cosmological models. One of the most promising observables is the weak gravitational lensing effect. Using the statistical power from many small distortions, called shear, DES has provided excellent constraints. However, the standard 2-point statistics do not capture the full information in the data. In the late Universe, gravitational collapse has led to a highly non-Gaussian density field, for which 2-point correlations are not a unique statistical description, and even all N-point functions cannot completely characterize. The research presented in this thesis focuses on methods to reconstruct mass maps from DES weak lensing data and using map-based statistics to infer cosmological parameters and assess theoretical models in a principled Bayesian framework. In Chapter 2, I compare three mass mapping methods with closed-form priors using DES SV data and simulations. In Chapter 3, I demonstrate how the Wiener filter (one of the above methods) computation can be sped up by an order of magnitude using Dataflow Engines (reconfigurable hardware). In Chapter 4, I present a Bayesian hierarchical model which takes into account added uncertainty introduced when noisy simulations are used to generate theoretical predictions. In Chapter 5, with my publicly available DeepMass code, I demonstrate how mass maps reconstructions can be improved (> 10% mean-square-error compared with previously presented methods) using deep learning techniques trained on simulations. In Chapter 6, I discuss future work and the applicability of likelihood-free inference methods for map-based statistics

Using Teachers’ Judgments of Quality to Establish Performance Standards in Technology Education Across Schools, Communities, and Nations

The establishment and maintenance of national examination standards remains a serious issue for teachers and learners, whilst the levers of control remain firmly in the hands of Awarding Bodies and supervising politicians. Significantly, holistic assessment presents an agility and collective approach to establishing in the minds of teachers “what is of value” when determining the comparative evidence of pupil performance. It is argued in this paper that the collation of the comparative judgment process can initially identify and subsequently maintain standards of performance that can be defined on a cluster, regional or even national level. Much comparative judgment research centers on the formative benefits for learners, but here we place the focus on teachers operating in collaborative groups to establish standards within and beyond their own schools, and ultimately across the nation. We model a proof-of-concept research project. A rank is produced by the collective consensus of the participating teachers and used to simulate a definition of standard. Extrapolations are statistically modeled to demonstrate the potential for this approach to establishing a robust definition of national standards. But central to the process is what is going on in the minds of teachers as they make their judgements of quality. The research aims to draw out teachers’ constructs of quality; to make them explicit; to share them across classrooms and schools; and to empower teachers to debate and agree their standards across schools. This research brings to the fore the symbiotic relationship between teaching, learning and assessment

A Review of the Valid Methodological Use of Adaptive Comparative Judgment in Technology Education Research

There is a continuing rise in studies examining the impact that adaptive comparative judgment (ACJ) can have on practice in technology education. This appears to stem from ACJ being seen to offer a solution to the difficulties faced in the assessment of designerly activity which is prominent in contemporary technology education internationally. Central research questions to date have focused on whether ACJ was feasible, reliable, and offered broad educational merit. With exploratory evidence indicating this to be the case, there is now a need to progress this research agenda in a more systematic fashion. To support this, a critical review of how ACJ has been used and studied in prior work was conducted. The findings are presented thematically and suggest the existence of internal validity threats in prior research, the need for a theoretical framework and the consideration of falsifiability, and the need to justify and make transparent methodological and analytical procedures. Research questions now of pertinent importance are presented, and it is envisioned that the observations made through this review will support the design of future inquiry

Modelling approaches to combining and comparing independent adaptive comparative judgement ranks

The use of Adaptive Comparative Judgement (ACJ) for educational assessment addresses one need within technology education for the reliable assessment of responses to open-ended activities which are characteristic within the field. The output of an ACJ session is a rank order of the piece of student work with relative “ability scores”. However, the use of ACJ has been limited to date in that ranks are not directly comparable. For example, a rank produced from one class group has no reference information against which to compare a rank produced of the work of another class group. In this type of case a solution has been to combine the work of both classes into one ACJ session, but this has limitation when considering scaling up. A new goal for the use of ACJ involves solving this issue. The ability to compare or merge ranks presents a new capacity for ACJ – to use a rank as a “ruler” against which other ranks can be compared. In practice this would allow for two possibilities. The first is that a single rank could be developed which presents a national standard against which teachers could compare the work of their students to see where they are performing on a national level. The second is that communities of practice could complete ACJ sessions within their own classrooms, and when meeting as a group they could merge and compare relative performance of their own students to support professional development. In a previous article a proof of concept of this process conducted via simulation was presented (Buckley and Canty, 2022). In this article we present the results of a project with authentic data – student work completed in response to meaningful activities with teachers acting as ACJ judges – which indicate that the use of ACJ in this way is now possible

Framing Spatial Cognition: Establishing a Research Agenda

A significant aim of research concerning human intelligence is to develop a comprehensive cognitive map of the human intelligence structure. The evolution of this knowledge base is mirrored through the chronological development of models which frame cognitive domains. The domain of Visual Processing (Gv), commonly known as spatial ability, is a domain which has seen significant advances in the pertinent knowledge base. Models framing this cognitive structure are arguably under-evolved through a lack of representation of factors identified in contemporary research. This paper presents the initial conception of a more comprehensive theoretical framework which builds upon existing theory. It is envisioned that such a framework could support further research exploring the nature of thinking in graphics and other related disciplines. A research agenda is discussed concerning the validation of this framework and its utilization in the holistic assessment of spatial ability

Year in review 2006: Critical Care – respirology

The present article summarises and places in context original research articles from the respirology section published in Critical Care in 2006. Twenty papers were identified and were grouped by topic into those addressing acute lung injury and ventilator-induced lung injury, those examining high-frequency oscillation, those studying pulmonary physiology and mechanics, those assessing tracheostomy, and those exploring other topics