Bayesian Dropout

Dropout has recently emerged as a powerful and simple method for training neural networks preventing co-adaptation by stochastically omitting neurons. Dropout is currently not grounded in explicit modelling assumptions which so far has precluded its adoption in Bayesian modelling. Using Bayesian entropic reasoning we show that dropout can be interpreted as optimal inference under constraints. We demonstrate this on an analytically tractable regression model providing a Bayesian interpretation of its mechanism for regularizing and preventing co-adaptation as well as its connection to other Bayesian techniques. We also discuss two general approximate techniques for applying Bayesian dropout for general models, one based on an analytical approximation and the other on stochastic variational techniques. These techniques are then applied to a Baysian logistic regression problem and are shown to improve performance as the model become more misspecified. Our framework roots dropout as a theoretically justified and practical tool for statistical modelling allowing Bayesians to tap into the benefits of dropout training.Comment: 21 pages, 3 figures. Manuscript prepared 2014 and awaiting submissio

Integrating economic values and catchment modelling

Integrated catchment policies are widely used to manage natural resources in Australian catchments. Decision support tools available to aid integrated catchment management are often limited in their integration of environmental processes with socio-economic systems. Fully integrated models are required to support assessments of the environmental and economic trade-offs of catchment management changes. A Bayesian Network (BN) model is demonstrated to provide a suitable approach to integrate environmental modelling with economic valuation. The model incorporates hydrological, ecological and economic models for the George catchment in Tasmania. Information about the non-market costs and benefits of environmental changes is elicited using Choice Experiments, allowing an assessment of the efficiency of alternative management scenarios.Integrated catchment modelling, Bayesian networks, Uncertainty, Environmental values, Non-market valuation, Choice Modelling.,

Bayesian modelling of clusters of galaxies from multi-frequency pointed Sunyaev--Zel'dovich observations

We present a Bayesian approach to modelling galaxy clusters using multi-frequency pointed observations from telescopes that exploit the Sunyaev--Zel'dovich effect. We use the recently developed MultiNest technique (Feroz, Hobson & Bridges, 2008) to explore the high-dimensional parameter spaces and also to calculate the Bayesian evidence. This permits robust parameter estimation as well as model comparison. Tests on simulated Arcminute Microkelvin Imager observations of a cluster, in the presence of primary CMB signal, radio point sources (detected as well as an unresolved background) and receiver noise, show that our algorithm is able to analyse jointly the data from six frequency channels, sample the posterior space of the model and calculate the Bayesian evidence very efficiently on a single processor. We also illustrate the robustness of our detection process by applying it to a field with radio sources and primordial CMB but no cluster, and show that indeed no cluster is identified. The extension of our methodology to the detection and modelling of multiple clusters in multi-frequency SZ survey data will be described in a future work.Comment: 12 pages, 7 figures, submitted to MNRA

Forecasting environmental migration to the United Kingdom, 2010 - 2060: an exploration using Bayesian models

Over the next fifty years the potential impact on human livelihoods of environmental change could be considerable. One possible response may be increased levels of human mobility. This paper offers a first quantification of the levels of environmental migration to the United Kingdom that might be expected. The authors apply Bijak and Wi?niowski’s (2010) methodology for forecasting migration using Bayesian models. They seek to advance the conceptual understanding of forecasting in three ways. First, the paper is believed to be the first time that the Bayesian modelling approach has been attempted in relation to environmental mobility. Second, the paper examines the plausibility of Bayesian modelling of UK immigration by cross-checking expert responses to a Delphi survey with the expectations about environmental mobility evident in the recent research literature. Third, the values and assumptions of the expert evidence provided in the Delphi survey are interrogated to illustrate the limited set of conditions under which the forecasts of environmental mobility, as set out in this paper, are likely to hold

A geoadditive Bayesian latent variable model for Poisson indicators

We introduce a new latent variable model with count variable indicators, where usual linear parametric effects of covariates, nonparametric effects of continuous covariates and spatial effects on the continuous latent variables are modelled through a geoadditive predictor. Bayesian modelling of nonparametric functions and spatial effects is based on penalized spline and Markov random field priors. Full Bayesian inference is performed via an auxiliary variable Gibbs sampling technique, using a recent suggestion of Frühwirth-Schnatter and Wagner (2006). As an advantage, our Poisson indicator latent variable model can be combined with semiparametric latent variable models for mixed binary, ordinal and continuous indicator variables within an unified and coherent framework for modelling and inference. A simulation study investigates performance, and an application to post war human security in Cambodia illustrates the approach

Bayesian modelling and quantification of Raman spectroscopy

Raman spectroscopy can be used to identify molecules such as DNA by the characteristic scattering of light from a laser. It is sensitive at very low concentrations and can accurately quantify the amount of a given molecule in a sample. The presence of a large, nonuniform background presents a major challenge to analysis of these spectra. To overcome this challenge, we introduce a sequential Monte Carlo (SMC) algorithm to separate each observed spectrum into a series of peaks plus a smoothly-varying baseline, corrupted by additive white noise. The peaks are modelled as Lorentzian, Gaussian, or pseudo-Voigt functions, while the baseline is estimated using a penalised cubic spline. This latent continuous representation accounts for differences in resolution between measurements. The posterior distribution can be incrementally updated as more data becomes available, resulting in a scalable algorithm that is robust to local maxima. By incorporating this representation in a Bayesian hierarchical regression model, we can quantify the relationship between molecular concentration and peak intensity, thereby providing an improved estimate of the limit of detection, which is of major importance to analytical chemistry