Measuring Cluster Stability for Bayesian Nonparametrics Using the Linear Bootstrap

Clustering procedures typically estimate which data points are clustered together, a quantity of primary importance in many analyses. Often used as a preliminary step for dimensionality reduction or to facilitate interpretation, finding robust and stable clusters is often crucial for appropriate for downstream analysis. In the present work, we consider Bayesian nonparametric (BNP) models, a particularly popular set of Bayesian models for clustering due to their flexibility. Because of its complexity, the Bayesian posterior often cannot be computed exactly, and approximations must be employed. Mean-field variational Bayes forms a posterior approximation by solving an optimization problem and is widely used due to its speed. An exact BNP posterior might vary dramatically when presented with different data. As such, stability and robustness of the clustering should be assessed. A popular mean to assess stability is to apply the bootstrap by resampling the data, and rerun the clustering for each simulated data set. The time cost is thus often very expensive, especially for the sort of exploratory analysis where clustering is typically used. We propose to use a fast and automatic approximation to the full bootstrap called the "linear bootstrap", which can be seen by local data perturbation. In this work, we demonstrate how to apply this idea to a data analysis pipeline, consisting of an MFVB approximation to a BNP clustering posterior of time course gene expression data. We show that using auto-differentiation tools, the necessary calculations can be done automatically, and that the linear bootstrap is a fast but approximate alternative to the bootstrap.Comment: 9 pages, NIPS 2017 Advances in Approximate Bayesian Inference Worksho

Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics (Rejoinder)

One can typically form a local robustness metric for a particular problem quite directly, for Markov chain Monte Carlo applications as well as optimization problems such as variational Bayes. However, we argue that simply forming a local robustness metric is not enough: the hard work is showing that it is useful. Computability, interpretability, and the ability of a local robustness metric to extrapolate well, are more important -- and often more difficult to establish -- than mere computation of derivatives.Comment: Rejoinder for the discussion article "Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics'' in Bayesian Analysi

Scalable Bayesian Inference for Detection and Deblending in Astronomical Images

We present a new probabilistic method for detecting, deblending, and cataloging astronomical sources called the Bayesian Light Source Separator (BLISS). BLISS is based on deep generative models, which embed neural networks within a Bayesian model. For posterior inference, BLISS uses a new form of variational inference known as Forward Amortized Variational Inference. The BLISS inference routine is fast, requiring a single forward pass of the encoder networks on a GPU once the encoder networks are trained. BLISS can perform fully Bayesian inference on megapixel images in seconds, and produces highly accurate catalogs. BLISS is highly extensible, and has the potential to directly answer downstream scientific questions in addition to producing probabilistic catalogs.Comment: Accepted to the ICML 2022 Workshop on Machine Learning for Astrophysics. 5 pages, 2 figure

Methods and applications in large-scale variational inference

This dissertation can be viewed as a collection of case studies in applying variational inferenceto analyze large data problems. The largest, most computationally difficult data problem we tackle is the task of cataloging light sources imaged by astronomical surveys (Chapter 2). For some surveys, the size of the raw data is on the scale of petabytes. To do inference, we employ two recent advances in variational inference: amortization and the wake-sleep algorithm.The data analysis problems in Chapters 3, 4, and 5 are more tractable. The analyses inthese chapters are exploratory in nature. However, such exploratory data analyses often require fitting either several related models or fitting the same model on subsamples of the data. Repeatedly solving for variational optima after each model or data perturbation may be unnecessarily expensive, particularly for exploratory settings where approximate optima might suffice. In these chapters, we present the notion of local sensitivity which we use to quickly extrapolate, from an initial variational optimum, the posterior quantities that would be obtained after a model or data perturbation. Our approach is particularly apt for sensitivity analysis, where the goal is to understand how conclusions might change should a different model be specified or should different data be observed.Finally, Chapter 6 considers probabilistic machine learning problems where the trainingobjective is an expectation over a discrete latent variable. The standard reparameterization and backpropagation method for computing stochastic gradients do not apply in this setting. Many alternative stochastic gradient estimators have been proposed specifically for this problem. Chapter 6 outlines a technique to lower the variance of any gradient estimator by employing a general statistical method called Rao-Blackwellization

Modeling the Effects of Positive and Negative Feedback in Kidney Blood Flow Control

This paper models the interactions of three key feedback mechanisms that regulate blood flow in the mammalian kidney: (1) the myogenic response, triggered by blood pressure in the afferent arteriole; (2) tubuloglomerular feedback (TGF), a negative feedback mechanism responding to chloride concentrations at the mascula densa (MD); and (3) connecting tubule glomerular feedback (CTGF), a positive feedback mechanism responding to chloride concentrations in the connecting tubule, downstream of the mascula densa. Previous models have studied the myogenic response and TGF. However, CTGF is much less well understood, and we thus aim to construct a mathematical model incorporating all three mechanisms. A bifurcation analysis was performed on this expanded model to predict the behavior of the system over a range of physiologically realistic parameters, and numerical simulations of the model equations were computed to supplement the results of the bifurcation analysis. In doing so, we seek to elucidate the interactions of all three feedback mechanisms and their effects on kidney blood flow. In particular, numerical simulations were able to confirm our hypothesis that the interactions between TGF and CTGF give rise to an experimentally observed low frequency oscillation that could not be explained by previous models incorporating TGF alone.Honors thesi

The Wako-Saitô-Muñoz-Eaton Model for Predicting Protein Folding and Dynamics

Despite the recent advances in the prediction of protein structures by deep neutral networks, the elucidation of protein-folding mechanisms remains challenging. A promising theory for describing protein folding is a coarse-grained statistical mechanical model called the Wako-Saitô-Muñoz-Eaton (WSME) model. The model can calculate the free-energy landscapes of proteins based on a three-dimensional structure with low computational complexity, thereby providing a comprehensive understanding of the folding pathways and the structure and stability of the intermediates and transition states involved in the folding reaction. In this review, we summarize previous and recent studies on protein folding and dynamics performed using the WSME model and discuss future challenges and prospects. The WSME model successfully predicted the folding mechanisms of small single-domain proteins and the effects of amino-acid substitutions on protein stability and folding in a manner that was consistent with experimental results. Furthermore, extended versions of the WSME model were applied to predict the folding mechanisms of multi-domain proteins and the conformational changes associated with protein function. Thus, the WSME model may contribute significantly to solving the protein-folding problem and is expected to be useful for predicting protein folding, stability, and dynamics in basic research and in industrial and medical applications