Role of Bootstrap Averaging in Generalized Approximate Message Passing

Generalized approximate message passing (GAMP) is a computationally efficient algorithm for estimating an unknown signal $w_0\in\mathbb{R}^N$ from a random linear measurement $y= Xw_0 + \epsilon\in\mathbb{R}^M$, where $X\in\mathbb{R}^{M\times N}$ is a known measurement matrix and $\epsilon$ is the noise vector. The salient feature of GAMP is that it can provide an unbiased estimator $\hat{r}^{\rm G}\sim\mathcal{N}(w_0, \hat{s}^2I_N)$, which can be used for various hypothesis-testing methods. In this study, we consider the bootstrap average of an unbiased estimator of GAMP for the elastic net. By numerically analyzing the state evolution of \emph{approximate message passing with resampling}, which has been proposed for computing bootstrap statistics of the elastic net estimator, we investigate when the bootstrap averaging reduces the variance of the unbiased estimator and the effect of optimizing the size of each bootstrap sample and hyperparameter of the elastic net regularization in the asymptotic setting $M, N\to\infty, M/N\to\alpha\in(0,\infty)$. The results indicate that bootstrap averaging effectively reduces the variance of the unbiased estimator when the actual data generation process is inconsistent with the sparsity assumption of the regularization and the sample size is small. Furthermore, we find that when $w_0$ is less sparse, and the data size is small, the system undergoes a phase transition. The phase transition indicates the existence of the region where the ensemble average of unbiased estimators of GAMP for the elastic net norm minimization problem yields the unbiased estimator with the minimum variance.Comment: 6 pages, 5 figure

Fitting a function to time-dependent ensemble averaged data

Time-dependent ensemble averages, i.e., trajectory-based averages of some observable, are of importance in many fields of science. A crucial objective when interpreting such data is to fit these averages (for instance, squared displacements) with a function and extract parameters (such as diffusion constants). A commonly overlooked challenge in such function fitting procedures is that fluctuations around mean values, by construction, exhibit temporal correlations. We show that the only available general purpose function fitting methods, correlated chi-square method and the weighted least squares method (which neglects correlation), fail at either robust parameter estimation or accurate error estimation. We remedy this by deriving a new closed-form error estimation formula for weighted least square fitting. The new formula uses the full covariance matrix, i.e., rigorously includes temporal correlations, but is free of the robustness issues, inherent to the correlated chi-square method. We demonstrate its accuracy in four examples of importance in many fields: Brownian motion, damped harmonic oscillation, fractional Brownian motion and continuous time random walks. We also successfully apply our method, weighted least squares including correlation in error estimation (WLS-ICE), to particle tracking data. The WLS-ICE method is applicable to arbitrary fit functions, and we provide a publically available WLS-ICE software.Comment: 47 pages (main text: 15 pages, supplementary: 32 pages

A Modern Look at Freedman's Box Model

This paper revisits the box model, a metaphor developed by David Freedman to explain sampling distributions and  statistical inference to introductory statistics students. The basic idea is to represent all random phenomena in terms of drawing tickets at random from a box. In this way, random sampling from a population can be described in the same way as everyday phenomena, like coin tossing and card dealing. For Freedman, box models were merely a thought experiment; calculations were still done using normal approximations. In this paper, we propose a more modern view that treats the box model as a practical simulation framework for conducting inference. We show how concepts in introductory statistics and probability classes can be motivated by simulating from a box model. To facilitate this simulation-based approach to teaching box models, we developed an online, open-source "box model simulator"

Self-Referential Noise and the Synthesis of Three-Dimensional Space

Generalising results from Godel and Chaitin in mathematics suggests that self-referential systems contain intrinsic randomness. We argue that this is relevant to modelling the universe and show how three-dimensional space may arise from a non-geometric order-disorder model driven by self-referential noise.Comment: Figure labels correcte

Force-dependent unbinding rate of molecular motors from stationary optical trap data

Molecular motors walk along filaments until they detach stochastically with a force-dependent unbinding rate. Here, we show that this unbinding rate can be obtained from the analysis of experimental data of molecular motors moving in stationary optical traps. Two complementary methods are presented, based on the analysis of the distribution for the unbinding forces and of the motor's force traces. In the first method, analytically derived force distributions for slip bonds, slip-ideal bonds, and catch bonds are used to fit the cumulative distributions of the unbinding forces. The second method is based on the statistical analysis of the observed force traces. We validate both methods with stochastic simulations and apply them to experimental data for kinesin-1

Bootstrap confidence intervals for the contributions of individual variables to a Mahalanobis distance

Hotelling's T 2 and Mahalanobis distance are widely used in the statistical analysis of multivariate data. When either of these quantities is large, a natural question is: How do individual variables contribute to its size? The Garthwaite–Koch partition has been proposed as a means of assessing the contribution of each variable. This yields point estimates of each variable's contribution and here we consider bootstrap methods for forming interval estimates of these contributions. New bootstrap methods are proposed and compared with the percentile, bias-corrected percentile, non-studentized pivotal and studentized pivotal methods via a large simulation study. The new methods enable use of a broader range of pivotal quantities than with standard pivotal methods, including vector pivotal quantities. In the context considered here, this obviates the need for transformations and leads to intervals that have higher coverage, and yet are narrower, than intervals given by the standard pivotal methods. These results held both when the population distributions were multivariate normal and when they were skew with heavy tails. Both equal-tailed intervals and shortest intervals are constructed; the latter are particularly attractive when (as here) squared quantities are of interest