Speeding Up MCMC by Efficient Data Subsampling

We propose Subsampling MCMC, a Markov Chain Monte Carlo (MCMC) framework where the likelihood function for $n$ observations is estimated from a random subset of $m$ observations. We introduce a highly efficient unbiased estimator of the log-likelihood based on control variates, such that the computing cost is much smaller than that of the full log-likelihood in standard MCMC. The likelihood estimate is bias-corrected and used in two dependent pseudo-marginal algorithms to sample from a perturbed posterior, for which we derive the asymptotic error with respect to $n$ and $m$, respectively. We propose a practical estimator of the error and show that the error is negligible even for a very small $m$ in our applications. We demonstrate that Subsampling MCMC is substantially more efficient than standard MCMC in terms of sampling efficiency for a given computational budget, and that it outperforms other subsampling methods for MCMC proposed in the literature.Comment: Main changes: The theory has been significantly revise

Speeding Up MCMC by Delayed Acceptance and Data Subsampling

The complexity of the Metropolis-Hastings (MH) algorithm arises from the requirement of a likelihood evaluation for the full data set in each iteration. Payne and Mallick (2015) propose to speed up the algorithm by a delayed acceptance approach where the acceptance decision proceeds in two stages. In the first stage, an estimate of the likelihood based on a random subsample determines if it is likely that the draw will be accepted and, if so, the second stage uses the full data likelihood to decide upon final acceptance. Evaluating the full data likelihood is thus avoided for draws that are unlikely to be accepted. We propose a more precise likelihood estimator which incorporates auxiliary information about the full data likelihood while only operating on a sparse set of the data. We prove that the resulting delayed acceptance MH is more efficient compared to that of Payne and Mallick (2015). The caveat of this approach is that the full data set needs to be evaluated in the second stage. We therefore propose to substitute this evaluation by an estimate and construct a state-dependent approximation thereof to use in the first stage. This results in an algorithm that (i) can use a smaller subsample m by leveraging on recent advances in Pseudo-Marginal MH (PMMH) and (ii) is provably within $O(m^{-2})$ of the true posterior.Comment: Accepted for publication in Journal of Computational and Graphical Statistic

Subsampling MCMC - An introduction for the survey statistician

The rapid development of computing power and efficient Markov Chain Monte Carlo (MCMC) simulation algorithms have revolutionized Bayesian statistics, making it a highly practical inference method in applied work. However, MCMC algorithms tend to be computationally demanding, and are particularly slow for large datasets. Data subsampling has recently been suggested as a way to make MCMC methods scalable on massively large data, utilizing efficient sampling schemes and estimators from the survey sampling literature. These developments tend to be unknown by many survey statisticians who traditionally work with non-Bayesian methods, and rarely use MCMC. Our article explains the idea of data subsampling in MCMC by reviewing one strand of work, Subsampling MCMC, a so called pseudo-marginal MCMC approach to speeding up MCMC through data subsampling. The review is written for a survey statistician without previous knowledge of MCMC methods since our aim is to motivate survey sampling experts to contribute to the growing Subsampling MCMC literature.Comment: Accepted for publication in Sankhya A. Previous uploaded version contained a bug in generating the figures and reference

A non-local inequality and global existence

In this article we prove a collection of new non-linear and non-local integral inequalities. As an example for $u\ge 0$ and $p\in (0,\infty)$ we obtain \int_{\threed} dx ~ u^{p+1}(x) \le (\frac{p+1}{p})^2 \int_{\threed} dx ~ \{(-\triangle)^{-1} u(x) \} \nsm \nabla u^{\frac{p}{2}}(x)\nsm^2. We use these inequalities to deduce global existence of solutions to a non-local heat equation with a quadratic non-linearity for large radial monotonic positive initial conditions. Specifically, we improve \cite{ksLM} to include all $\alpha\in (0, 74/75)$.Comment: 6 pages, to appear in Advances in Mathematic

Environmental Influences on Soil Macroarthropod Behavior in Agricultural Systems

Many basic and applied studies in insect ecology have focused on the proximate and ultimate responses of insect populations to their physical and chemical environment (2, 15, 21, 27, 32, 74, 82, 111, 112). From an economic perspective, macro- and microclimatic factors can influence the stress that insect populations inflict on plants and the efficacy of management tactics. For above-ground insects, the mechanisms of behavioral response to environmental factors are often observable, if not always apparent to the researcher. However, this is not typically the situation with soil insects. As a result, field studies of soil insects often quantify only the consequences of behavior while the behaviors themselves remain hidden within the soil matrix (14, 103, 105). Soil ecology research has been productive at the ecosystem level on such topics as nutrient cycling (18), arthropod regulation of micro- and meso-fauna in below-ground detrital food webs (75), impact of microfauna on soil genesis and structure (87), rhizosphere dynamics (17), and energy dynamics of soil systems (79). These examples highlight the importance of multidisciplinary approaches to research programs that unite expertise in insect ecology, soil physics, chemistry, and microbiology as well as systems analysis and modeling (87). Considerable interest also exists in the relationships within soil communities, but these studies have focused primarily upon nonagricultural systems (71, 108) and on the more abundant microarthropod members of the soil fauna ( 106-109). Ecological, morphological, and physiological adaptations of nonagricultural soil arthropods have been discussed in the literature (8, 10, 26, 59); however, insects that are agricultural pests primarily in their immature soil-inhabiting stages have often been studied in detail only in their more accessible adult stage. Although the mobile adult stages of soil pests often determine initial habitat and host selection, a considerable proportion of subsequent host and habitat selection is performed by immatures in the soil, if host or habitat quality deteriorates over time. A major obstacle to the study of soil insect ecology has been the inability to follow soil insect movement and feeding behavior in situ (3, 14, 33, 34, 103, 105). It is critically important in these studies to minimize the disturbance of the soil system through experimental manipulations. R. L. Rabb (cited in 103) notes that the greatest problem with studies of soil insects is that the system is altered through its study. Also, research workers often fail to consider dominant mass and energy transport mechanisms in soil ecosystems. Differences in above- and below-ground environments may alter soil insect sensitivity (over ecological and evolutionary time) to shifting environmental conditions, the movement of chemical cues from potential food sources to soil herbivores, and the mechanisms for soil insect host-finding behavior when compared to terrestrial organisms. In this review we briefly outline several basic principles of soil physics as they relate to soil insect movement and host-finding behavior, to provide a general understanding of the environment in which soil macroarthropods exist. We then selectively review the entomological literature in light of these principles to stress the need to evaluate soil insect behavior within the soil matrix when trying to understand the underlying mechanisms that produce observable behavior. Finally we briefly discuss the importance of behavior in the management of soil insects

Regression density estimation using smooth adaptive Gaussian mixtures

We model a regression density flexibly so that at each value of the covariates the density is a mixture of normals with the means, variances and mixture probabilities of the components changing smoothly as a function of the covariates. The model extends existing models in two important ways. First, the components are allowed to be heteroscedastic regressions as the standard model with homoscedastic regressions can give a poor fit to heteroscedastic data, especially when the number of covariates is large. Furthermore, we typically need fewer components, which makes it easier to interpret the model and speeds up the computation. The second main extension is to introduce a novel variable selection prior into all the components of the model. The variable selection prior acts as a self-adjusting mechanism that prevents overfitting and makes it feasible to fit flexible high-dimensional surfaces. We use Bayesian inference and Markov Chain Monte Carlo methods to estimate the model. Simulated and real examples are used to show that the full generality of our model is required to fit a large class of densities, but also that special cases of the general model are interesting models for economic data

Use of Radiography in Behavioral Studies of Turfgrass-Infesting Scarab Grub Species (Coleoptera: Scarabaeidae)

The behavior of turfgrass-infesting scarab grubs in response to soil physical properties may affect the stress that each species exerts on turfgrass and the efficacy of control tactics. To gain a more realistic picture of the events that occur within the soil matrix, we have developed a nondestructive X-ray technique to study soil insect movement and behavior in simulated and natural soil blocks in the laboratory Laboratory studies using this technique were done to determine the effect of some soil physical factors on scarab grub movement patterns. Species-specific differences were demonstrated in the responses of four scarab grub species (Japanese beetle, Popillia japonica Newman; European chafer, Rhizotrogus mqalis (Razoumowsky); oriental beetle, Anomala orientalis Waterhouse; and northern masked chafer Cyclocephala borealis (Arrow) ) to changing temperature and moisture conditions. Studies also were done to determine the effect of soil moisture on the movement and persistence of an insecticide (isofenphos) applied to turfgrass and its effect on European chafer grub movement and mortality This study showed that isofenphos was relatively nonmobile under our experimental conditions, and that insecticide efficacy depended on factors, such as soil moisture, that influence the position of grubs in the soil profile. We believe that a better understanding of the interactions among grub behavior, insecticide persistence, and movement, as illustrated by this research, will improve our ability to manage scarab grubs in turfgrass and will be applicable to additional soil systems

Environmental Influences on Soil Macroarthropod Behavior in Agricultural Systems

Many basic and applied studies in insect ecology have focused on the proximate and ultimate responses of insect populations to their physical and chemical environment (2, 15, 21, 27, 32, 74, 82, 111, 112). From an economic perspective, macro- and microclimatic factors can influence the stress that insect populations inflict on plants and the efficacy of management tactics. For above-ground insects, the mechanisms of behavioral response to environmental factors are often observable, if not always apparent to the researcher. However, this is not typically the situation with soil insects. As a result, field studies of soil insects often quantify only the consequences of behavior while the behaviors themselves remain hidden within the soil matrix (14, 103, 105). Soil ecology research has been productive at the ecosystem level on such topics as nutrient cycling (18), arthropod regulation of micro- and meso-fauna in below-ground detrital food webs (75), impact of microfauna on soil genesis and structure (87), rhizosphere dynamics (17), and energy dynamics of soil systems (79). These examples highlight the importance of multidisciplinary approaches to research programs that unite expertise in insect ecology, soil physics, chemistry, and microbiology as well as systems analysis and modeling (87). Considerable interest also exists in the relationships within soil communities, but these studies have focused primarily upon nonagricultural systems (71, 108) and on the more abundant microarthropod members of the soil fauna ( 106-109). Ecological, morphological, and physiological adaptations of nonagricultural soil arthropods have been discussed in the literature (8, 10, 26, 59); however, insects that are agricultural pests primarily in their immature soil-inhabiting stages have often been studied in detail only in their more accessible adult stage. Although the mobile adult stages of soil pests often determine initial habitat and host selection, a considerable proportion of subsequent host and habitat selection is performed by immatures in the soil, if host or habitat quality deteriorates over time. A major obstacle to the study of soil insect ecology has been the inability to follow soil insect movement and feeding behavior in situ (3, 14, 33, 34, 103, 105). It is critically important in these studies to minimize the disturbance of the soil system through experimental manipulations. R. L. Rabb (cited in 103) notes that the greatest problem with studies of soil insects is that the system is altered through its study. Also, research workers often fail to consider dominant mass and energy transport mechanisms in soil ecosystems. Differences in above- and below-ground environments may alter soil insect sensitivity (over ecological and evolutionary time) to shifting environmental conditions, the movement of chemical cues from potential food sources to soil herbivores, and the mechanisms for soil insect host-finding behavior when compared to terrestrial organisms. In this review we briefly outline several basic principles of soil physics as they relate to soil insect movement and host-finding behavior, to provide a general understanding of the environment in which soil macroarthropods exist. We then selectively review the entomological literature in light of these principles to stress the need to evaluate soil insect behavior within the soil matrix when trying to understand the underlying mechanisms that produce observable behavior. Finally we briefly discuss the importance of behavior in the management of soil insects