Using simulation studies to evaluate statistical methods

Simulation studies are computer experiments that involve creating data by pseudorandom sampling. The key strength of simulation studies is the ability to understand the behaviour of statistical methods because some 'truth' (usually some parameter/s of interest) is known from the process of generating the data. This allows us to consider properties of methods, such as bias. While widely used, simulation studies are often poorly designed, analysed and reported. This tutorial outlines the rationale for using simulation studies and offers guidance for design, execution, analysis, reporting and presentation. In particular, this tutorial provides: a structured approach for planning and reporting simulation studies, which involves defining aims, data-generating mechanisms, estimands, methods and performance measures ('ADEMP'); coherent terminology for simulation studies; guidance on coding simulation studies; a critical discussion of key performance measures and their estimation; guidance on structuring tabular and graphical presentation of results; and new graphical presentations. With a view to describing recent practice, we review 100 articles taken from Volume 34 of Statistics in Medicine that included at least one simulation study and identify areas for improvement.Comment: 31 pages, 9 figures (2 in appendix), 8 tables (1 in appendix

Standardising terminology and notation for the analysis of demographic processes in marked populations

The development of statistical methods for the analysis of demographic processes in marked animal populations has brought with it the challenges of communication between the disciplines of statistics, ecology, evolutionary biology and computer science. In order to aid communication and comprehension, we sought to root out a number of cases of ambiguity, redundancy and inaccuracy in notation and terminology that have developed in the literature. We invited all working in this field to submit topics for resolution and to express their own views. In the ensuing discussion forum it was then possible to establish a series of general principles which were, almost without exception, unanimously accepted. Here we set out the background to the areas of confusion, how these were debated and the conclusions which were reached in each case. We hope that the resulting guidelines will be widely adopted as standard terminology in publications and in software for the analysis of demographic processes in marked animal populationspostprin

Mixture models for distance sampling detection functions

Funding: EPSRC DTGWe present a new class of models for the detection function in distance sampling surveys of wildlife populations, based on finite mixtures of simple parametric key functions such as the half-normal. The models share many of the features of the widely-used “key function plus series adjustment” (K+A) formulation: they are flexible, produce plausible shapes with a small number of parameters, allow incorporation of covariates in addition to distance and can be fitted using maximum likelihood. One important advantage over the K+A approach is that the mixtures are automatically monotonic non-increasing and non-negative, so constrained optimization is not required to ensure distance sampling assumptions are honoured. We compare the mixture formulation to the K+A approach using simulations to evaluate its applicability in a wide set of challenging situations. We also re-analyze four previously problematic real-world case studies. We find mixtures outperform K+A methods in many cases, particularly spiked line transect data (i.e., where detectability drops rapidly at small distances) and larger sample sizes. We recommend that current standard model selection methods for distance sampling detection functions are extended to include mixture models in the candidate set.Publisher PDFPeer reviewe

Stability in Ecosystem Functioning across a Climatic Threshold and Contrasting Forest Regimes

Classical ecological theory predicts that changes in the availability of essential resources such as nitrogen should lead to changes in plant community composition due to differences in species-specific nutrient requirements. What remains unknown, however, is the extent to which climate change will alter the relationship between plant communities and the nitrogen cycle. During intervals of climate change, do changes in nitrogen cycling lead to vegetation change or do changes in community composition alter the nitrogen dynamics? We used long-term ecological data to determine the role of nitrogen availability in changes of forest species composition under a rapidly changing climate during the early Holocene (16k to 8k cal. yrs. BP). A statistical computational analysis of ecological data spanning 8,000 years showed that secondary succession from a coniferous to deciduous forest occurred independently of changes in the nitrogen cycle. As oak replaced pine under a warming climate, nitrogen cycling rates increased. Interestingly, the mechanism by which the species interacted with nitrogen remained stable across this threshold change in climate and in the dominant tree species. This suggests that changes in tree population density over successional time scales are not driven by nitrogen availability. Thus, current models of forest succession that incorporate the effects of available nitrogen may be over-estimating tree population responses to changes in this resource, which may result in biased predictions of future forest dynamics under climate warming

Understanding Animal Group-Size Distributions

One of the most striking aspects of animal groups is their remarkable variation in size, both within and between species. While a number of mechanistic models have been proposed to explain this variation, there are few comprehensive datasets against which these models have been tested. In particular, we only vaguely understand how environmental factors and behavioral activities affect group-size distributions. Here we use observations of House sparrows (Passer domesticus) to investigate the factors determining group-size distribution. Over a wide range of conditions, we observed that animal group sizes followed a single parameter distribution known as the logarithmic distribution. This single parameter is the mean group size experienced by a randomly chosen individual (including the individual itself). For sparrows, the experienced mean group size, and hence the distribution, was affected by four factors: morning temperature, place, behavior and the degree of food spillage. Our results further indicate that the sparrows regulate the mean group size they experience, either by groups splitting more or merging less when local densities are high. We suggest that the mean experienced group size provides a simple but general tool for assessing the ecology and evolution of grouping

Early efforts in modeling the incubation period of infectious diseases with an acute course of illness

The incubation period of infectious diseases, the time from infection with a microorganism to onset of disease, is directly relevant to prevention and control. Since explicit models of the incubation period enhance our understanding of the spread of disease, previous classic studies were revisited, focusing on the modeling methods employed and paying particular attention to relatively unknown historical efforts. The earliest study on the incubation period of pandemic influenza was published in 1919, providing estimates of the incubation period of Spanish flu using the daily incidence on ships departing from several ports in Australia. Although the study explicitly dealt with an unknown time of exposure, the assumed periods of exposure, which had an equal probability of infection, were too long, and thus, likely resulted in slight underestimates of the incubation period

Estimation of population size when capture probability depends on individual state

We develop a multi-state model to estimate the size of a closed population from capture–recapture studies. We consider the case where capture–recapture data are not of a simple binary form, but where the state of an individual is also recorded upon every capture as a discrete variable. The proposed multi-state model can be regarded as a generalisation of the commonly applied set of closed population models to a multi-state form. The model allows for heterogeneity within the capture probabilities associated with each state while also permitting individuals to move between the different discrete states. A closed-form expression for the likelihood is presented in terms of a set of sufficient statistics. The link between existing models for capture heterogeneity is established, and simulation is used to show that the estimate of population size can be biased when movement between states is not accounted for. The proposed unconditional approach is also compared to a conditional approach to assess estimation bias. The model derived in this paper is motivated by a real ecological data set on great crested newts, Triturus cristatus. Supplementary materials accompanying this paper appear online