Taking error into account when fitting models using approximate Bayesian computation

Stochastic computer simulations are often the only practical way of answering questions relating to ecological management. However, due to their complexity, such models are difficult to calibrate and evaluate. Approximate Bayesian Computation (ABC) offers an increasingly popular approach to this problem, widely applied across a variety of fields. However, ensuring the accuracy of ABC's estimates has been difficult. Here, we obtain more accurate estimates by incorporating estimation of error into the ABC protocol. We show how this can be done where the data consist of repeated measures of the same quantity and errors may be assumed to be normally distributed and independent. We then derive the correct acceptance probabilities for a probabilistic ABC algorithm, and update the 'coverage test' with which accuracy is assessed. We apply this method - which we call 'error-calibrated ABC' - to a toy example and a realistic 14-parameter simulation model of earthworms that is used in environmental risk assessment. A comparison with exact methods and the diagnostic 'coverage test' show that our approach improves estimation of parameter values and their credible intervals for both models

Calibration and evaluation of individual-based models using Approximate Bayesian Computation

AbstractThis paper investigates the feasibility of using Approximate Bayesian Computation (ABC) to calibrate and evaluate complex individual-based models (IBMs). As ABC evolves, various versions are emerging, but here we only explore the most accessible version, rejection-ABC. Rejection-ABC involves running models a large number of times, with parameters drawn randomly from their prior distributions, and then retaining the simulations closest to the observations. Although well-established in some fields, whether ABC will work with ecological IBMs is still uncertain.Rejection-ABC was applied to an existing 14-parameter earthworm energy budget IBM for which the available data consist of body mass growth and cocoon production in four experiments. ABC was able to narrow the posterior distributions of seven parameters, estimating credible intervals for each. ABC's accepted values produced slightly better fits than literature values do. The accuracy of the analysis was assessed using cross-validation and coverage, currently the best-available tests. Of the seven unnarrowed parameters, ABC revealed that three were correlated with other parameters, while the remaining four were found to be not estimable given the data available.It is often desirable to compare models to see whether all component modules are necessary. Here, we used ABC model selection to compare the full model with a simplified version which removed the earthworm's movement and much of the energy budget. We are able to show that inclusion of the energy budget is necessary for a good fit to the data. We show how our methodology can inform future modelling cycles, and briefly discuss how more advanced versions of ABC may be applicable to IBMs. We conclude that ABC has the potential to represent uncertainty in model structure, parameters and predictions, and to embed the often complex process of optimising an IBM's structure and parameters within an established statistical framework, thereby making the process more transparent and objective

Predicting how many animals will be where: how to build, calibrate and evaluate individual-based models

Individual-based models (IBMs) can simulate the actions of individual animals as they interact with one another and the landscape in which they live. When used in spatially-explicit landscapes IBMs can show how populations change over time in response to management actions. For instance, IBMs are being used to design strategies of conservation and of the exploitation of fisheries, and for assessing the effects on populations of major construction projects and of novel agricultural chemicals. In such real world contexts, it becomes especially important to build IBMs in a principled fashion, and to approach calibration and evaluation systematically. We argue that insights from physiological and behavioural ecology offer a recipe for building realistic models, and that Approximate Bayesian Computation (ABC) is a promising technique for the calibration and evaluation of IBMs. IBMs are constructed primarily from knowledge about individuals. In ecological applications the relevant knowledge is found in physiological and behavioural ecology, and we approach these from an evolutionary perspective by taking into account how physiological and behavioural processes contribute to life histories, and how those life histories evolve. Evolutionary life history theory shows that, other things being equal, organisms should grow to sexual maturity as fast as possible, and then reproduce as fast as possible, while minimising per capita death rate. Physiological and behavioural ecology are largely built on these principles together with the laws of conservation of matter and energy. To complete construction of an IBM information is also needed on the effects of competitors, conspecifics and food scarcity; the maximum rates of ingestion, growth and reproduction, and life-history parameters. Using this knowledge about physiological and behavioural processes provides a principled way to build IBMs, but model parameters vary between species and are often difficult to measure. A common solution is to manually compare model outputs with observations from real landscapes and so to obtain parameters which produce acceptable fits of model to data. However, this procedure can be convoluted and lead to over-calibrated and thus inflexible models. Many formal statistical techniques are unsuitable for use with IBMs, but we argue that ABC offers a potential way forward. It can be used to calibrate and compare complex stochastic models and to assess the uncertainty in their predictions. We describe methods used to implement ABC in an accessible way and illustrate them with examples and discussion of recent studies. Although much progress has been made, theoretical issues remain, and some of these are outlined and discussed

Syndromic approach to arboviral diagnostics for global travelers as a basis for infectious disease surveillance

Background Arboviruses have overlapping geographical distributions and can cause symptoms that coincide with more common infections. Therefore, arbovirus infections are often neglected by travel diagnostics. Here, we assessed the potential of syndrome-based approaches for diagnosis and surveillance of neglected arboviral diseases in returning travelers. Method To map the patients high at risk of missed clinical arboviral infections we compared the quantity of all arboviral diagnostic requests by physicians in the Netherlands, from 2009 through 2013, with a literature-based assessment of the travelers’ likely exposure to an arbovirus. Results 2153 patients, with travel and clinical history were evaluated. The diagnostic assay for dengue virus (DENV) was the most commonly requested (86%). Of travelers returning from Southeast Asia with symptoms compatible with chikungunya virus (CHIKV), only 55% were tested. For travelers in Europe, arbovirus diagnostics were rarely requested. Over all, diagnostics for most arboviruses were requested only on severe clinical presentation. Conclusion Travel destination and syndrome were used inconsistently for triage of diagnostics, likely resulting in vast under-diagnosis of arboviral infections of public health significance. This study shows the need for more awareness among physicians and standardization of syndromic diagnostic algorithm

Corvid Re-Caching without ‘Theory of Mind’: A Model

Scrub jays are thought to use many tactics to protect their caches. For instance, they predominantly bury food far away from conspecifics, and if they must cache while being watched, they often re-cache their worms later, once they are in private. Two explanations have been offered for such observations, and they are intensely debated. First, the birds may reason about their competitors' mental states, with a ‘theory of mind’; alternatively, they may apply behavioral rules learned in daily life. Although this second hypothesis is cognitively simpler, it does seem to require a different, ad-hoc behavioral rule for every caching and re-caching pattern exhibited by the birds. Our new theory avoids this drawback by explaining a large variety of patterns as side-effects of stress and the resulting memory errors. Inspired by experimental data, we assume that re-caching is not motivated by a deliberate effort to safeguard specific caches from theft, but by a general desire to cache more. This desire is brought on by stress, which is determined by the presence and dominance of onlookers, and by unsuccessful recovery attempts. We study this theory in two experiments similar to those done with real birds with a kind of ‘virtual bird’, whose behavior depends on a set of basic assumptions about corvid cognition, and a well-established model of human memory. Our results show that the ‘virtual bird’ acts as the real birds did; its re-caching reflects whether it has been watched, how dominant its onlooker was, and how close to that onlooker it has cached. This happens even though it cannot attribute mental states, and it has only a single behavioral rule assumed to be previously learned. Thus, our simulations indicate that corvid re-caching can be explained without sophisticated social cognition. Given our specific predictions, our theory can easily be tested empirically

A cognitive model of caching by corvids

Elske van der Vaart concludeert uit haar onderzoek dat het nog te vroeg is om kraaiachtige vogels ‘theory of mind’ toe te dichten. Kraaien en hun familieleden, zoals raven, gaaien en eksters, hebben verassend grote hersenen en vertonen opzienbarend gedrag. Zo is van één soort bekend dat hij gereedschap maakt en van een andere dat hij zichzelf herkent in de spiegel. Ook qua sociale intelligentie lijken deze vogels bijzonder. Zo wordt voor westelijke struikgaaien wel gespeculeerd dat ze over een ‘theory of mind’ beschikken, het vermogen om na te denken over de mentale toestanden van anderen. Dit idee komt voort uit het gedrag dat struikgaaien vertonen bij het verstoppen van voedsel. Ze lijken heel tactisch te anticiperen op toekomstig diefstal, bijvoorbeeld door hun eten later te verplaatsen als anderen hebben meegekeken bij het verstoppen. De vraag blijft echter of dit gedrag even slim is als het eruit ziet. Deze vraag onderzocht Van der Vaart met een nieuwe methode, namelijk een computationeel cognitief model. In de psychologie wordt deze methode al veel toegepast, maar voor dieronderzoek wordt hij nog weinig gebruikt. Voor het onderzoek werd een soort ‘virtuele vogel’ gebouwd, uitgaande van een aantal basisprincipes over leren en geheugen. Vervolgens wordt deze computervogel gebruikt om bestaande experimenten na te bootsen en zo te onderzoeken welke aannames leiden tot hetzelfde gedrag als voor de echte vogels gevonden is. Hieruit blijkt dat verschillende experimenten op een cognitief eenvoudiger manier verklaard kunnen worden dan tot nu toe gebruikelijk is. Daarom is het nog te vroeg om kraaiachtige vogels ‘theory of mind’ toe te dichten. “Er is veel discussie tussen diercognitieonderzoekers over hoe ‘intelligent’ verschillende resultaten geïnterpreteerd moeten worden, maar die discussie zit vaak een beetje vast, omdat de claims van verschillende partijen vaak lastig toetsbaar zijn,” zegt Vander Vaart. “Mijn onderzoek demonstreert hoe computationele modellen bij kunnen dragen aan deze discussie: ze dwingen af dat theorieën heel precies gespecificeerd zijn en ze helpen bij het analyseren van data en het bedenken van alternatieve verklaringen.” Elske van der Vaart (Delft, 1984) studeerde behavioral and cognitive neurosciences aan de Universiteit van Amsterdam. Haar promotieonderzoek deed zij aan de RUG, bij Kunstmatige Intelligentie (KI) en bij Gedragsecologie & Zelf-Organisatie (BESO), allebei aan de Faculteit Wiskunde en Natuurwetenschappen. Haar onderzoek sluit aan bij het Vici-onderzoek naar ‘theory of mind’ van prof.dr. Rineke Verbrugge (KI) en het onderzoek naar zelf-organisatie in sociale systemen van prof.dr. Charlotte Hemelrijk (BESO), maar is een onafhankelijk project, gefinancierd door een eigen TopTalent beurs van NWO. Van der Vaart werkt inmiddels aan de Universiteit van Amsterdam, als docent academische vaardigheden voor psychobiologen. Naar aanleiding van een artikel in Science is door de RUG een persbericht uitgebracht, getiteld Verstopgedrag struikgaai: meer stress dan slimheid?

Error-Calibrated ABC

<div><p>This repository includes all of the code and results described in the following paper:</p><p><br></p><p>van der Vaart, E., Prangle, D. & Sibly, R.M. "Taking Error Into Account When Fitting Models with Approximate Bayesian Computation" (2018) Ecological Applications</p></div><div><div></div></div