Using Emulation to Engineer and Understand Simulations of Biological Systems

Modeling and simulation techniques have demonstrated success in studying biological systems. As the drive to better capture biological complexity leads to more sophisticated simulators, it becomes challenging to perform statistical analyses that help translate predictions into increased understanding. These analyses may require repeated executions and extensive sampling of high-dimensional parameter spaces: analyses that may become intractable due to time and resource limitations. Significant reduction in these requirements can be obtained using surrogate models, or emulators, that can rapidly and accurately predict the output of an existing simulator. We apply emulation to evaluate and enrich understanding of a previously published agent-based simulator of lymphoid tissue organogenesis, showing an ensemble of machine learning techniques can reproduce results obtained using a suite of statistical analyses within seconds. This performance improvement permits incorporation of previously intractable analyses, including multi-objective optimization to obtain parameter sets that yield a desired response, and Approximate Bayesian Computation to assess parametric uncertainty. To facilitate exploitation of emulation in simulation-focused studies, we extend our open source statistical package, spartan, to provide a suite of tools for emulator development, validation, and application. Overcoming resource limitations permits enriched evaluation and refinement, easing translation of simulator insights into increased biological understanding

Utilising a simulation platform to understand the effect of domain model assumptions

© The Author(s) 2014. This article is published with open access at Springerlink.com. Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.Computational and mathematical modelling approaches are increasingly being adopted in attempts to further our understanding of complex biological systems. This approach can be subjected to strong criticism as substantial aspects of the biological system being captured are not currently known, meaning assumptions need to be made that could have a critical impact on simulation response. We have utilised the CoSMoS process in the development of an agent-based simulation of the formation of Peyer's patches (PP), gut-associated lymphoid organs that have a key role in the initiation of adaptive immune responses to infection. Although the use of genetic tools, imaging technologies and ex vivo culture systems has provided significant insight into the cellular components and associated pathways involved in PP development, interesting questions remain that cannot be addressed using these approaches, and as such well justified assumptions have been introduced into our model to counter this. Here we focus not on the development of the model itself, but instead demonstrate how the resultant simulation can be used to assess how these assumptions impact the simulation response. For example, we consider the impact of our assumption that the migration rate of lymphoid tissue cells into the gut remains constant throughout PP development. We demonstrate that an analysis of the assumptions made in the construction of the domain model may either increase confidence in the model as a representation of the biological system it captures, or may suggest areas where further biological experimentation is required.This work was funded by the Wellcome Trust [ref:097829] through the Centre for Chronic Diseases and Disorders (C2D2) at the University of York. Paul Andrews is funded by EPSRC grant EP/I005943/1 “Resilient Futures.” Henrique Veiga-Fernandes is funded by Fundação para a Ciência e Tecnologia (PTDC/SAU-MII/100016/2008), Portugal, European Molecular Biology Organisation (Project 1648) and European Research Council (Project 207057). Jon Timmis is part funded by the Royal Society and the Royal Academy of Engineering. Funding for Mark Coles comes from grants from the Human Frontiers Science Program (RGP0006/2009) and the Medical Research Council (G0601156).info:eu-repo/semantics/publishedVersio

Strategies for calibrating models of biology

Computational and mathematical modelling has become a valuable tool for investigating biological systems. Modelling enables prediction of how biological components interact to deliver system-level properties and extrapolation of biological system performance to contexts and experimental conditions where this is unknown. A model's value hinges on knowing that it faithfully represents the biology under the contexts of use, or clearly ascertaining otherwise and thus motivating further model refinement. These qualities are evaluated through calibration, typically formulated as identifying model parameter values that align model and biological behaviours as measured through a metric applied to both. Calibration is critical to modelling but is often underappreciated. A failure to appropriately calibrate risks unrepresentative models that generate erroneous insights. Here, we review a suite of strategies to more rigorously challenge a model's representation of a biological system. All are motivated by features of biological systems, and illustrative examples are drawn from the modelling literature. We examine the calibration of a model against distributions of biological behaviours or outcomes, not only average values. We argue for calibration even where model parameter values are experimentally ascertained. We explore how single metrics can be non-distinguishing for complex systems, with multiple-component dynamic and interaction configurations giving rise to the same metric output. Under these conditions, calibration is insufficiently constraining and the model non-identifiable: multiple solutions to the calibration problem exist. We draw an analogy to curve fitting and argue that calibrating a biological model against a single experiment or context is akin to curve fitting against a single data point. Though useful for communicating model results, we explore how metrics that quantify heavily emergent properties may not be suitable for use in calibration. Lastly, we consider the role of sensitivity and uncertainty analysis in calibration and the interpretation of model results. Our goal in this manuscript is to encourage a deeper consideration of calibration, and how to increase its capacity to either deliver faithful models or demonstrate them otherwise

Applying spartan to Understand Parameter Uncertainty in Simulations

Abstract In attempts to further understand the dynamics of complex systems, the application of computer simulation is becoming increasingly prevalent. Whereas a great deal of focus has been placed in the development of software tools that aid researchers develop simulations, similar focus has not been applied in the creation of tools that perform a rigorous statistical analysis of results generated through simulation: vital in understanding how these results offer an insight into the captured system. This encouraged us to develop spartan, a package of statistical techniques designed to assist researchers in understanding the relationship between their simulation and the real system. Previously we have described each technique within spartan in detail, with an accompanying immunology case study examining the development of lymphoid tissue. Here we provide a practical introduction to the package, demonstrating how each technique is run in R, to assist researchers in integrating this package alongside their chosen simulation platform

A study of error diversity in robotic swarms for task partitioning in foraging tasks

Often in swarm robotics, an assumption is made that all robots in the swarm behave the same and will have a similar (if not the same) error model. However, in reality this is not the case and this lack of uniformity in the error model, and other operations, can lead to various emergent behaviours. This paper considers the impact of the error model and compares robots in a swarm that operate using the same error model (uniform error) against each robot in the swarm having a different error model (thus introducing error diversity). Experiments are presented in the context of a foraging task. Simulation and physical experimental results show the importance of the error model and diversity in achieving expected swarm behaviour

Automated multi-objective calibration of biological agent-based simulations

Computational agent-based simulation (ABS) is increasingly used to complement laboratory techniques in advancing our understanding of biological systems. Calibration, the identification of parameter values that align simulation with biological behaviours, becomes challenging as increasingly complex biological domains are simulated. Complex domains cannot be characterized by single metrics alone, rendering simulation calibration a fundamentally multi-metric optimization problem that typical calibration techniques cannot handle. Yet calibration is an essential activity in simulation-based science; the baseline calibration forms a control for subsequent experimentation and hence is fundamental in the interpretation of results. Here, we develop and showcase a method, built around multi-objective optimization, for calibrating ABSs against complex target behaviours requiring several metrics (termed objectives) to characterize. Multi-objective calibration (MOC) delivers those sets of parameter values representing optimal trade-offs in simulation performance against each metric, in the form of a Pareto front. We use MOC to calibrate a well-understood immunological simulation against both established a priori and previously unestablished target behaviours. Furthermore, we show that simulation-borne conclusions are broadly, but not entirely, robust to adopting baseline parameter values from different extremes of the Pareto front, highlighting the importance of MOC's identification of numerous calibration solutions. We devise a method for detecting overfitting in a multi-objective context, not previously possible, used to save computational effort by terminating MOC when no improved solutions will be found. MOC can significantly impact biological simulation, adding rigour to and speeding up an otherwise time-consuming calibration process and highlighting inappropriate biological capture by simulations that cannot be well calibrated. As such, it produces more accurate simulations that generate more informative biological predictions

Extending and Applying Spartan to Perform Temporal Sensitivity Analyses for Predicting Changes in Influential Biological Pathways in Computational Models

Through integrating real time imaging, computational modelling, and statistical analysis approaches, previous work has suggested that the induction of and response to cell adhesion factors is the key initiating pathway in early lymphoid tissue development, in contrast to the previously accepted view that the process is triggered by chemokine mediated cell recruitment. These model derived hypotheses were developed using spartan, an open-source sensitivity analysis toolkit designed to establish and understand the relationship between a computational model and the biological system that model captures. Here, we extend the functionality available in spartan to permit the production of statistical analyses that contrast the behavior exhibited by a computational model at various simulated time-points, enabling a temporal analysis that could suggest whether the influence of biological mechanisms changes over time. We exemplify this extended functionality by using the computational model of lymphoid tissue development as a time-lapse tool. By generating results at twelve- hour intervals, we show how the extensions to spartan have been used to suggest that lymphoid tissue development could be biphasic, and predict the time-point when a switch in the influence of biological mechanisms might occur

Is it selfish to be filamentous in biofilms? Individual-based modeling links microbial growth strategies with morphology using the new and modular iDynoMiCS 2.0

Microbial communities are found in all habitable environments and often occur in assemblages with self-organized spatial structures developing over time. This complexity can only be understood, predicted, and managed by combining experiments with mathematical modeling. Individual-based models are particularly suited if individual heterogeneity, local interactions, and adaptive behavior are of interest. Here we present the completely overhauled software platform, the individual-based Dynamics of Microbial Communities Simulator, iDynoMiCS 2.0, which enables researchers to specify a range of different models without having to program. Key new features and improvements are: (1) Substantially enhanced ease of use (graphical user interface, editor for model specification, unit conversions, data analysis and visualization and more). (2) Increased performance and scalability enabling simulations of up to 10 million agents in 3D biofilms. (3) Kinetics can be specified with any arithmetic function. (4) Agent properties can be assembled from orthogonal modules for pick and mix flexibility. (5) Force-based mechanical interaction framework enabling attractive forces and non-spherical agent morphologies as an alternative to the shoving algorithm. The new iDynoMiCS 2.0 has undergone intensive testing, from unit tests to a suite of increasingly complex numerical tests and the standard Benchmark 3 based on nitrifying biofilms. A second test case was based on the “biofilms promote altruism” study previously implemented in BacSim because competition outcomes are highly sensitive to the developing spatial structures due to positive feedback between cooperative individuals. We extended this case study by adding morphology to find that (i) filamentous bacteria outcompete spherical bacteria regardless of growth strategy and (ii) non-cooperating filaments outcompete cooperating filaments because filaments can escape the stronger competition between themselves. In conclusion, the new substantially improved iDynoMiCS 2.0 joins a growing number of platforms for individual-based modeling of microbial communities with specific advantages and disadvantages that we discuss, giving users a wider choice

Simulating CXCR5 Dynamics in Complex Tissue Microenvironments

To effectively navigate complex tissue microenvironments, immune cells sense molecular concentration gradients using G-protein coupled receptors. However, due to the complexity of receptor activity, and the multimodal nature of chemokine gradients in vivo, chemokine receptor activity in situ is poorly understood. To address this issue, we apply a modelling and simulation approach that permits analysis of the spatiotemporal dynamics of CXCR5 expression within an in silico B-follicle with single-cell resolution. Using this approach, we show that that in silico B-cell scanning is robust to changes in receptor numbers and changes in individual kinetic rates of receptor activity, but sensitive to global perturbations where multiple parameters are altered simultaneously. Through multi-objective optimization analysis we find that the rapid modulation of CXCR5 activity through receptor binding, desensitization and recycling is required for optimal antigen scanning rates. From these analyses we predict that chemokine receptor signaling dynamics regulate migration in complex tissue microenvironments to a greater extent than the total numbers of receptors on the cell surface

Using argument notation to engineer biological simulations with increased confidence

The application of computational and mathematical modelling to explore the mechanics of biological systems is becoming prevalent. To significantly impact biological research, notably in developing novel therapeutics, it is critical that the model adequately represents the captured system. Confidence in adopting in silico approaches can be improved by applying a structured argumentation approach, alongside model development and results analysis. We propose an approach based on argumentation from safety-critical systems engineering, where a system is subjected to a stringent analysis of compliance against identified criteria. We show its use in examining the biological information upon which a model is based, identifying model strengths, highlighting areas requiring additional biological experimentation and providing documentation to support model publication. We demonstrate our use of structured argumentation in the development of a model of lymphoid tissue formation, specifically Peyer's Patches. The argumentation structure is captured using Artoo (www.york.ac.uk/ycil/software/artoo), our Web-based tool for constructing fitness-for-purpose arguments, using a notation based on the safety-critical goal structuring notation. We show how argumentation helps in making the design and structured analysis of a model transparent, capturing the reasoning behind the inclusion or exclusion of each biological feature and recording assumptions, as well as pointing to evidence supporting model-derived conclusions