Automated multi-objective calibration of biological agent-based simulations

Computational agent-based simulation (ABS) is increasingly used to complement laboratory techniques in advancing our understanding of biological systems. Calibration, the identification of parameter values that align simulation with biological behaviours, becomes challenging as increasingly complex biological domains are simulated. Complex domains cannot be characterized by single metrics alone, rendering simulation calibration a fundamentally multi-metric optimization problem that typical calibration techniques cannot handle. Yet calibration is an essential activity in simulation-based science; the baseline calibration forms a control for subsequent experimentation and hence is fundamental in the interpretation of results. Here, we develop and showcase a method, built around multi-objective optimization, for calibrating ABSs against complex target behaviours requiring several metrics (termed objectives) to characterize. Multi-objective calibration (MOC) delivers those sets of parameter values representing optimal trade-offs in simulation performance against each metric, in the form of a Pareto front. We use MOC to calibrate a well-understood immunological simulation against both established a priori and previously unestablished target behaviours. Furthermore, we show that simulation-borne conclusions are broadly, but not entirely, robust to adopting baseline parameter values from different extremes of the Pareto front, highlighting the importance of MOC's identification of numerous calibration solutions. We devise a method for detecting overfitting in a multi-objective context, not previously possible, used to save computational effort by terminating MOC when no improved solutions will be found. MOC can significantly impact biological simulation, adding rigour to and speeding up an otherwise time-consuming calibration process and highlighting inappropriate biological capture by simulations that cannot be well calibrated. As such, it produces more accurate simulations that generate more informative biological predictions

Strategies for calibrating models of biology

Computational and mathematical modelling has become a valuable tool for investigating biological systems. Modelling enables prediction of how biological components interact to deliver system-level properties and extrapolation of biological system performance to contexts and experimental conditions where this is unknown. A model's value hinges on knowing that it faithfully represents the biology under the contexts of use, or clearly ascertaining otherwise and thus motivating further model refinement. These qualities are evaluated through calibration, typically formulated as identifying model parameter values that align model and biological behaviours as measured through a metric applied to both. Calibration is critical to modelling but is often underappreciated. A failure to appropriately calibrate risks unrepresentative models that generate erroneous insights. Here, we review a suite of strategies to more rigorously challenge a model's representation of a biological system. All are motivated by features of biological systems, and illustrative examples are drawn from the modelling literature. We examine the calibration of a model against distributions of biological behaviours or outcomes, not only average values. We argue for calibration even where model parameter values are experimentally ascertained. We explore how single metrics can be non-distinguishing for complex systems, with multiple-component dynamic and interaction configurations giving rise to the same metric output. Under these conditions, calibration is insufficiently constraining and the model non-identifiable: multiple solutions to the calibration problem exist. We draw an analogy to curve fitting and argue that calibrating a biological model against a single experiment or context is akin to curve fitting against a single data point. Though useful for communicating model results, we explore how metrics that quantify heavily emergent properties may not be suitable for use in calibration. Lastly, we consider the role of sensitivity and uncertainty analysis in calibration and the interpretation of model results. Our goal in this manuscript is to encourage a deeper consideration of calibration, and how to increase its capacity to either deliver faithful models or demonstrate them otherwise

In silico investigation into dendritic cell regulation of CD8Treg mediated killing of Th1 cells in murine experimental autoimmune encephalomyelitis

Background Experimental autoimmune encephalomyelitis has been used extensively as an animal model of T cell mediated autoimmunity. A down-regulatory pathway through which encephalitogenic CD4Th1 cells are killed by CD8 regulatory T cells (Treg) has recently been proposed. With the CD8Treg cells being primed by dendritic cells, regulation of recovery may be occuring around these antigen presenting cells. CD4Treg cells provide critical help within this process, by licensing dendritic cells to prime CD8Treg cells, however the spatial and temporal aspects of this help in the CTL response is currently unclear. Results We have previously developed a simulator of experimental autoimmune encephalomyelitis (ARTIMMUS). We use ARTIMMUS to perform novel in silico experimentation regarding the priming of CD8Treg cells by dendritic cells, and the resulting CD8Treg mediated killing of encephalitogenic CD4Th1 cells. Simulations using dendritic cells that present antigenic peptides in a mutually exclusive manner (either MBP or TCR-derived, but not both) suggest that there is no significant reliance on dendritic cells that can prime both encephalitogenic CD4Th1 and Treg cells. Further, in silico experimentation suggests that dynamics of CD8Treg priming are significantly influenced through their spatial competition with CD4Treg cells and through the timing of Qa-1 expression by dendritic cells. Conclusion There is no requirement for the encephalitogenic CD4Th1 cells and cytotoxic CD8Treg cells to be primed by the same dendritic cells. We conjecture that no significant portion of CD4Th1 regulation by Qa-1 restricted CD8Treg cells occurs around individual dendritic cells, and as such, that CD8Treg mediated killing of CD4Th1 cells occurring around dendritic cells is not critical for recovery from the murine autoimmune disease. Furthermore, the timing of the CD4Treg licensing of dendritic cells and the spatial competition between CD4Treg and CD8Treg cells around the dendritic cell is critical for the size of the cytotoxic T lymphocyte response, because dendritic cells have a limited lifespan. If treatments can be found to either speed up the licensing process, or increase the spatial competitiveness of CD8Treg cells, the magnitude of the cytotoxic T lymphocyte response can be increased

In silico investigation into dendritic cell regulation of CD8Treg mediated killing of Th1 cells in murine experimental autoimmune encephalomyelitis

Background Experimental autoimmune encephalomyelitis has been used extensively as an animal model of T cell mediated autoimmunity. A down-regulatory pathway through which encephalitogenic CD4Th1 cells are killed by CD8 regulatory T cells (Treg) has recently been proposed. With the CD8Treg cells being primed by dendritic cells, regulation of recovery may be occuring around these antigen presenting cells. CD4Treg cells provide critical help within this process, by licensing dendritic cells to prime CD8Treg cells, however the spatial and temporal aspects of this help in the CTL response is currently unclear. Results We have previously developed a simulator of experimental autoimmune encephalomyelitis (ARTIMMUS). We use ARTIMMUS to perform novel in silico experimentation regarding the priming of CD8Treg cells by dendritic cells, and the resulting CD8Treg mediated killing of encephalitogenic CD4Th1 cells. Simulations using dendritic cells that present antigenic peptides in a mutually exclusive manner (either MBP or TCR-derived, but not both) suggest that there is no significant reliance on dendritic cells that can prime both encephalitogenic CD4Th1 and Treg cells. Further, in silico experimentation suggests that dynamics of CD8Treg priming are significantly influenced through their spatial competition with CD4Treg cells and through the timing of Qa-1 expression by dendritic cells. Conclusion There is no requirement for the encephalitogenic CD4Th1 cells and cytotoxic CD8Treg cells to be primed by the same dendritic cells. We conjecture that no significant portion of CD4Th1 regulation by Qa-1 restricted CD8Treg cells occurs around individual dendritic cells, and as such, that CD8Treg mediated killing of CD4Th1 cells occurring around dendritic cells is not critical for recovery from the murine autoimmune disease. Furthermore, the timing of the CD4Treg licensing of dendritic cells and the spatial competition between CD4Treg and CD8Treg cells around the dendritic cell is critical for the size of the cytotoxic T lymphocyte response, because dendritic cells have a limited lifespan. If treatments can be found to either speed up the licensing process, or increase the spatial competitiveness of CD8Treg cells, the magnitude of the cytotoxic T lymphocyte response can be increased

Using MapReduce Streaming for Distributed Life Simulation on the Cloud

Distributed software simulations are indispensable in the study of large-scale life models but often require the use of technically complex lower-level distributed computing frameworks, such as MPI. We propose to overcome the complexity challenge by applying the emerging MapReduce (MR) model to distributed life simulations and by running such simulations on the cloud. Technically, we design optimized MR streaming algorithms for discrete and continuous versions of Conway’s life according to a general MR streaming pattern. We chose life because it is simple enough as a testbed for MR’s applicability to a-life simulations and general enough to make our results applicable to various lattice-based a-life models. We implement and empirically evaluate our algorithms’ performance on Amazon’s Elastic MR cloud. Our experiments demonstrate that a single MR optimization technique called strip partitioning can reduce the execution time of continuous life simulations by 64%. To the best of our knowledge, we are the first to propose and evaluate MR streaming algorithms for lattice-based simulations. Our algorithms can serve as prototypes in the development of novel MR simulation algorithms for large-scale lattice-based a-life models.https://digitalcommons.chapman.edu/scs_books/1014/thumbnail.jp