Uncertainty Analysis in Population-Based Disease Microsimulation Models

Objective. Uncertainty analysis (UA) is an important part of simulation model validation. However, literature is imprecise as to how UA should be performed in the context of population-based microsimulation (PMS) models. In this expository paper, we discuss a practical approach to UA for such models. Methods. By adapting common concepts from published UA guidelines, we developed a comprehensive, step-by-step approach to UA in PMS models, including sample size calculation to reduce the computational time. As an illustration, we performed UA for POHEM-OA, a microsimulation model of osteoarthritis (OA) in Canada. Results. The resulting sample size of the simulated population was 500,000 and the number of Monte Carlo (MC) runs was 785 for 12-hour computational time. The estimated 95% uncertainty intervals for the prevalence of OA in Canada in 2021 were 0.09 to 0.18 for men and 0.15 to 0.23 for women. The uncertainty surrounding the sex-specific prevalence of OA increased over time. Conclusion. The proposed approach to UA considers the challenges specific to PMS models, such as selection of parameters and calculation of MC runs and population size to reduce computational burden. Our example of UA shows that the proposed approach is feasible. Estimation of uncertainty intervals should become a standard practice in the reporting of results from PMS models

Accounting for heterogeneous invasion rates reveals management impacts on the spatial expansion of an invasive species

Success of large-scale control programs for established invasive species is challenging to evaluate because of spatial variability in expansion rates, management techniques, and the strength of management intensity. For a well-established invasive species in the spreading phase of invasion, a useful metric of impact is the magnitude by which control slows the rate of spatial spread. The prevention of spatial spreading likely results in substantial benefits in terms of ecosystem or economic damage that is prevented by an expanding invasive species. To understand how local management actions could impact the spatial spread of an established invasive species, we analyzed distribution and management data for feral swine across contiguous United States using occupancy analysis. We quantified changes in the rate of spatial expansion of feral swine and its relationship to local management actions. We found that after 4 yr of enhanced control, invasion probability decreased by 8% on average relative to pre-program rates. This decrease was as high as 15% on average in states with low-density populations of feral swine. The amount of decrease in invasion rate was attributed to removal intensity in neighboring counties and depended on the extent of neighboring counties with feral swine (spatial heterogeneity in local invasion pressure). Although we did not find a significant overall increase in the probability of elimination, increased elimination probability tended to occur in regions with low invasion pressure. Accounting for spatial heterogeneity in invasion pressure was important for quantifying management impacts (i.e., the relationship between management intensity and spatial spreading processes) because management impacts changed depending on the strength of invasion pressure from neighboring counties. Predicting reduction in spatial spread of an invasive species is an important first step in valuation of overall damage reduction for invasive species control programs by providing estimates of where a species may be, and thus which natural and agricultural resources would be affected, if the control program had not been operating. For minimizing losses from spatial expansion of an invasive species, our framework can be used for adaptive resource prioritization to areas where spatial expansion and underlying damage potential are concurrently highest

Accounting for heterogeneous invasion rates reveals management impacts on the spatial expansion of an invasive species

Success of large-scale control programs for established invasive species is challenging to evaluate because of spatial variability in expansion rates, management techniques, and the strength of management intensity. For a well-established invasive species in the spreading phase of invasion, a useful metric of impact is the magnitude by which control slows the rate of spatial spread. The prevention of spatial spreading likely results in substantial benefits in terms of ecosystem or economic damage that is prevented by an expanding invasive species. To understand how local management actions could impact the spatial spread of an established invasive species, we analyzed distribution and management data for feral swine across contiguous United States using occupancy analysis. We quantified changes in the rate of spatial expansion of feral swine and its relationship to local management actions. We found that after 4 yr of enhanced control, invasion probability decreased by 8% on average relative to pre-program rates. This decrease was as high as 15% on average in states with low-density populations of feral swine. The amount of decrease in invasion rate was attributed to removal intensity in neighboring counties and depended on the extent of neighboring counties with feral swine (spatial heterogeneity in local invasion pressure). Although we did not find a significant overall increase in the probability of elimination, increased elimination probability tended to occur in regions with low invasion pressure. Accounting for spatial heterogeneity in invasion pressure was important for quantifying management impacts (i.e., the relationship between management intensity and spatial spreading processes) because management impacts changed depending on the strength of invasion pressure from neighboring counties. Predicting reduction in spatial spread of an invasive species is an important first step in valuation of overall damage reduction for invasive species control programs by providing estimates of where a species may be, and thus which natural and agricultural resources would be affected, if the control program had not been operating. For minimizing losses from spatial expansion of an invasive species, our framework can be used for adaptive resource prioritization to areas where spatial expansion and underlying damage potential are concurrently highest

Validation of population-based disease simulation models: a review of concepts and methods

Abstract Background Computer simulation models are used increasingly to support public health research and policy, but questions about their quality persist. The purpose of this article is to review the principles and methods for validation of population-based disease simulation models. Methods We developed a comprehensive framework for validating population-based chronic disease simulation models and used this framework in a review of published model validation guidelines. Based on the review, we formulated a set of recommendations for gathering evidence of model credibility. Results Evidence of model credibility derives from examining: 1) the process of model development, 2) the performance of a model, and 3) the quality of decisions based on the model. Many important issues in model validation are insufficiently addressed by current guidelines. These issues include a detailed evaluation of different data sources, graphical representation of models, computer programming, model calibration, between-model comparisons, sensitivity analysis, and predictive validity. The role of external data in model validation depends on the purpose of the model (e.g., decision analysis versus prediction). More research is needed on the methods of comparing the quality of decisions based on different models. Conclusion As the role of simulation modeling in population health is increasing and models are becoming more complex, there is a need for further improvements in model validation methodology and common standards for evaluating model credibility

Potential of polygenic risk scores for improving population estimates of women’s breast cancer genetic risks

Funder: Genome Canada; doi: https://doi.org/10.13039/http://dx.doi.org/10.13039/100008762Abstract: Purpose: Breast cancer risk has conventionally been assessed using family history (FH) and rare high/moderate penetrance pathogenic variants (PVs), notably in BRCA1/2, and more recently PALB2, CHEK2, and ATM. In addition to these PVs, it is now possible to use increasingly predictive polygenic risk scores (PRS) as well. The comparative population-level predictive capability of these three different indicators of genetic risk for risk stratification is, however, unknown. Methods: The Canadian heritable breast cancer risk distribution was estimated using a novel genetic mixing model (GMM). A realistically representative sample of women was synthesized based on empirically observed demographic patterns for appropriately correlated family history, inheritance of rare PVs, PRS, and residual risk from an unknown polygenotype. Risk assessment was simulated using the BOADICEA risk algorithm for 10-year absolute breast cancer incidence, and compared to heritable risks as if the overall polygene, including its measured PRS component, and PV risks were fully known. Results: Generally, the PRS was most predictive for identifying women at high risk, while family history was the weakest. Only the PRS identified any women at low risk of breast cancer. Conclusion: PRS information would be the most important advance in enabling effective risk stratification for population-wide breast cancer screening

Therapeutic exploitation of IPSE, a urogenital parasite-derived host modulatory protein, for chemotherapy-induced hemorrhagic cystitis

Chemotherapy-induced hemorrhagic cystitis (CHC) can be difficult to manage. Prior work suggests IL-4 alleviates ifosfamide-induced hemorrhagic cystitis (IHC), but systemically administered IL-4 causes significant side effects. We hypothesized that the Schistosoma haematobium homolog of Interleukin-4-inducing principle from Schistosoma mansoni Eggs (H-IPSE), would reduce IHC and associated bladder pathology. IPSE binds IgE on basophils and mast cells, triggering IL-4 secretion by these cells. IPSE is also an “infiltrin”, translocating into the host nucleus to modulate gene transcription. Mice were administered IL-4, H-IPSE protein or its nuclear localization sequence (NLS) mutant with or without neutralizing anti-IL-4 antibody, or MESNA, followed by ifosfamide. Bladder tissue damage and hemoglobin content were measured. Spontaneous and evoked pain, urinary frequency and gene expression were assessed. Pain behaviors were interpreted in a blinded fashion. One dose of H-IPSE was superior to MESNA and IL-4 in suppressing bladder hemorrhage in an IL-4-and NLS-dependent fashion, and comparable to MESNA in dampening ifosfamide-triggered pain behaviors in an NLS-dependent manner. H-IPSE also accelerated urothelial repair following IHC. Our work represents the first therapeutic exploitation of a uropathogen-derived host modulatory molecule in a clinically relevant bladder disease model, and indicates that IPSE may be an alternative to MESNA for mitigating CHC

Improving the accessibility and transferability of machine learning algorithms for identification of animals in camera trap images: MLWIC2

Motion-activated wildlife cameras (or “camera traps”) are frequently used to remotely and noninvasively observe animals. The vast number of images collected from camera trap projects has prompted some biologists to employ machine learning algorithms to automatically recognize species in these images, or at least filter-out images that do not contain animals. These approaches are often limited by model transferability, as a model trained to recognize species from one location might not work as well for the same species in different locations. Furthermore, these methods often require advanced computational skills, making them inaccessible to many biologists. We used 3 million camera trap images from 18 studies in 10 states across the United States of America to train two deep neural networks, one that recognizes 58 species, the “species model,” and one that determines if an image is empty or if it contains an animal, the “empty-animal model.” Our species model and empty-animal model had accuracies of 96.8% and 97.3%, respectively. Furthermore, the models performed well on some out-of-sample datasets, as the species model had 91% accuracy on species from Canada (accuracy range 36%–91% across all out-of-sample datasets) and the empty-animal model achieved an accuracy of 91%–94% on out-of-sample datasets from different continents. Our software addresses some of the limitations of using machine learning to classify images from camera traps. By including many species from several locations, our species model is potentially applicable to many camera trap studies in North America. We also found that our empty-animal model can facilitate removal of images without animals globally. We provide the trained models in an R package (MLWIC2: Machine Learning for Wildlife Image Classification in R), which contains Shiny Applications that allow scientists with minimal programming experience to use trained models and train new models in six neural network architectures with varying depths