Calibration of individual-based models to epidemiological data : a systematic review

Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of calibration methods used in IBMs modelling infectious disease spread. We identified articles on PubMed employing simulation-based methods to calibrate IBMs informing public health policy in HIV, tuberculosis, and malaria epidemiology published between 1 January 2013 and 31 December 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies, GOF measures, and model validation. The PubMed search identified 653 candidate articles, of which 84 met the review criteria. Of the included articles, 40 (48%) combined a quantitative GOF measure with an algorithmic parameter-search strategy–either an optimisation algorithm (14/40) or a sampling algorithm (26/40). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 (52%) articles, the parameter-search strategy could either not be identified (32/44) or was described as an informal, non-reproducible method (12/44). Of these 44 articles, the majority (25/44) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Only a minority of the included articles, 14 (17%) provided a rationale for their choice of model-calibration method. Model validation was reported in 31 (37%) articles. Reporting on calibration methods is far from optimal in epidemiological modelling studies of HIV, malaria and TB transmission dynamics. The adoption of better documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. There is a need for research comparing the performance of calibration methods to inform decisions about the parameter-search strategies and GOF measures

Calibration of individual-based models to epidemiological data: A systematic review.

Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of calibration methods used in IBMs modelling infectious disease spread. We identified articles on PubMed employing simulation-based methods to calibrate IBMs informing public health policy in HIV, tuberculosis, and malaria epidemiology published between 1 January 2013 and 31 December 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies, GOF measures, and model validation. The PubMed search identified 653 candidate articles, of which 84 met the review criteria. Of the included articles, 40 (48%) combined a quantitative GOF measure with an algorithmic parameter-search strategy-either an optimisation algorithm (14/40) or a sampling algorithm (26/40). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 (52%) articles, the parameter-search strategy could either not be identified (32/44) or was described as an informal, non-reproducible method (12/44). Of these 44 articles, the majority (25/44) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Only a minority of the included articles, 14 (17%) provided a rationale for their choice of model-calibration method. Model validation was reported in 31 (37%) articles. Reporting on calibration methods is far from optimal in epidemiological modelling studies of HIV, malaria and TB transmission dynamics. The adoption of better documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. There is a need for research comparing the performance of calibration methods to inform decisions about the parameter-search strategies and GOF measures

African based researchers' output on models for the transmission dynamics of infectious diseases and public health interventions: A scoping review.

BackgroundApplied epidemiological models are used in predicting future trends of diseases, for the basic understanding of disease and health dynamics, and to improve the measurement of health indicators. Mapping the research outputs of epidemiological modelling studies concerned with transmission dynamics of infectious diseases and public health interventions in Africa will help to identify the areas with substantial levels of research activities, areas with gaps, and research output trends.MethodsA scoping review of applied epidemiological models of infectious disease studies that involved first or last authors affiliated to African institutions was conducted. Eligible studies were those concerned with the transmission dynamics of infectious diseases and public health interventions. The review was consistent with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) extension for scoping reviews. Four electronic databases were searched for peer-reviewed publications up to the end of April 2020.ResultsOf the 5927 publications identified, 181 met the inclusion criteria. The review identified 143 publications with first authors having an African institutional affiliation (AIA), while 81 had both first and last authors with an AIA. The publication authors were found to be predominantly affiliated with institutions based in South Africa and Kenya. Furthermore, human immunodeficiency virus, malaria, tuberculosis, and Ebola virus disease were found to be the most researched infectious diseases. There has been a gradual increase in research productivity across Africa especially in the last ten years, with several collaborative efforts spread both within and beyond Africa.ConclusionsResearch productivity in applied epidemiological modelling studies of infectious diseases may have increased, but there remains an under-representation of African researchers as leading authors. The study findings indicate a need for the development of research capacity through supporting existing institutions in Africa and promoting research funding that will address local health priorities

The Impact of Rubella Vaccine Introduction on Rubella Infection and Congenital Rubella Syndrome: A Systematic Review of Mathematical Modelling Studies

Introduction: Rubella vaccines have been used to prevent rubella and congenital rubella syndrome (CRS) in several World Health Organization (WHO) regions. Mathematical modelling studies have simulated introduction of rubella-containing vaccines (RCVs), and their results have been used to inform rubella introduction strategies in several countries. This systematic review aimed to synthesize the evidence from mathematical models regarding the impact of introducing RCVs. Methods: We registered the review in the international prospective register of systematic reviews (PROSPERO) with registration number CRD42020192638. Systematic review methods for classical epidemiological studies and reporting guidelines were followed as far as possible. A comprehensive search strategy was used to identify published and unpublished studies with no language restrictions. We included deterministic and stochastic models that simulated RCV introduction into the public sector vaccination schedule, with a time horizon of at least five years. Models focused only on estimating epidemiological parameters were excluded. Outcomes of interest were time to rubella and CRS elimination, trends in incidence of rubella and CRS, number of vaccinated individuals per CRS case averted, and cost-effectiveness of vaccine introduction strategies. The methodological quality of included studies was assessed using a modified risk of bias tool, and a qualitative narrative was provided, given that data synthesis was not feasible. Results: Seven studies were included from a total of 1393 records retrieved. The methodological quality was scored high for six studies and very high for one study. Quantitative data synthesis was not possible, because only one study reported point estimates and uncertainty intervals for the outcomes. All seven included studies presented trends in rubella incidence, six studies reported trends in CRS incidence, two studies reported the number vaccinated individuals per CRS case averted, and two studies reported an economic evaluation measure. Time to CRS elimination and time to rubella elimination were not reported by any of the included studies. Reported trends in CRS incidence showed elimination within five years of RCV introduction with scenarios involving mass vaccination of older children in addition to routine infant vaccination. CRS incidence was higher with RCV introduction than without RCV when public vaccine coverage was lower than 50% or only private sector vaccination was implemented. Although vaccination of children at a given age achieved slower declines in CRS incidence compared to mass campaigns targeting a wide age range, this approach resulted in the lowest number of vaccinated individuals per CRS case averted. Conclusion and recommendations: We were unable to conduct data synthesis of included studies due to discrepancies in outcome reporting. However, qualitative assessment of results of individual studies suggests that vaccination of infants should be combined with vaccination of older children to achieve rapid elimination of CRS. Better outcomes are obtained when rubella vaccination is introduced into public vaccination schedules at coverage figures of 80%, as recommended by WHO, or higher. Guidelines for reporting of outcomes in mathematical modelling studies and the conduct of systematic reviews of mathematical modelling studies are required

The Impact of Rubella Vaccine Introduction on Rubella Infection and Congenital Rubella Syndrome: A Systematic Review of Mathematical Modelling Studies

Introduction: Rubella vaccines have been used to prevent rubella and congenital rubella syndrome (CRS) in several World Health Organization (WHO) regions. Mathematical modelling studies have simulated introduction of rubella-containing vaccines (RCVs), and their results have been used to inform rubella introduction strategies in several countries. This systematic review aimed to synthesize the evidence from mathematical models regarding the impact of introducing RCVs. Methods: We registered the review in the international prospective register of systematic reviews (PROSPERO) with registration number CRD42020192638. Systematic review methods for classical epidemiological studies and reporting guidelines were followed as far as possible. A comprehensive search strategy was used to identify published and unpublished studies with no language restrictions. We included deterministic and stochastic models that simulated RCV introduction into the public sector vaccination schedule, with a time horizon of at least five years. Models focused only on estimating epidemiological parameters were excluded. Outcomes of interest were time to rubella and CRS elimination, trends in incidence of rubella and CRS, number of vaccinated individuals per CRS case averted, and cost-effectiveness of vaccine introduction strategies. The methodological quality of included studies was assessed using a modified risk of bias tool, and a qualitative narrative was provided, given that data synthesis was not feasible. Results: Seven studies were included from a total of 1393 records retrieved. The methodological quality was scored high for six studies and very high for one study. Quantitative data synthesis was not possible, because only one study reported point estimates and uncertainty intervals for the outcomes. All seven included studies presented trends in rubella incidence, six studies reported trends in CRS incidence, two studies reported the number vaccinated individuals per CRS case averted, and two studies reported an economic evaluation measure. Time to CRS elimination and time to rubella elimination were not reported by any of the included studies. Reported trends in CRS incidence showed elimination within five years of RCV introduction with scenarios involving mass vaccination of older children in addition to routine infant vaccination. CRS incidence was higher with RCV introduction than without RCV when public vaccine coverage was lower than 50% or only private sector vaccination was implemented. Although vaccination of children at a given age achieved slower declines in CRS incidence compared to mass campaigns targeting a wide age range, this approach resulted in the lowest number of vaccinated individuals per CRS case averted. Conclusion and recommendations: We were unable to conduct data synthesis of included studies due to discrepancies in outcome reporting. However, qualitative assessment of results of individual studies suggests that vaccination of infants should be combined with vaccination of older children to achieve rapid elimination of CRS. Better outcomes are obtained when rubella vaccination is introduced into public vaccination schedules at coverage figures of 80%, as recommended by WHO, or higher. Guidelines for reporting of outcomes in mathematical modelling studies and the conduct of systematic reviews of mathematical modelling studies are required