Bias due to differential and non-differential disease- and exposure misclassification in studies of vaccine effectiveness

Background Studies of vaccine effectiveness (VE) rely on accurate identification of vaccination and cases of vaccine-preventable disease. In practice, diagnostic tests, clinical case definitions and vaccination records often present inaccuracies, leading to biased VE estimates. Previous studies investigated the impact of non-differential disease misclassification on VE estimation. Methods We explored, through simulation, the impact of non-differential and differential disease- and exposure misclassification when estimating VE using cohort, case-control, test-negative case-control and case-cohort designs. The impact of misclassification on the estimated VE is demonstrated for VE studies on childhood seasonal influenza and pertussis vaccination. We additionally developed a web-application graphically presenting bias for user-selected parameters. Results Depending on the scenario, the misclassification parameters had differing impacts. Decreased exposure specificity had greatest impact for influenza VE estimation when vaccination coverage was low. Decreased exposure sensitivity had greatest impact for pertussis VE estimation for which high vaccination coverage is typically achieved. The impact of the exposure misclassification parameters was found to be more noticeable than that of the disease misclassification parameters. When misclassification is limited, all study designs perform equally. In case of substantial (differential) disease misclassification, the test-negative design performs worse. Conclusions Misclassification can lead to significant bias in VE estimates and its impact strongly depends on the scenario. We developed a web-application for assessing the potential (joint) impact of possibly differential disease- and exposure misclassification that can be modified by users to their own study scenario. Our results and the simulation tool may be used to guide better design, conduct and interpretation of future VE studies

Estimation of the individual residual risk of cervical cancer after vaccination with the nonavalent HPV vaccine

Background: The nonavalent HPV (9vHPV) vaccine is indicated for active immunisation of individuals from the age of 9 years against cervical, vulvar, vaginal and anal premalignant lesions and cancers causally related to vaccine HPV high risk types 16, 18, 31, 33, 45, 52 and 58, and to the HPV low risk types 6 and 11, causing genital warts. Objective: To estimate the lifetime risk (up to the age of 75 years) for developing cervical cancer after vaccinating a HPV naive girl (e.g. 9 to 12 years old) with the 9vHPV vaccine in the hypothetical absence of cervical cancer screening. Methods: We built Monte Carlo simulation models using historical pre-screening age-specific cancer incidence data and current mortality data from Denmark, Finland, Norway, Sweden and the UK. Estimates of genotype contribution fractions and vaccine efficacy were used to estimate the residual lifetime risk after vaccination assuming lifelong protection. Results: We estimated that, in the hypothetical absence of cervical screening and assuming lifelong protection, 9vHPV vaccination reduced the lifetime cervical cancer and mortality risks 7-fold with a residual lifetime cancer risks ranging from 1/572 (UK) to 1/238 (Denmark) and mortality risks ranging from 1/1488 (UK) to 1/851 (Denmark). After decades of repetitive cervical screenings, the lifetime cervical cancer and mortality risks was reduced between 2- and 4-fold depending on the country. Conclusion: Our simulations demonstrate how evidence can be generated to support decision-making by individual healthcare seekers regarding cervical cancer prevention

ADVANCE system testing: Can coverage of pertussis vaccination be estimated in European countries using electronic healthcare databases: An example

Introduction: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines, using existing healthcare databases in Europe. The objective of this paper was to assess the feasibility of using electronic healthcare databases to estimate dose-specific acellular pertussis (aP) and whole cell pertussis (wP) vaccine coverage. Methods: Seven electronic healthcare databases in four European countries (Denmark (n = 2), UK (n = 2), Spain (n = 2) and Italy (n = 1)) participated in this study. Children were included from birth and followed up to age six years. Vaccination exposure was obtained from the databases and classified by type (aP or wP), and dose 1, 2 or 3. Coverage was estimated using period prevalence. For the 2006 birth cohort, two estimation methods for pertussis vaccine coverage, period prevalence and cumulative incidence were compared for each database. Results: The majority of the 2,575,576 children included had been vaccinated at the country-specific recommended ages. Overall, the estimated dose 3 coverage was 88–97% in Denmark (birth cohorts from 2003 to 2014), 96–100% in the UK (2003–2014), 95–98% in Spain (2004–2014) and 94% in Italy (2006–2007). The estimated dose 3 coverage per birth cohort in Denmark and the UK differed by 1–6% compared with national estimates, with our estimates mostly higher. The estimated dose 3 coverage in Spain differed by 0–2% with no consistent over- or underestimation. In Italy, the estimates were 3% lower compared with the national estimates. Except for Italy, for which the two coverage estimation methods generated the same results, the estimated cumulative incidence coverages were consistently 1–10% lower than period prevalence estimates. Conclusion: Thi

ADVANCE system testing: Can safety studies be conducted using electronic healthcare data? An example using pertussis vaccination

Introduction: The Accelerated Development of Vaccine benefit-risk Collaboration in Europe (ADVANCE) public-private collaboration, aimed to develop and test a system for rapid benefit-risk monitoring of vaccines using healthcare databases in Europe. The objective of this proof-of-concept (POC) study was to test the feasibility of the ADVANCE system to generate incidence rates (IRs) per 1000 person-years and incidence rate ratios (IRRs) for risks associated with whole cell- (wP) and acellular- (aP) pertussis vaccines, occurring in event-specific risk windows in children prior to their pre-school-entry booster. Methods: The study population comprised almost 5.1 million children aged 1 month to <6 years vaccinated with wP or aP vaccines during the study period from 1 January 1990 to 31 December 2015. Data from two Danish hospital (H) databases (AUH and SSI) and five primary care (PC) databases from, UK (THIN and RCGP RSC), Spain (SIDIAP and BIFAP) and Italy (Pedianet) were analysed. Database-specific IRRs between risk vs. non-risk periods were estimated in a self-controlled case series study and pooled using random-effects meta-analyses. Results: The overall IRs were: fever, 58.2 (95% CI: 58.1; 58.3), 96.9 (96.7; 97.1) for PC DBs and 8.56 (8.5; 8.6) for H DBs; convulsions, 7.6 (95% CI: 7.6; 7.7), 3.55 (3.5; 3.6) for PC and 12.87 (12.8; 13) for H; persistent crying, 3.9 (95% CI: 3.8; 3.9) for PC, injection-site reactions, 2.2 (95% CI 2.1; 2.2) for PC, hypotonic hypo-responsive episode (HHE), 0.4 (95% CI: 0.4; 0.4), 0.6 (0.6; 0.6) for PC and 0.2 (0.2; 0.3) for H; and somnolence: 0.3 (95% CI: 0.3; 0.3) for PC. The pooled IRRs for persistent crying, fever, and ISR, adjusted for age and healthy vaccinee period were higher after wP vs. aP vaccination, and lower for convulsions, for all doses. The IRR for HHE was slightly lower for wP than aP, while wP was associated with somnolence only for dose 1 and dose 3 compared with aP. Conclusions: The estimated IRs and IRRs were comparable with published data, therefore demonstrating that the ADVANCE system was able to combine several European healthcare databases to assess vaccine safety data for wP and aP vaccination

Quantifying outcome misclassification in multi-database studies: The case study of pertussis in the ADVANCE project

Background: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using European healthcare databases. Event misclassification can result in biased estimates. Using different algorithms for identifying cases of Bordetella pertussis (BorPer) infection as a test case, we aimed to describe a strategy to quantify event misclassification, when manual chart review is not feasible. Methods: Four participating databases retrieved data from primary care (PC) setting: BIFAP: (Spain), THIN and RCGP RSC (UK) and PEDIANET (Italy); SIDIAP (Spain) retrieved data from both PC and hospital settings. BorPer algorithms were defined by healthcare setting, data domain (diagnoses, drugs, or laboratory tests) and concept sets (specific or unspecified pertussis). Algorithm- and database-specific BorPer incidence rates (IRs) were estimated in children aged 0–14 years enrolled in 2012 and 2014 and followed up until the end of each calendar year and compared with IRs of confirmed pertussis from the ECDC surveillance system (TESSy). Novel formulas were used to approximate validity indices, based on a small set of assumptions. They were applied to approximately estimate positive predictive value (PPV) and sensitivity in SIDIAP. Results: The number of cases and the estimated BorPer IRs per 100,000 person-years in PC, using data representing 3,173,268 person-years, were 0 (IR = 0.0), 21 (IR = 4.3), 21 (IR = 5.1), 79 (IR = 5.7), and 2 (IR = 2.3) in BIFAP, SIDIAP, THIN, RCGP RSC and PEDIANET respectively. The IRs for combined specific/unspecified pertussis were higher than TESSy, suggesting that some false positives had been included. In SIDIAP the estimated IR was 45.0 when discharge diagnoses were included. The sensitivity and PPV of combined PC specific and unspecific diagnoses for BorPer cases in SIDIAP were approximately 85% and 72%, respectively. Conclusion: Retrieving BorPer cases using only specific concepts has low sensitivity in PC databases, while including cases retrieved by unspecified concepts introduces false positives, which were approximately estimated to be 28% in one database. The share of cases that cannot be retrieved from a PC database because they are only seen in hospital was approximately estimated to be 15% in one database. This study demonstrated that quantifying the impact of different event-finding algorithms across databases and benchmarking with disease surveillance data can provide approximate estimates of algorithm validity

ADVANCE system testing: Benefit-risk analysis of a marketed vaccine using multi-criteria decision analysis and individual-level state transition modelling

Background: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using electronic health record (eHR) databases in Europe. Proof-of-concept studies were designed to assess the proposed processes and system for generating the required evidence to perform B/R assessment and near-real time monitoring of vaccines. We aimed to test B/R methodologies for vaccines, using the comparison of the B/R profiles of whole-cell (wP) and acellular pertussis (aP) vaccine formulations in children as an example. Methods: We used multi-criteria decision analysis (MCDA) to structure the B/R assessment combined with individual-level state transition modelling to build the B/R effects table. In the state transition model, we simulated the number of events in two hypothetical cohorts of 1 million children followed from first pertussis dose till pre-school-entry booster (or six years of age, whichever occurred first), with one cohort receiving wP, and the other aP. The benefits were reductions in pertussis incidence and complications. The risks were increased incidences of febrile convulsions, fever, hypotonic-hyporesponsive episodes, injection-site reactions and persistent crying. Most model parameters were informed by estimates (coverage, background incidences, relative risks) from eHR databases from Denmark (SSI), Spain (BIFAP and SIDIAP), Italy (Pedianet) and the UK (RCGP-RSC and THIN). Preferences were elicited from clinical and epidemiological experts. Results: Using state transition modelling to build the B/R effects table facilitated the comparison of different vaccine effects (e.g. immediate vaccine risks vs long-term vaccine benefits). Estimates from eHR databases could be used to inform the simulation model. The model results could be easily combined with preference weights to obtain B/R scores. Conclusion: Existing B/R methodology, modelling and estimates from eHR databases can be successfully used for B/R assessment of vaccines

Advance system testing: Vaccine benefit studies using multi-country electronic health data – The example of pertussis vaccination

The Accelerated Development of VAccine benefit-risk Collaboration in Europe (ADVANCE), a public-private consortium, implemented and tested a distributed network system for the generation of evidence on the benefits-risks of marketed vaccines in Europe. We tested the system by estimating the incidence rate (IR) of pertussis and pertussis-related complications in children vaccinated with acellular (aP) and whole-cell (wP) pertussis vaccine. Data from seven electronic databases from four countries (Denmark: AUH and SSI, Spain: SIDIAP and BIFAP, UK: THIN and RCGP RSC and Italy: Pedianet) were included in a retrospective cohort analysis. Exposure was defined as any pertussis vaccination (aP or wP). The follow-up time started 14 days after the first dose. Children who had received any pertussis vaccine from January 1990 to December 2015 were included (those w