Evaluation of the methodological quality of studies of the performance of diagnostic tests for bovine tuberculosis using QUADAS

There has been little assessment of the methodological quality of studies measuring the performance (sensitivity and/or specificity) of diagnostic tests for animal diseases. In a systematic review, 190 studies of tests for bovine tuberculosis (bTB) in cattle (published 1934-2009) were assessed by at least one of 18 reviewers using the QUADAS (Quality Assessment of Diagnostic Accuracy Studies) checklist adapted for animal disease tests. VETQUADAS (VQ) included items measuring clarity in reporting (n = 3), internal validity (n = 9) and external validity (n = 2). A similar pattern for compliance was observed in studies of different diagnostic test types. Compliance significantly improved with year of publication for all items measuring clarity in reporting and external validity but only improved in four of the nine items measuring internal validity (p < 0.05). 107 references, of which 83 had performance data eligible for inclusion in a meta-analysis were reviewed by two reviewers. In these references, agreement between reviewers' responses was 71% for compliance, 32% for unsure and 29% for non-compliance. Mean compliance with reporting items was 2, 5.2 for internal validity and 1.5 for external validity. The index test result was described in sufficient detail in 80.1% of studies and was interpreted without knowledge of the reference standard test result in only 33.1%. Loss to follow-up was adequately explained in only 31.1% of studies. The prevalence of deficiencies observed may be due to inadequate reporting but may also reflect lack of attention to methodological issues that could bias the results of diagnostic test performance estimates. QUADAS was a useful tool for assessing and comparing the quality of studies measuring the performance of diagnostic tests but might be improved further by including explicit assessment of population sampling strategy.The SE3238 project “Meta-analysis of diagnostic tests and modelling to identify appropriate testing strategies to reduce M. bovis infection in GB herds” was funded by the UK Department for Environment, Food and Rural Affairs (Defra).http://www.elsevier.com/locate/prevetmedam2018Veterinary Tropical Disease

Meta-analyses of the sensitivity and specificity of ante-mortem and post-mortem diagnostic tests for bovine tuberculosis in the UK and Ireland

Publication history: Accepted - 26 February 2017; Published online - 6th March 2017.Bovine Tuberculosis (bTB) in cattle is a global health problem and eradication of the disease requires accu-rate estimates of diagnostic test performance to optimize their efficiency. The objective of this study was,through statistical meta-analyses, to obtain estimates of sensitivity (Se) and specificity (Sp), for 14 differ-ent ante-mortem and post-mortem diagnostic tests for bTB in cattle. Using data from a systematic reviewof the scientific literature (published 1934–2009) diagnostic Se and Sp were estimated using Bayesianlogistic regression models adjusting for confounding factors. Random effect terms were used to accountfor unexplained heterogeneity. Parameters in the models were implemented using Markov Chain MonteCarlo (MCMC), and posterior distributions for the diagnostic parameters with adjustment for covariates(confounding factors) were obtained using the inverse logit function. Estimates for Se and/or Sp of thetuberculin skin tests and the IFN- blood test were compared with estimates published 2010–2015.Median Se for the single intradermal comparative cervical tuberculin skin (SICCT) test (standard inter-pretation) was 0.50 and Bayesian credible intervals (CrI) were wide (95% CrI 0.26, 0.78). Median Sp forthe SICCT test was 1.00 (95% CrI 0.99, 1.00). Estimates for the IFN- blood test Bovine Purified ProteinDerivative (PPD)-Avian PPD and Early Secreted Antigen target 6 and Culture Filtrate Protein 10 (ESAT-6/CFP10) ESAT6/CFP10 were 0.67 (95% CrI 0.49, 0.82) and 0.78 (95% CrI 0.60, 0.90) respectively for Se,and 0.98 (95% CrI 0.96, 0.99) and 0.99 (95% CrI 0.99, 1.00) for Sp. The study provides an overview of theaccuracy of a range of contemporary diagnostic tests for bTB in cattle. Better understanding of diagnostictest performance is essential for the design of effective control strategies and their evaluation.The SE3238 project “Meta-analysis of diagnostic tests and modelling to identify appropriate testing strategies to reduce M. bovis infection in GB herds” was funded by the UK Department for Envi-ronment, Food and Rural Affairs (Defra)

Genomic prediction for tuberculosis resistance in dairy cattle

The increasing prevalence of bovine tuberculosis (bTB) in the UK and the limitations of the currently available diagnostic and control methods require the development of complementary approaches to assist in the sustainable control of the disease. One potential approach is the identification of animals that are genetically more resistant to bTB, to enable breeding of animals with enhanced resistance. This paper focuses on prediction of resistance to bTB. We explore estimation of direct genomic estimated breeding values (DGVs) for bTB resistance in UK dairy cattle, using dense SNP chip data, and test these genomic predictions for situations when disease phenotypes are not available on selection candidates.We estimated DGVs using genomic best linear unbiased prediction methodology, and assessed their predictive accuracies with a cross validation procedure and receiver operator characteristic (ROC) curves. Furthermore, these results were compared with theoretical expectations for prediction accuracy and area-under-the-ROC-curve (AUC). The dataset comprised 1151 Holstein-Friesian cows (bTB cases or controls). All individuals (592 cases and 559 controls) were genotyped for 727,252 loci (Illumina Bead Chip). The estimated observed heritability of bTB resistance was 0.23±0.06 (0.34 on the liability scale) and five-fold cross validation, replicated six times, provided a prediction accuracy of 0.33 (95% C.I.: 0.26, 0.40). ROC curves, and the resulting AUC, gave a probability of 0.58, averaged across six replicates, of correctly classifying cows as diseased or as healthy based on SNP chip genotype alone using these data.These results provide a first step in the investigation of the potential feasibility of genomic selection for bTB resistance using SNP data. Specifically, they demonstrate that genomic selection is possible, even in populations with no pedigree data and on animals lacking bTB phenotypes. However, a larger training population will be required to improve prediction accuracies

Estimation of the relative sensitivity of the comparative tuberculin skin test in tuberculous cattle herds subjected to depopulation

Bovine tuberculosis (bTB) is one of the most serious economic animal health problems affecting the cattle industry in Great Britain (GB), with incidence in cattle herds increasing since the mid-1980s. The single intradermal comparative cervical tuberculin (SICCT) test is the primary screening test in the bTB surveillance and control programme in GB and Ireland. The sensitivity (ability to detect infected cattle) of this test is central to the efficacy of the current testing regime, but most previous studies that have estimated test sensitivity (relative to the number of slaughtered cattle with visible lesions [VL] and/or positive culture results) lacked post-mortem data for SICCT test-negative cattle. The slaughter of entire herds (“whole herd slaughters” or “depopulations”) that are infected by bTB are occasionally conducted in GB as a last-resort control measure to resolve intractable bTB herd breakdowns. These provide additional post-mortem data for SICCT test-negative cattle, allowing a rare opportunity to calculate the animal-level sensitivity of the test relative to the total number of SICCT test-positive and negative VL animals identified post-mortem (rSe). In this study, data were analysed from 16 whole herd slaughters (748 SICCT test-positive and 1031 SICCT test-negative cattle) conducted in GB between 1988 and 2010, using a Bayesian hierarchical model. The overall rSe estimate of the SICCT test at the severe interpretation was 85% (95% credible interval [CI]: 78–91%), and at standard interpretation was 81% (95% CI: 70–89%). These estimates are more robust than those previously reported in GB due to inclusion of post-mortem data from SICCT test-negative cattle

Overall mean <i>relative</i> Single Intradermal Comparative Cervical Tuberculin (SICCT) test sensitivity (rSe) estimates (%).

#<p>At severe test interpretation, results are based on 16 herds. At standard interpretation, results are based on 11 herds. 95% CI represent the 95% credible intervals.</p><p>Estimates are relative to visible lesion (VL) detection.</p

Overall mean <i>relative</i> Single Intradermal Comparative Cervical Tuberculin test sensitivity (rSe) estimates for each herd.

<p>Estimates are shown for severe and standard test interpretation, derived from logistic regression models fitted with and without herd-level effects. The solid lines represent the <i>mean</i> estimates, and the dashed lines represent the 95% credible intervals. The red lines are from the model <i>including</i> herd-level effects, and the blue lines are from the model <i>excluding</i> herd-level effects.</p

Evaluation of the methodological quality of studies of the performance of diagnostic tests for bovine tuberculosis using QUADAS

Publication history: Accepted - 18 March 2017; Published online - 22 March 2017There has been little assessment of the methodological quality of studies measuring the performance (sensitivity and/or specificity) of diagnostic tests for animal diseases. In a systematic review, 190 studies of tests for bovine tuberculosis (bTB) in cattle (published 1934–2009) were assessed by at least one of 18 reviewers using the QUADAS (Quality Assessment of Diagnostic Accuracy Studies) checklist adapted for animal disease tests. VETQUADAS (VQ) included items measuring clarity in reporting (n = 3), internal validity (n = 9) and external validity (n = 2). A similar pattern for compliance was observed in studies of different diagnostic test types. Compliance significantly improved with year of publication for all items measuring clarity in reporting and external validity but only improved in four of the nine items measuring internal validity (p < 0.05). 107 references, of which 83 had performance data eligible for inclusion in a meta-analysis were reviewed by two reviewers. In these references, agreement between reviewers’ responses was 71% for compliance, 32% for unsure and 29% for non-compliance. Mean compliance with reporting items was 2, 5.2 for internal validity and 1.5 for external validity. The index test result was described in sufficient detail in 80.1% of studies and was interpreted without knowledge of the reference standard test result in only 33.1%. Loss to follow-up was adequately explained in only 31.1% of studies. The prevalence of deficiencies observed may be due to inadequate reporting but may also reflect lack of attention to methodological issues that could bias the results of diagnostic test performance estimates. QUADAS was a useful tool for assessing and comparing the quality of studies measuring the performance of diagnostic tests but might be improved further by including explicit assessment of population sampling strategy.The SE3238 project “Meta-analysis of diagnostic tests and modelling to identify appropriate testing strategies to reduce M. bovis infection in GB herds” was funded by the UK Department for Environment, Food and Rural Affairs (Defra)

Methodology and preliminary results of a systematic literature review of ante-mortem and post-mortem diagnostic tests for bovine tuberculosis

A systematic review was conducted to identify studies with data for statistical meta-analyses of sensitivity (Se) and specificity (Sp) of ante-mortem and post-mortem diagnostic tests for bovine tuberculosis (bTB) in cattle. Members of a working group (WG) developed and tested search criteria and developed a standardised two-stage review process, to identify primary studies with numerator and denominator data for test performance and an agreed range of covariate data. No limits were applied to year, language, region or type of test in initial searches of electronic databases. In stage 1, titles and available abstracts were reviewed. References that complied with stage 1 selection criteria were reviewed in entirety and agreed data were extracted from references that complied with stage 2 selection criteria. At stage 1, 9782 references were reviewed and 261 (2.6%) passed through to stage 2 where 215 English language references were each randomly allocated to two of 18 WG reviewers and 46 references in other languages were allocated to native speakers. Agreement regarding eligibility between reviewers of the same reference at stage 2 was moderate (Kappa statistic = 0.51) and a resolution procedure was conducted. Only 119 references (published 1934–2009) were identified with eligible performance estimates for one or more of 14 different diagnostic test types; despite a comprehensive search strategy and the global impact of bTB. Searches of electronic databases for diagnostic test performance data were found to be nonspecific with regard to identifying references with diagnostic test Se or Sp data. Guidelines for the content of abstracts to research papers reporting diagnostic test performance are presented. The results of meta-analyses of the sensitivity and specificity of the tests, and of an evaluation of the methodological quality of the source references, are presented in accompanying papers (Nuñez-Garcia et al., 2017; Downs et al., 2017).http://www.elsevier.com/locate/prevetmed2019-05-01hj2018Veterinary Tropical Disease