How well do health professionals interpret diagnostic information? A systematic review.

Published onlineResearch Support, Non-U.S. Gov'tOBJECTIVE: To evaluate whether clinicians differ in how they evaluate and interpret diagnostic test information. DESIGN: Systematic review. DATA SOURCES: MEDLINE, EMBASE and PsycINFO from inception to September 2013; bibliographies of retrieved studies, experts and citation search of key included studies. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: Primary studies that provided information on the accuracy of any diagnostic test (eg, sensitivity, specificity, likelihood ratios) to health professionals and that reported outcomes relating to their understanding of information on or implications of test accuracy. RESULTS: We included 24 studies. 6 assessed ability to define accuracy metrics: health professionals were less likely to identify the correct definition of likelihood ratios than of sensitivity and specificity. -25 studies assessed Bayesian reasoning. Most assessed the influence of a positive test result on the probability of disease: they generally found health professionals' estimation of post-test probability to be poor, with a tendency to overestimation. 3 studies found that approaches based on likelihood ratios resulted in more accurate estimates of post-test probability than approaches based on estimates of sensitivity and specificity alone, while 3 found less accurate estimates. 5 studies found that presenting natural frequencies rather than probabilities improved post-test probability estimation and speed of calculations. CONCLUSIONS: Commonly used measures of test accuracy are poorly understood by health professionals. Reporting test accuracy using natural frequencies and visual aids may facilitate improved understanding and better estimation of the post-test probability of disease.This work was partially funded by the UK Medical Research Council (Grant Code G0801405)

Systematic review with meta-analysis: the accuracy of serological tests to support the diagnosis of coeliac disease

BACKGROUND: There is growing support for a biopsy avoidant approach to diagnose coeliac disease in both children and adults, using a serological diagnosis instead. AIMS: To assess the diagnostic accuracy of serological tests for coeliac disease in adults and children. METHODS: Seven electronic databases were searched between January 1990 and August 2020. Eligible diagnostic studies evaluated the accuracy of serological tests for coeliac disease against duodenal biopsy. Risk of bias assessment was performed using QUADAS-2. Bivariate random-effects meta-analyses were used to estimate serology sensitivity and specificity at the most commonly reported thresholds. RESULTS: 113 studies (n = 28,338) were included, all in secondary care populations. A subset of studies were included in meta-analyses due to variations in diagnostic thresholds. Summary sensitivity and specificity of immunoglobulin A (IgA) anti-tissue transglutaminase were 90.7% (95% confidence interval: 87.3%, 93.2%) and 87.4% (84.4%, 90.0%) in adults (5 studies) and 97.7% (91.0%, 99.4%) and 70.2% (39.3%, 89.6%) in children (6 studies); and of IgA endomysial antibodies were 88.0% (75.2%, 94.7%) and 99.6% (92.3%, 100%) in adults (5 studies) and 94.5% (88.9%, 97.3%) and 93.8% (85.2%, 97.5%) in children (5 studies). CONCLUSIONS: Anti-tissue transglutaminase sensitivity appears to be sufficient to rule out coeliac disease in children. The high specificity of endomysial antibody in adults supports its use to rule in coeliac disease. This evidence underpins the current development of clinical guidelines for a serological diagnosis of coeliac disease. Studies in primary care are needed to evaluate serological testing strategies in this setting

Selecting and implementing overview methods: implications from five exemplar overviews

This is the final version of the article. Available from BioMed Central via the DOI in this record.Background Overviews of systematic reviews are an increasingly popular method of evidence synthesis; there is a lack of clear guidance for completing overviews and a number of methodological challenges. At the UK Cochrane Symposium 2016, methodological challenges of five overviews were explored. Using data from these five overviews, practical implications to support methodological decision making of authors writing protocols for future overviews are proposed. Methods Methods, and their justification, from the five exemplar overviews were tabulated and compared with areas of debate identified within current literature. Key methodological challenges and implications for development of overview protocols were generated and synthesised into a list, discussed and refined until there was consensus. Results Methodological features of three Cochrane overviews, one overview of diagnostic test accuracy and one mixed methods overview have been summarised. Methods of selection of reviews and data extraction were similar. Either the AMSTAR or ROBIS tool was used to assess quality of included reviews. The GRADE approach was most commonly used to assess quality of evidence within the reviews. Eight key methodological challenges were identified from the exemplar overviews. There was good agreement between our findings and emerging areas of debate within a recent published synthesis. Implications for development of protocols for future overviews were identified. Conclusions Overviews are a relatively new methodological innovation, and there are currently substantial variations in the methodological approaches used within different overviews. There are considerable methodological challenges for which optimal solutions are not necessarily yet known. Lessons learnt from five exemplar overviews highlight a number of methodological decisions which may be beneficial to consider during the development of an overview protocol.The overview conducted by Pollock [19] was supported by a project grant from the Chief Scientist Office of the Scottish Government. The overview conducted by McClurg [21] was supported by a project grant by the Physiotherapy Research Foundation. The overview by Hunt [22] was supported as part of doctoral programme funding by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula (PenCLAHRC). The overview conducted by Estcourt [20] was supported by an NIHR Cochrane Programme Grant for the Safe and Appropriate Use of Blood Components. The overview conducted by Brunton [23] was commissioned by the Department of Health as part of an ongoing programme of work on health policy research synthesis. Alex Pollock is employed by the Nursing, Midwifery and Allied Health Professions (NMAHP) Research Unit, which is supported by the Chief Scientist Office of the Scottish Government. Pauline Campbell is supported by the Chief Nurses Office of the Scottish Government

The value of predicting restriction of fetal growth and compromise of its wellbeing: Systematic quantitative overviews (meta-analysis) of test accuracy literature

BACKGROUND: Restriction of fetal growth and compromise of fetal wellbeing remain significant causes of perinatal death and childhood disability. At present, there is a lack of scientific consensus about the best strategies for predicting these conditions before birth. Therefore, there is uncertainty about the best management of pregnant women who might have a growth restricted baby. This is likely to be due to a dearth of clear collated information from individual research studies drawn from different sources on this subject. METHODS/DESIGN: A series of systematic reviews and meta-analyses will be undertaken to determine, among pregnant women, the accuracy of various tests to predict and/or diagnose fetal growth restriction and compromise of fetal wellbeing. We will search Medline, Embase, Cochrane Library, MEDION, citation lists of review articles and eligible primary articles and will contact experts in the field. Independent reviewers will select studies, extract data and assess study quality according to established criteria. Language restrictions will not be applied. Data synthesis will involve meta-analysis (where appropriate), exploration of heterogeneity and publication bias. DISCUSSION: The project will collate and synthesise the available evidence regarding the value of the tests for predicting restriction of fetal growth and compromise of fetal wellbeing. The systematic overviews will assess the quality of the available evidence, estimate the magnitude of potential benefits, identify those tests with good predictive value and help formulate practice recommendations

Design Characteristics Influence Performance of Clinical Prediction Rules in Validation: A Meta-Epidemiological Study

BACKGROUND: Many new clinical prediction rules are derived and validated. But the design and reporting quality of clinical prediction research has been less than optimal. We aimed to assess whether design characteristics of validation studies were associated with the overestimation of clinical prediction rules' performance. We also aimed to evaluate whether validation studies clearly reported important methodological characteristics. METHODS: Electronic databases were searched for systematic reviews of clinical prediction rule studies published between 2006 and 2010. Data were extracted from the eligible validation studies included in the systematic reviews. A meta-analytic meta-epidemiological approach was used to assess the influence of design characteristics on predictive performance. From each validation study, it was assessed whether 7 design and 7 reporting characteristics were properly described. RESULTS: A total of 287 validation studies of clinical prediction rule were collected from 15 systematic reviews (31 meta-analyses). Validation studies using case-control design produced a summary diagnostic odds ratio (DOR) 2.2 times (95% CI: 1.2-4.3) larger than validation studies using cohort design and unclear design. When differential verification was used, the summary DOR was overestimated by twofold (95% CI: 1.2 -3.1) compared to complete, partial and unclear verification. The summary RDOR of validation studies with inadequate sample size was 1.9 (95% CI: 1.2 -3.1) compared to studies with adequate sample size. Study site, reliability, and clinical prediction rule was adequately described in 10.1%, 9.4%, and 7.0% of validation studies respectively. CONCLUSION: Validation studies with design shortcomings may overestimate the performance of clinical prediction rules. The quality of reporting among studies validating clinical prediction rules needs to be improved

Methodological Deficits in Diagnostic Research Using ‘-Omics’ Technologies: Evaluation of the QUADOMICS Tool and Quality of Recently Published Studies

Background: QUADOMICS is an adaptation of QUADAS (a quality assessment tool for use in systematic reviews of diagnostic accuracy studies), which takes into account the particular challenges presented by '-omics' based technologies. Our primary objective was to evaluate the applicability and consistency of QUADOMICS. Subsequently we evaluated and describe the methodological quality of a sample of recently published studies using the tool. Methodology/Principal Findings: 45'-omics'- based diagnostic studies were identified by systematic search of Pubmed using suitable MeSH terms (>Genomics>, >Sensitivity and specificity>, >Diagnosis>). Three investigators independently assessed the quality of the articles using QUADOMICS and met to compare observations and generate a consensus. Consistency and applicability was assessed by comparing each reviewer's original rating with the consensus. Methodological quality was described using the consensus rating. Agreement was above 80% for all three reviewers. Four items presented difficulties with application, mostly due to the lack of a clearly defined gold standard. Methodological quality of our sample was poor; studies met roughly half of the applied criteria (mean ± sd, 54.7±18.4°%). Few studies were carried out in a population that mirrored the clinical situation in which the test would be used in practice, (6, 13.3%);none described patient recruitment sufficiently; and less than half described clinical and physiological factors that might influence the biomarker profile (20, 44.4%). Conclusions: The QUADOMICS tool can consistently be applied to diagnostic '-omics' studies presently published in biomedical journals. A substantial proportion of reports in this research field fail to address design issues that are fundamental to make inferences relevant for patient care. © 2010 Parker et al.This work was supported by the Spanish Agency for Health Technology Assessment, Exp PI06/90311, Instituto de Salud Carlos III and CIBER en Epidemiología y Salud Pública (CIBERESP) in SpainPeer Reviewe

Accuracy of magnetic resonance studies in the detection of chondral and labral lesions in femoroacetabular impingement : systematic review and meta-analysis

Background: Several types of Magnetic resonance imaging (MRI) are commonly used in imaging of femoroacetabular impingement (FAI), however till now there are no clear protocols and recommendations for each type. The aim of this meta-analysis is to detect the accuracy of conventional magnetic resonance imaging (cMRI), direct magnetic resonance arthrography (dMRA) and indirect magnetic resonance arthrography (iMRA) in the diagnosis of chondral and labral lesions in femoroacetabular impingement (FAI). Methods: A literature search was finalized on the 17th of May 2016 to collect all studies identifying the accuracy of cMRI, dMRA and iMRA in diagnosing chondral and labral lesions associated with FAI using surgical results (arthroscopic or open) as a reference test. Pooled sensitivity and specificity with 95% confidence intervals using a random-effects meta-analysis for MRI, dMRA and iMRA were calculated also area under receiver operating characteristic (ROC) curve (AUC) was retrieved whenever possible where AUC is equivocal to diagnostic accuracy. Results: The search yielded 192 publications which were reviewed according inclusion and exclusion criteria then 21 studies fulfilled the eligibility criteria for the qualitative analysis with a total number of 828 cases, lastly 12 studies were included in the quantitative meta-analysis. Meta-analysis showed that as regard labral lesions the pooled sensitivity, specificity and AUC for cMRI were 0.864, 0.833 and 0.88 and for dMRA were 0.91, 0.58 and 0.92. While in chondral lesions the pooled sensitivity, specificity and AUC for cMRI were 0.76, 0.72 and 0.75 and for dMRA were 0.75, 0.79 and 0.83, while for iMRA were sensitivity of 0.722 and specificity of 0.917. Conclusions: The present meta-analysis showed that the diagnostic test accuracy was superior for dMRA when compared with cMRI for detection of labral and chondral lesions. The diagnostic test accuracy was superior for labral lesions when compared with chondral lesions in both cMRI and dMRA. Promising results are obtained concerning iMRA but further studies still needed to fully assess its diagnostic accuracy