The systematic guideline review: method, rationale, and test on chronic heart failure

Background: Evidence-based guidelines have the potential to improve healthcare. However, their de-novo-development requires substantial resources-especially for complex conditions, and adaptation may be biased by contextually influenced recommendations in source guidelines. In this paper we describe a new approach to guideline development-the systematic guideline review method (SGR), and its application in the development of an evidence-based guideline for family physicians on chronic heart failure (CHF). Methods: A systematic search for guidelines was carried out. Evidence-based guidelines on CHF management in adults in ambulatory care published in English or German between the years 2000 and 2004 were included. Guidelines on acute or right heart failure were excluded. Eligibility was assessed by two reviewers, methodological quality of selected guidelines was appraised using the AGREE instrument, and a framework of relevant clinical questions for diagnostics and treatment was derived. Data were extracted into evidence tables, systematically compared by means of a consistency analysis and synthesized in a preliminary draft. Most relevant primary sources were re-assessed to verify the cited evidence. Evidence and recommendations were summarized in a draft guideline. Results: Of 16 included guidelines five were of good quality. A total of 35 recommendations were systematically compared: 25/35 were consistent, 9/35 inconsistent, and 1/35 un-rateable (derived from a single guideline). Of the 25 consistencies, 14 were based on consensus, seven on evidence and four differed in grading. Major inconsistencies were found in 3/9 of the inconsistent recommendations. We re-evaluated the evidence for 17 recommendations (evidence-based, differing evidence levels and minor inconsistencies) - the majority was congruent. Incongruity was found where the stated evidence could not be verified in the cited primary sources, or where the evaluation in the source guidelines focused on treatment benefits and underestimated the risks. The draft guideline was completed in 8.5 man-months. The main limitation to this study was the lack of a second reviewer. Conclusion: The systematic guideline review including framework development, consistency analysis and validation is an effective, valid, and resource saving-approach to the development of evidence-based guidelines

Systematic review and meta-analysis of the diagnostic accuracy of ultrasonography for deep vein thrombosis

Background Ultrasound (US) has largely replaced contrast venography as the definitive diagnostic test for deep vein thrombosis (DVT). We aimed to derive a definitive estimate of the diagnostic accuracy of US for clinically suspected DVT and identify study-level factors that might predict accuracy. Methods We undertook a systematic review, meta-analysis and meta-regression of diagnostic cohort studies that compared US to contrast venography in patients with suspected DVT. We searched Medline, EMBASE, CINAHL, Web of Science, Cochrane Database of Systematic Reviews, Cochrane Controlled Trials Register, Database of Reviews of Effectiveness, the ACP Journal Club, and citation lists (1966 to April 2004). Random effects meta-analysis was used to derive pooled estimates of sensitivity and specificity. Random effects meta-regression was used to identify study-level covariates that predicted diagnostic performance. Results We identified 100 cohorts comparing US to venography in patients with suspected DVT. Overall sensitivity for proximal DVT (95% confidence interval) was 94.2% (93.2 to 95.0), for distal DVT was 63.5% (59.8 to 67.0), and specificity was 93.8% (93.1 to 94.4). Duplex US had pooled sensitivity of 96.5% (95.1 to 97.6) for proximal DVT, 71.2% (64.6 to 77.2) for distal DVT and specificity of 94.0% (92.8 to 95.1). Triplex US had pooled sensitivity of 96.4% (94.4 to 97.1%) for proximal DVT, 75.2% (67.7 to 81.6) for distal DVT and specificity of 94.3% (92.5 to 95.8). Compression US alone had pooled sensitivity of 93.8 % (92.0 to 95.3%) for proximal DVT, 56.8% (49.0 to 66.4) for distal DVT and specificity of 97.8% (97.0 to 98.4). Sensitivity was higher in more recently published studies and in cohorts with higher prevalence of DVT and more proximal DVT, and was lower in cohorts that reported interpretation by a radiologist. Specificity was higher in cohorts that excluded patients with previous DVT. No studies were identified that compared repeat US to venography in all patients. Repeat US appears to have a positive yield of 1.3%, with 89% of these being confirmed by venography. Conclusion Combined colour-doppler US techniques have optimal sensitivity, while compression US has optimal specificity for DVT. However, all estimates are subject to substantial unexplained heterogeneity. The role of repeat scanning is very uncertain and based upon limited data

How does study quality affect the results of a diagnostic meta-analysis?

Background: The use of systematic literature review to inform evidence based practice in diagnostics is rapidly expanding. Although the primary diagnostic literature is extensive, studies are often of low methodological quality or poorly reported. There has been no rigorously evaluated, evidence based tool to assess the methodological quality of diagnostic studies. The primary objective of this study was to determine the extent to which variations in the quality of primary studies impact the results of a diagnostic meta-analysis and whether this differs with diagnostic test type. A secondary objective was to contribute to the evaluation of QUADAS, an evidence-based tool for the assessment of quality in diagnostic accuracy studies. Methods: This study was conducted as part of large systematic review of tests used in the diagnosis and further investigation of urinary tract infection (UTI) in children. All studies included in this review were assessed using QUADAS, an evidence-based tool for the assessment of quality in systematic reviews of diagnostic accuracy studies. The impact of individual components of QUADAS on a summary measure of diagnostic accuracy was investigated using regression analysis. The review divided the diagnosis and further investigation of UTI into the following three clinical stages: diagnosis of UTI, localisation of infection, and further investigation of the UTI. Each stage used different types of diagnostic test, which were considered to involve different quality concerns. Results: Many of the studies included in our review were poorly reported. The proportion of QUADAS items fulfilled was similar for studies in different sections of the review. However, as might be expected, the individual items fulfilled differed between the three clinical stages. Regression analysis found that different items showed a strong association with test performance for the different tests evaluated. These differences were observed both within and between the three clinical stages assessed by the review. The results of regression analyses were also affected by whether or not a weighting (by sample size) was applied. Our analysis was severely limited by the completeness of reporting and the differences between the index tests evaluated and the reference standards used to confirm diagnoses in the primary studies. Few tests were evaluated by sufficient studies to allow meaningful use of meta-analytic pooling and investigation of heterogeneity. This meant that further analysis to investigate heterogeneity could only be undertaken using a subset of studies, and that the findings are open to various interpretations. Conclusion: Further work is needed to investigate the influence of methodological quality on the results of diagnostic meta-analyses. Large data sets of well-reported primary studies are needed to address this question. Without significant improvements in the completeness of reporting of primary studies, progress in this area will be limited

Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review

BACKGROUND: Systematic reviewers seek to comprehensively search for relevant studies and summarize these to present the most valid estimate of intervention effectiveness. The more resources searched, the higher the yield, and thus time and costs required to conduct a systematic review. While there is an abundance of evidence to suggest how extensive a search for randomized controlled trials (RCTs) should be, it is neither conclusive nor consistent. This systematic review was conducted in order to assess the value of different resources to identify trials for inclusion in systematic reviews. METHODS: Seven electronic databases, four journals and Cochrane Colloquia were searched. Key authors were contacted and references of relevant articles screened. Included studies compared two or more sources to find RCTs or controlled clinical trials (CCTs). A checklist was developed and applied to assess quality of reporting. Data were extracted by one reviewer and checked by a second. Medians and ranges for precision and recall were calculated; results were grouped by comparison. Meta-analysis was not performed due to large heterogeneity. Subgroup analyses were conducted for: search strategy (Cochrane, Simple, Complex, Index), expertise of the searcher (Cochrane, librarian, non-librarian), and study design (RCT and CCT). RESULTS: Sixty-four studies representing 13 electronic databases met inclusion criteria. The most common comparisons were MEDLINE vs. handsearching (n = 23), MEDLINE vs. MEDLINE+handsearching (n = 13), and MEDLINE vs. reference standard (n = 13). Quality was low, particularly for the reporting of study selection methodology. Overall, recall and precision varied substantially by comparison and ranged from 0 to 100% and 0 to 99%, respectively. The trial registries performed the best with median recall of 89% (range 84, 95) and median precision of 96.5% (96, 97), although these results are based on a small number of studies. Inadequate or inappropriate indexing was the reason most cited for missing studies. Complex and Cochrane search strategies (SS) performed better than Simple SS. CONCLUSION: Multiple-source comprehensive searches are necessary to identify all RCTs for a systematic review, although indexing needs to be improved. Although trial registries demonstrated the highest recall and precision, the Cochrane SS or a Complex SS in consultation with a librarian are recommended. Continued efforts to develop CENTRAL should be supported

A systematic review of the diagnostic accuracy of physical examination for the detection of cirrhosis

BACKGROUND: We conducted a review of the diagnostic accuracy of clinical examination for the diagnosis of cirrhosis. The objectives were: to identify studies assessing the accuracy of clinical examination in the detection of cirrhosis; to summarize the diagnostic accuracy of reported physical examination findings; and to define the effects of study characteristics on estimates of diagnostic accuracy. METHODS: Studies were identified through electronic literature search of MEDLINE (1966 to 2000), search of bibliographic references, and contact with authors. Studies that evaluated indicants from physical examination of patients with known or suspected liver disease undergoing liver biopsy were included. Qualitative data on study characteristics were extracted. Two-by-two tables of presence or absence of physical findings for patients with and without cirrhosis were created from study data. Data for physical findings reported in each study were combined using Summary Receiver Operating Characteristic (SROC) curves or random effects modeling, as appropriate. RESULTS: Twelve studies met inclusion criteria, including a total of 1895 patients, ranging in age from 3 to 90 years. Most studies were conducted in referral populations with elevated aminotransferase levels. Ten physical signs were reported in three or more studies and ten signs in only a single study. Signs for which there was more study data were associated with high specificity (range 75–98%), but low sensitivity (range 15–68%) for histologically-proven cirrhosis. CONCLUSIONS: Physical findings are generally of low sensitivity for the diagnosis of cirrhosis, and signs with higher specificity represent decompensated disease. Most studies have been undertaken in highly selected populations

Reporting and Methods in Clinical Prediction Research: A Systematic Review

Walter Bouwmeester and colleagues investigated the reporting and methods of prediction studies in 2008, in six high-impact general medical journals, and found that the majority of prediction studies do not follow current methodological recommendations

Serum screening with Down's syndrome markers to predict pre-eclampsia and small for gestational age: Systematic review and meta-analysis

<p>Abstract</p> <p>Background</p> <p>Reliable antenatal identification of pre-eclampsia and small for gestational age is crucial to judicious allocation of monitoring resources and use of preventative treatment with the prospect of improving maternal/perinatal outcome. The purpose of this systematic review was to determine the accuracy of five serum analytes used in Down's serum screening for prediction of pre-eclampsia and/or small for gestational age.</p> <p>Methods</p> <p>The data sources included Medline, Embase, Cochrane library, Medion (inception to February 2007), hand searching of relevant journals, reference list checking of included articles, contact with experts. Two reviewers independently selected the articles in which the accuracy of an analyte used in Downs's serum screening before the 25^thgestational week was associated with the occurrence of pre-eclampsia and/or small for gestational age without language restrictions. Two authors independently extracted data on study characteristics, quality and results.</p> <p>Results</p> <p>Five serum screening markers were evaluated. 44 studies, testing 169,637 pregnant women (4376 pre-eclampsia cases) and 86 studies, testing 382,005 women (20,339 fetal growth restriction cases) met the selection criteria. The results showed low predictive accuracy overall. For pre-eclampsia the best predictor was inhibin A>2.79MoM positive likelihood ratio 19.52 (8.33,45.79) and negative likelihood ratio 0.30 (0.13,0.68) (single study). For small for gestational age it was AFP>2.0MoM to predict birth weight < 10^thcentile with birth < 37 weeks positive likelihood ratio 27.96 (8.02,97.48) and negative likelihood ratio 0.78 (0.55,1.11) (single study). A potential clinical application using aspirin as a treatment is given as an example.</p> <p>There were methodological and reporting limitations in the included studies thus studies were heterogeneous giving pooled results with wide confidence intervals.</p> <p>Conclusion</p> <p>Down's serum screening analytes have low predictive accuracy for pre-eclampsia and small for gestational age. They may be a useful means of risk assessment or of use in prediction when combined with other tests.</p