An Evaluation of Epidemiological and Reporting Characteristics of Complementary and Alternative Medicine (CAM) Systematic Reviews (SRs)

<div><h3>Background</h3><p>Systematic reviews (SRs) are abundant. The optimal reporting of SRs is critical to enable clinicians to use their findings to make informed treatment decisions. Complementary and alternative medicine (CAM) therapies are widely used therefore it is critical that conduct and reporting of systematic research in this field be of high quality. Here, methodological and reporting characteristics of a sample of CAM-related SRs and a sample of control SRs are evaluated and compared.</p> <h3>Methods</h3><p>MEDLINE® was searched to identify non-Cochrane SRs indexed from January 2010 to May 2011. Control SRs were retrieved and a search filter was used to identify CAM SRs. Citations were screened and publications that met a pre-specified definition of a SR were included. Pre-designed, standardized data extraction forms were developed to capture reporting and methodological characteristics of the included reviews. Where appropriate, samples were compared descriptively.</p> <h3>Results</h3><p>A total of 349 SRs were identified, of which 174 were CAM-related SRs and 175 were conventional SRs. We compared 131 CAM-related non-Cochrane SRs to the 175 conventional non-Cochrane reviews. Fifty-seven percent (75/131) of CAM SRs specified a primary outcome compared to 21% (37/175) of conventional sample reviews. Reporting of publication bias occurred in less than 5% (6/131) of the CAM sample versus 46% (80/175) of the conventional sample of SRs. Source of funding was frequently and consistently under-reported. Less than 5% (11/306) of all SRs reported public availability of a review protocol.</p> <h3>Conclusion</h3><p>The two samples of reviews exhibited different strengths and weaknesses. In some cases there were consistencies across items which indicate the need for continued improvements in reporting for all SR reports. We advise authors to utilise the PRISMA Statement or other SR guidance when reporting SRs.</p> </div

Reporting Characteristics of systematic reviews.

h<p>Reported as either “systematic review” or “meta-analysis”.</p>i<p>Includes 100% verification.</p>j<p><b>Control Group:</b> GRADE <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Guyatt1" target="_blank">[37]</a>; AHRQ Guidance <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Viswanathan1" target="_blank">[41]</a>; Egger’s tool <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Egger2" target="_blank">[44]</a>; Downs and Black <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Downs1" target="_blank">[22]</a>; Zaza et al. <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Zaza1" target="_blank">[23]</a>; publication bias only assessed; International Society of Pharmacoeconomics and Outcomes Research <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Cox1" target="_blank">[45]</a>; The Delphi list <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Verhagen1" target="_blank">[46]</a>; US Preventative Services Task Force criteria <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Harris3" target="_blank">[47]</a>; Cho and Bero <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Cho1" target="_blank">[48]</a>; Sauerland <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Sauerland1" target="_blank">[49]</a>; America academy of neurology <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-9" target="_blank">[50]</a>; PEDro <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-deMorton1" target="_blank">[51]</a>; COREQ <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Tong1" target="_blank">[52]</a>; West <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-West1" target="_blank">[53]</a>; Schulz’s Allocation concealment <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Schulz2" target="_blank">[54]</a>; outcome reporting bias only; Centre of evidence-based medicine at the University of Oxford <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Phillips2" target="_blank">[55]</a>; MINORS <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-4" target="_blank">[24]</a>; DTA assessment; “assessed based on study design”; labelled sensitivity analysis as quality assessment; adjusted analysis by characteristics calling it quality assessment; <b>CAM Group</b>: GRADE <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Guyatt1" target="_blank">[37]</a>; NICE <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-8" target="_blank">[40]</a>; EPC based <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Viswanathan1" target="_blank">[41]</a>; Downs and Black <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Downs1" target="_blank">[22]</a>; Delphi list <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Verhagen1" target="_blank">[46]</a>; PEDro <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-deMorton1" target="_blank">[51]</a>; Allocation concealment <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Schulz2" target="_blank">[54]</a>; Centre of Evidence-Based Medicine at the University of Oxford <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Phillips2" target="_blank">[55]</a>; McMaster Quality Assessment Scale of Harms (McHarm) <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Santaguida1" target="_blank">[56]</a>; Oxman and Guyatt <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Oxman1" target="_blank">[57]</a>; Centre for reviews and dissemination <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Khan1" target="_blank">[58]</a>; MINORS <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-4" target="_blank">[24]</a>; CASP <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-10" target="_blank">[59]</a>; Scottish Intercollegiate Guidelines Network <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Harbour1" target="_blank">[60]</a>; Stetler’s Evidence Ranking system <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Stetler1" target="_blank">[61]</a>; Tulder Score <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-van1" target="_blank">[62]</a>; MINORS <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-4" target="_blank">[24]</a>; Wilson and Lawrence Scores; RAC; Ostello.</p>k<p>Gray Literature searching refers to systematic review search methods to identify primary studies which are not identified via standard searching methods <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Debachere1" target="_blank">[63]</a>.</p>l<p>Independent of I².</p

Flow diagram of included systematic reviews.

<p>Flow diagram of included systematic reviews.</p

Defining a ‘systematic review’.

<p>Defining a ‘systematic review’.</p

Epidemiology of Systematic reviews.

a<p><b>Moher 2007 </b><a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0053536#pone.0053536-Moher2" target="_blank">[<b>4</b>]</a><b>:</b> 30 countries <10 reviews/country. <b>Control Group:</b> India, Norway, Sweden, Belgium, Brazil, Denmark, Greece, Israel, Italy, Poland, Singapore, Spain, Switzerland. <b>CAM Group:</b> Japan, New Zealand, Thailand, Belgium, Chile, Denmark, Hong Kong, Ireland, Italy, Malaysia, Nigeria, Oman, Peru, Saudi Arabia, South Africa, Spain, Taiwan.</p>b<p>Methodological, educational, prevalence of use, overall health effects and mindfulness.</p

Summary of Considerations for Reporting and Interpretation of Results.

<p>Summary of Considerations for Reporting and Interpretation of Results.</p

Additional file 3: of Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study

List of included studies [33, 40, 42–51, 63–126]. (DOCX 22 kb

Summary of Included Studies.

<p>Summary of Included Studies.</p

Summary of Considerations for Methodologic Reporting.

<p>Summary of Considerations for Methodologic Reporting.</p

Additional file 2: of Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study

Search strategy. (DOCX 17 kb