Outcome reporting bias in trials: a methodological approach for assessment and adjustment in systematic reviews

Systematic reviews of clinical trials aim to include all relevant studies conducted on a particular topic and to provide an unbiased summary of their results, producing the best evidence about the benefits and harms of medical treatments. Relevant studies, however, may not provide the results for all measured outcomes or may selectively report only some of the analyses undertaken, leading to unnecessary waste in the production and reporting of research, and potentially biasing the conclusions to systematic reviews. In this article, Kirkham and colleagues provide a methodological approach, with an example of how to identify missing outcome data and how to assess and adjust for outcome reporting bias in systematic reviews

Are pediatric Open Access journals promoting good publication practice? An analysis of author instructions

<p>Abstract</p> <p>Background</p> <p>Several studies analyzed whether conventional journals in general medicine or specialties such as pediatrics endorse recommendations aiming to improve publication practice. Despite evidence showing benefits of these recommendations, the proportion of endorsing journals has been moderate to low and varied considerably for different recommendations. About half of pediatric journals indexed in the Journal Citation Report referred to the Uniform Requirements for Manuscripts of the International Committee of Medical Journal Editors (ICMJE) but only about a quarter recommended registration of trials. We aimed to investigate to what extent pediatric open-access (OA) journals endorse these recommendations. We hypothesized that a high proportion of these journals have adopted recommendations on good publication practice since OA electronic publishing has been associated with a number of editorial innovations aiming at improved access and transparency.</p> <p>Methods</p> <p>We identified 41 journals publishing original research in the subject category "Health Sciences, Medicine (General), Pediatrics" of the Directory of Open Access Journals <url>http://www.doaj.org</url>. From the journals' online author instructions we extracted information regarding endorsement of four domains of editorial policy: the Uniform Requirements for Manuscripts, trial registration, disclosure of conflicts of interest and five major reporting guidelines such as the CONSORT (Consolidated Standards of Reporting Trials) statement. Two investigators collected data independently.</p> <p>Results</p> <p>The Uniform Requirements were mentioned by 27 (66%) pediatric OA journals. Thirteen (32%) required or recommended trial registration prior to publication of a trial report. Conflict of interest policies were stated by 25 journals (61%). Advice about reporting guidelines was less frequent: CONSORT was referred to by 12 journals (29%) followed by other reporting guidelines (MOOSE, PRISMA or STARD) (8 journals, 20%) and STROBE (3 journals, 7%). The EQUATOR network, a platform of several guideline initiatives, was acknowledged by 4 journals (10%).</p> <p>Journals published by OA publishing houses gave more guidance than journals published by professional societies or other publishers.</p> <p>Conclusions</p> <p>Pediatric OA journals mentioned certain recommendations such as the Uniform Requirements or trial registration more frequently than conventional journals; however, endorsement is still only moderate. Further research should confirm these exploratory findings in other medical fields and should clarify what the motivations and barriers are in implementing such policies.</p

Standardised Outcomes in Nephrology-Polycystic Kidney Disease (SONG-PKD): study protocol for establishing a core outcome set in polycystic kidney disease

BACKGROUND: Autosomal dominant polycystic kidney disease (ADPKD) is the most common potentially life threatening inherited kidney disease and is responsible for 5-10% of cases of end-stage kidney disease (ESKD). Cystic kidneys may enlarge up to 20 times the weight of a normal kidney due to the growth of renal cysts, and patients with ADPKD have an increased risk of morbidity, premature mortality, and other life-time complications including renal and hepatic cyst and urinary tract infection, intracranial aneurysm, diverticulosis, and kidney pain which impair quality of life. Despite some therapeutic advances and the growing number of clinical trials in ADPKD, the outcomes that are relevant to patients and clinicians, such as symptoms and quality of life, are infrequently and inconsistently reported. This potentially limits the contribution of trials to inform evidence-based decision-making. The Standardised Outcomes in Nephrology-Polycystic Kidney Disease (SONG-PKD) project aims to establish a consensus-based set of core outcomes for trials in PKD (with an initial focus on ADPKD but inclusive of all stages) that patients and health professionals identify as critically important. METHODS: The five phases of SONG-PKD are: a systematic review to identify outcomes that have been reported in existing PKD trials; focus groups with nominal group technique with patients and caregivers to identify, rank, and describe reasons for their choices; qualitative stakeholder interviews with health professionals to elicit individual values and perspectives on outcomes for trials involving patients with PKD; an international three-round Delphi survey with all stakeholder groups (including patients, caregivers, healthcare providers, policy makers, researchers, and industry) to gain consensus on critically important core outcome domains; and a consensus workshop to review and establish a set of core outcome domains and measures for trials in PKD. DISCUSSION: The SONG-PKD core outcome set is aimed at improving the consistency and completeness of outcome reporting across ADPKD trials, leading to improvements in the reliability and relevance of trial-based evidence to inform decisions about treatment and ultimately improve the care and outcomes for people with ADPKD

The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews.

OBJECTIVE: To examine the prevalence of outcome reporting bias-the selection for publication of a subset of the original recorded outcome variables on the basis of the results-and its impact on Cochrane reviews. DESIGN: A nine point classification system for missing outcome data in randomised trials was developed and applied to the trials assessed in a large, unselected cohort of Cochrane systematic reviews. Researchers who conducted the trials were contacted and the reason sought for the non-reporting of data. A sensitivity analysis was undertaken to assess the impact of outcome reporting bias on reviews that included a single meta-analysis of the review primary outcome. RESULTS: More than half (157/283 (55%)) the reviews did not include full data for the review primary outcome of interest from all eligible trials. The median amount of review outcome data missing for any reason was 10%, whereas 50% or more of the potential data were missing in 70 (25%) reviews. It was clear from the publications for 155 (6%) of the 2486 assessable trials that the researchers had measured and analysed the review primary outcome but did not report or only partially reported the results. For reports that did not mention the review primary outcome, our classification regarding the presence of outcome reporting bias was shown to have a sensitivity of 88% (95% CI 65% to 100%) and specificity of 80% (95% CI 69% to 90%) on the basis of responses from 62 trialists. A third of Cochrane reviews (96/283 (34%)) contained at least one trial with high suspicion of outcome reporting bias for the review primary outcome. In a sensitivity analysis undertaken for 81 reviews with a single meta-analysis of the primary outcome of interest, the treatment effect estimate was reduced by 20% or more in 19 (23%). Of the 42 meta-analyses with a statistically significant result only, eight (19%) became non-significant after adjustment for outcome reporting bias and 11 (26%) would have overestimated the treatment effect by 20% or more. CONCLUSIONS: Outcome reporting bias is an under-recognised problem that affects the conclusions in a substantial proportion of Cochrane reviews. Individuals conducting systematic reviews need to address explicitly the issue of missing outcome data for their review to be considered a reliable source of evidence. Extra care is required during data extraction, reviewers should identify when a trial reports that an outcome was measured but no results were reported or events observed, and contact with trialists should be encouraged