Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals

<div><p>Background</p><p>Selective outcome reporting is a significant methodological concern. Comparisons between the outcomes reported in clinical trial registrations and those later published allow investigators to understand the extent of selection bias among trialists. We examined the possibility of selective outcome reporting in randomized controlled trials (RCTs) published in neurology journals.</p><p>Methods</p><p>We searched PubMed for randomized controlled trials from Jan 1, 2010 –Dec 31, 2015 published in the top 3 impact factor neurology journals. These articles were screened according to specific inclusion criteria. Each author individually extracted data from trials following a standardized protocol. A second author verified each extracted element and discrepancies were resolved. Consistency between registered and published outcomes was evaluated and correlations between discrepancies and funding, journal, and temporal trends were examined.</p><p>Results</p><p>180 trials were included for analysis. 10 (6%) primary outcomes were demoted, 38 (21%) primary outcomes were omitted from the publication, and 61 (34%) unregistered primary outcomes were added to the published report. There were 18 (10%) cases of secondary outcomes being upgraded to primary outcomes in the publication, and there were 53 (29%) changes in timing of assessment. Of 82 (46%) major discrepancies with reported p-values, 54 (66.0%) favored publication of statistically significant results.</p><p>Conclusion</p><p>Across trials, we found 180 major discrepancies. 66% of major discrepancies with a reported p-value (n = 82) favored statistically significant results. These results suggest a need within neurology to provide more consistent and timely registration of outcomes.</p></div

Characteristics of randomized controlled trials published in the three highest-impact journals between January 1, 2010 and December 31, 2015.

<p>Characteristics of randomized controlled trials published in the three highest-impact journals between January 1, 2010 and December 31, 2015.</p

Evidence of selective reporting bias in hematology journals: A systematic review

<div><p>Introduction</p><p>Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored.</p><p>Methods</p><p>Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in <i>Blood</i>; <i>British Journal of Haematology</i>; <i>American Journal of Hematology</i>; <i>Leukemia</i>; and <i>Haematologica</i> were included.</p><p>Results</p><p>Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirty-one major discrepancies were published with a <i>P</i>-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes.</p><p>Conclusion</p><p>Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.</p></div

Year-by-year frequency of major discrepancies between published and registered outcomes in RCTs registered before or during patient enrollment (n = 181), and the effect of these discrepancies on the statistical significance of published outcomes, by year.

<p>Year-by-year frequency of major discrepancies between published and registered outcomes in RCTs registered before or during patient enrollment (n = 181), and the effect of these discrepancies on the statistical significance of published outcomes, by year.</p

Published RCTs that were registered before or during trial completion and have major discrepancies with their trial registries, and the effect of these discrepancies on the statistical significance of published outcomes, by funding source.

<p>Published RCTs that were registered before or during trial completion and have major discrepancies with their trial registries, and the effect of these discrepancies on the statistical significance of published outcomes, by funding source.</p

PRISMA flow diagram.

<p>Details regarding exclusions at each level of analysis.</p

Number of major discrepancies per funding source.

<p>Published RCTs that were registered before or during trial completion and have major discrepancies with their trial registries and the effect of these discrepancies on the statistical significance of published outcomes, by funding source.</p

Demographic information.

<p>Characteristics of included RCTs published between January 1, 2010 and December 31, 2015.</p