Interpretation of Results of Studies Evaluating an Intervention Highlighted in Google Health News: A Cross-Sectional Study of News

Background Mass media through the Internet is a powerful means of disseminating medical research. We aimed to determine whether and how the interpretation of research results is misrepresented by the use of “spin” in the health section of Google News. Spin was defined as specific way of reporting, from whatever motive (intentional or unintentional), to emphasize that the beneficial effect of the intervention is greater than that shown by the results. Methods We conducted a cross-sectional study of news highlighted in the health section of US, UK and Canada editions of Google News between July 2013 and January 2014. We searched for news items for 3 days a week (i.e., Monday, Wednesday, and Friday) during 6 months and selected a sample of 130 news items reporting a scientific article evaluating the effect of an intervention on human health. Results In total, 78% of the news did not provide a full reference or electronic link to the scientific article. We found at least one spin in 114 (88%) news items and 18 different types of spin in news. These spin were mainly related to misleading reporting (59%) such as not reporting adverse events that were reported in the scientific article (25%), misleading interpretation (69%) such as claiming a causal effect despite non-randomized study design (49%) and overgeneralization/misleading extrapolation (41%) of the results such as extrapolating a beneficial effect from an animal study to humans (21%). We also identified some new types of spin such as highlighting a single patient experience for the success of a new treatment instead of focusing on the group results. Conclusions Interpretation of research results was frequently misrepresented in the health section of Google News. However, we do not know whether these spin were from the scientific articles themselves or added in the news

Impact of single centre status on estimates of intervention effects in trials with continuous outcomes: meta-epidemiological study

Objective To compare estimates of intervention effects between single centre and multicentre randomised controlled trials with continuous outcomes

Impact of adding a limitations section to abstracts of systematic reviews on readers’ interpretation: a randomized controlled trial

To allow an accurate evaluation of abstracts of systematic reviews, the PRISMA Statement recommends that the limitations of the evidence (e.g., risk of bias, publication bias, inconsistency, imprecision) should be described in the abstract. We aimed to evaluate the impact of adding such limitations sections on reader’s interpretation. We performed a two-arm parallel group randomized controlled trial (RCT) using a sample of 30 abstracts of systematic reviews evaluating the effects of healthcare intervention with conclusions favoring the beneficial effect of the experimental treatments. Two formats of these abstracts were derived: one reported without and one with a standardized limitations section written according to the PRISMA statement for abstracts. The primary outcome was readers’ confidence in the results of the systematic review as stated in the abstract assessed by a Likert scale from 0, not at all confident, to 10, very confident. In total, 300 participants (corresponding authors of RCT reports indexed in PubMed) were randomized by a web-based randomization procedure to interpret one abstract with a limitations section (n = 150) or without a limitations section (n = 150). Participants were blinded to the study hypothesis. Adding a limitations section did not modify readers’ interpretation of findings in terms of confidence in the results (mean difference [95% confidence interval] 0.19 [−0.37–0.74], p = 0.50), confidence in the validity of the conclusions (0.07 [−0.49–0.62], p = 0.80), or benefit of the experimental intervention (0.12 [−0.42–0.44], p = 0.65). This study is limited because the participants were expert-readers and are not representative of all systematic review readers. Adding a limitations section to abstracts of systematic reviews did not affect readers’ interpretation of the abstract results. Other studies are needed to confirm the results and explore the impact of a limitations section on a less expert panel of participants. Trial registration: ClinicalTrial.gov (NCT01848782)

Correction to: Three randomized controlled trials evaluating the impact of “spin” in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit

Figure 3 in the original article [1] is incorrect; labels for secondary outcomes have been shifted and do not correspond to the numbers reported in the table (Additional file 8). The corrected version can be seen ahead. This figure should be used over the figure 3 seen in the original article. This error does not affect the results, interpretation, or conclusion

Three randomized controlled trials evaluating the impact of “spin” in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit

Background News stories represent an important source of information. We aimed to evaluate the impact of “spin” (i.e., misrepresentation of study results) in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit. Methods We conducted three two-arm, parallel-group, Internet-based randomized trials (RCTs) comparing the interpretation of news stories reported with or without spin. Each RCT considered news stories reporting a different type of study: (1) pre-clinical study, (2) phase I/II non-RCT, and (3) phase III/IV RCT. For each type of study, we identified news stories reported with spin that had earned mention in the press. Two versions of the news stories were used: the version with spin and a version rewritten without spin. Participants were patients/caregivers involved in Inspire, a large online community of more than one million patients/caregivers. The primary outcome was participants’ interpretation assessed by one specific question “What do you think is the probability that ‘treatment X’ would be beneficial to patients?” (scale, 0 [very unlikely] to 10 [very likely]). Results For each RCT, 300 participants were randomly assigned to assess a news story with spin (n = 150) or without spin (n = 150), and 900 participants assessed a news story. Participants were more likely to consider that the treatment would be beneficial to patients when the news story was reported with spin. The mean (SD) score for the primary outcome for abstracts reported with and without spin for pre-clinical studies was 7.5 (2.2) versus 5.8 (2.8) (mean difference [95% CI] 1.7 [1.0–2.3], p < 0.001); for phase I/II non-randomized trials, 7.6 (2.2) versus 5.8 (2.7) (mean difference 1.8 [1.0–2.5], p < 0.001); and for phase III/IV RCTs, 7.2 (2.3) versus 4.9 (2.8) (mean difference 2.3 [1.4–3.2], p < 0.001). Conclusions Spin in health news stories reporting studies of pharmacologic treatments affects patients’/caregivers’ interpretation. Trial registration ClinicalTrials.gov, NCT03094078 , NCT03094104 , NCT0309558

Comparative efficacy and safety of second-line treatments for advanced non-small cell lung cancer with wild-type or unknown status for epidermal growth factor receptor: a systematic review and network meta-analysis

Abstract Background Docetaxel, pemetrexed, erlotinib, and gefitinib are recommended as second-line treatment for advanced non-small cell lung cancer (NSCLC) with wild-type or unknown status for epidermal growth factor receptor (EGFR). However, the number of published randomized clinical trials (RCTs) on this topic is increasing. Our objective was to assess the comparative effectiveness and tolerability of all second-line treatments for advanced NSCLC with wild-type or unknown status for EGFR by a systematic review and network meta-analysis. Methods MEDLINE, EMBASE, CENTRAL, ClinicalTrials.gov, and the US Food and Drug Administration website, as well as other sources, were searched for available reports up to June 6, 2017. Two reviewers independently selected published and unpublished reports of RCTs comparing any second-line treatments, extracted data and assessed the risk of bias of all included trials. We performed a Bayesian network meta-analysis. The primary outcomes were overall survival (OS) and progression-free survival (PFS). Secondary outcomes included objective response (ObR), the number of serious adverse events, and quality of life. Results We included 102 RCTs involving 36,058 patients (62% male, median age 61 years, 81% with stage IV cancer, 80% smokers, and 92% with performance status 0–1). We revealed a differential reporting of outcomes between efficacy and safety outcomes. Half of the trials reported safety outcomes and less than 20% quality of life. For OS, nivolumab was more effective than docetaxel (hazard ratio (HR) 0.69, 95% credible interval (CrI) 0.56–0.83), pemetrexed (0.67, 0.52–0.83), erlotinib (0.68, 0.53–0.86), and gefitinib (0.66, 0.53–0.83). Pembrolizumab, atezolizumab, and pemetrexed plus erlotinib were also significantly more effective than docetaxel, pemetrexed, erlotinib, and gefitinib. For PFS, erlotinib plus cabozantinib was more effective than docetaxel (HR 0.39, 95% CrI 0.18–0.84), pemetrexed (0.38, 0.18–0.82), erlotinib (0.37, 0.18–0.78), and gefitinib (0.38, 0.18–0.82). Cabozantinib and pemetrexed plus erlotinib were also significantly more effective than the four recommended treatments. For ObR, no treatment was significantly more effective. The effectiveness of the four recommended treatments was similar and they were ranked among the 25 less-effective treatments. For safety, evidence is insufficient to draw certain conclusions. Conclusions Nivolumab, pembrolizumab, atezolizumab, and pemetrexed plus erlotinib may be the most effective second-line treatments for NSCLC in terms of OS. The four recommended treatments seem to have relatively poor performance. However, the impact on life expectancy of immunotherapy versus other treatments should be further explored by future analyses, and more trials comparing the novel treatments are needed to reduce uncertainty in these results. Trial registration Registration number: PROSPERO (CRD42015017592

General characteristics of health news and scientific articles.

<p>*[IQR], interquartile range</p><p>General characteristics of health news and scientific articles.</p

Three randomized controlled trials evaluating the impact of “spin” in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit

International audienceBACKGROUND:News stories represent an important source of information. We aimed to evaluate the impact of "spin" (i.e., misrepresentation of study results) in health news stories reporting studies of pharmacologic treatments on patients'/caregivers' interpretation of treatment benefit.METHODS:We conducted three two-arm, parallel-group, Internet-based randomized trials (RCTs) comparing the interpretation of news stories reported with or without spin. Each RCT considered news stories reporting a different type of study: (1) pre-clinical study, (2) phase I/II non-RCT, and (3) phase III/IV RCT. For each type of study, we identified news stories reported with spin that had earned mention in the press. Two versions of the news stories were used: the version with spin and a version rewritten without spin. Participants were patients/caregivers involved in Inspire, a large online community of more than one million patients/caregivers. The primary outcome was participants' interpretation assessed by one specific question "What do you think is the probability that 'treatment X' would be beneficial to patients?" (scale, 0 [very unlikely] to 10 [very likely]).RESULTS:For each RCT, 300 participants were randomly assigned to assess a news story with spin (n = 150) or without spin (n = 150), and 900 participants assessed a news story. Participants were more likely to consider that the treatment would be beneficial to patients when the news story was reported with spin. The mean (SD) score for the primary outcome for abstracts reported with and without spin for pre-clinical studies was 7.5 (2.2) versus 5.8 (2.8) (mean difference [95% CI] 1.7 [1.0-2.3], p < 0.001); for phase I/II non-randomized trials, 7.6 (2.2) versus 5.8 (2.7) (mean difference 1.8 [1.0-2.5], p < 0.001); and for phase III/IV RCTs, 7.2 (2.3) versus 4.9 (2.8) (mean difference 2.3 [1.4-3.2], p < 0.001).CONCLUSIONS:Spin in health news stories reporting studies of pharmacologic treatments affects patients'/caregivers' interpretation.TRIAL REGISTRATION:ClinicalTrials.gov, NCT03094078 , NCT03094104 , NCT03095586

Examples of 18 types of spin in health news.

<p>‡ Only including human studies where adverse events were reported in scientific articles (n = 52)</p><p>¥ Applicable to observational studies (n = 48)</p><p><b>§</b> Applicable to observational & animal studies (n = 77)</p><p>Ѱ Applicable to human studies (n = 101)</p><p>ǂ Applicable to animal studies (n = 29)</p><p>Examples of 18 types of spin in health news.</p