Which placebo to cure depression? A thought-provoking network meta-analysis

BACKGROUND: Antidepressants are often considered to be mere placebos despite the fact that meta-analyses are able to rank them. It follows that it should also be possible to rank different placebos, which are all made of sucrose. To explore this issue, which is rather more epistemological than clinical, we designed an unusual meta-analysis to investigate whether the effects of placebo in one situation are different from the effects of placebo in another situation. METHODS: Published and unpublished studies were searched for by three reviewers on Medline, the Cochrane Library, Embase, clinicaltrials.gov, Current Controlled Trial, in bibliographies, and by mailing key organizations. The following studies in first-line treatment for major depressive disorder were considered to construct an “evidence network”: 1) randomized controlled trials (RCTs) versus placebo on fluoxetine, venlafaxine and 2) fluoxetine versus venlafaxine head-to-head RCTs. Two network meta-analyses were run to indirectly compare response and remission rates among three different placebos: 1) fluoxetine placebo, 2) venlafaxine placebo, and 3) venlafaxine/fluoxetine placebo (that is, placebo compared to both venlafaxine and fluoxetine). Publication biases were assessed using funnel plots and statistically tested. RESULTS: The three placebos were not significantly different in terms of response or remission. The antidepressant agents were significantly more efficacious than the placebos, and venlafaxine was more efficacious than fluoxetine. The funnel plots, however, showed a major publication bias. CONCLUSION: The presence of significant levels of publication bias indicates that we cannot even be certain of the conclusion that sucrose equals sucrose in trials of major depressive disorder. This result should remind clinicians to step back to take a more objective view when interpreting a scientific result. It is of crucial importance for their practice, far more so than ranking antidepressant efficacy

Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in the BMJ and PLOS Medicine

Objectives To explore the effectiveness of data sharing by randomized controlled trials (RCTs) in journals with a full data sharing policy and to describe potential difficulties encountered in the process of performing reanalyses of the primary outcomes. Design Survey of published RCTs. Setting PubMed/Medline. Eligibility criteria RCTs that had been submitted and published by The BMJ and PLOS Medicine subsequent to the adoption of data sharing policies by these journals. Main outcome measure The primary outcome was data availability, defined as the eventual receipt of complete data with clear labelling. Primary outcomes were reanalyzed to assess to what extent studies were reproduced. Difficulties encountered were described. Results 37 RCTs (21 from The BMJ and 16 from PLOS Medicine) published between 2013 and 2016 met the eligibility criteria. 17/37 (46%, 95% confidence interval 30% to 62%) satisfied the definition of data availability and 14 of the 17 (82%, 59% to 94%) were fully reproduced on all their primary outcomes. Of the remaining RCTs, errors were identified in two but reached similar conclusions and one paper did not provide enough information in the Methods section to reproduce the analyses. Difficulties identified included problems in contacting corresponding authors and lack of resources on their behalf in preparing the datasets. In addition, there was a range of different data sharing practices across study groups. Conclusions Data availability was not optimal in two journals with a strong policy for data sharing. When investigators shared data, most reanalyses largely reproduced the original results. Data sharing practices need to become more widespread and streamlined to allow meaningful reanalyses and reuse of data

Six principles for assessing scientists for hiring, promotion, and tenure

The negative consequences of relying too heavily on metrics to assess research quality are well known, potentially fostering practices harmful to scientific research such as p-hacking, salami science, or selective reporting. The "flourish or perish" culture defined by these metrics in turn drives the system of career advancement in academia, a system that empirical evidence has shown to be problematic and which fails to adequately take societal and broader impact into account. To address this systemic problem, Florian Naudet, John P. A. Ioannidis, Frank Miedema, Ioana A. Cristea, Steven N. Goodman and David Moher present six principles for assessing scientists for hiring, promotion, and tenure

Six principles for assessing scientists for hiring, promotion, and tenure

The negative consequences of relying too heavily on metrics to assess research quality are well known, potentially fostering practices harmful to scientific research such as p-hacking, salami science, or selective reporting. The "flourish or perish" culture defined by these metrics in turn drives the system of career advancement in academia, a system that empirical evidence has shown to be problematic and which fails to adequately take societal and broader impact into account. To address this systemic problem, Florian Naudet, John P. A. Ioannidis, Frank Miedema, Ioana A. Cristea, Steven N. Goodman and David Moher present six principles for assessing scientists for hiring, promotion, and tenure

Implementing clinical trial data sharing requires training a new generation of biomedical researchers

Data sharing enhances the value of medical research and builds trust in clinical trials, but more biomedical researchers need to be trained in these approaches, which include meta-research, data science and ethical, legal and social issues

Scoping review and evidence mapping of interventions aimed at improving reproducible and replicable science: Protocol

BACKGROUND: Many interventions, especially those linked to open science, have been proposed to improve reproducibility in science. To what extent these propositions are based on scientific evidence from empirical evaluations is not clear. AIMS: The primary objective is to identify Open Science interventions that have been formally investigated regarding their influence on reproducibility and replicability. A secondary objective is to list any facilitators or barriers reported and to identify gaps in the evidence. METHODS: We will search broadly by using electronic bibliographic databases, broad internet search, and contacting experts in the field of reproducibility, replicability, and open science. Any study investigating interventions for their influence on the reproducibility and replicability of research will be selected, including those studies additionally investigating drivers and barriers to the implementation and effectiveness of interventions. Studies will first be selected by title and abstract (if available) and then by reading the full text by at least two independent reviewers. We will analyze existing scientific evidence using scoping review and evidence gap mapping methodologies. RESULTS: The results will be presented in interactive evidence maps, summarized in a narrative synthesis, and serve as input for subsequent research. REVIEW REGISTRATION: This protocol has been pre-registered on OSF under doi https://doi.org/10.17605/OSF.IO/D65YS

The worldwide clinical trial research response to the COVID-19 pandemic - the first 100 days

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited <20% of the targeted sample; 27 between 20-50%; and 30 over 50% (median 14.8% [IQR 2.0-62.0%]). Conclusions: The size and speed of the COVID-19 clinical trials agenda is unprecedented. However, most trials were small investigating a small fraction of treatment options. The feasibility of this research agenda is questionable, and many trials may end in futility, wasting research resources. Much better coordination is needed to respond to global health threats

Antidepressant Response in Major Depressive Disorder: A Meta-Regression Comparison of Randomized Controlled Trials and Observational Studies

To compare response to antidepressants between randomized controlled trials (RCTs) and observational trials.Published and unpublished studies (from 1989 to 2009) were searched for by 2 reviewers on Medline, the Cochrane library, Embase, clinicaltrials.gov, Current Controlled Trial, bibliographies and by mailing key organisations and researchers. RCTs and observational studies on fluoxetine or venlafaxine in first-line treatment for major depressive disorder reported in English, French or Spanish language were included in the main analysis. Studies including patients from a wider spectrum of depressive disorders (anxious depression, minor depressive episode, dysthymia) were added in a second analysis. The main outcome was the pre-/post-treatment difference on depression scales standardised to 100 (17-item or 21-item Hamilton Rating Scale for Depression or Montgomery and Åsberg Rating Scale) in each study arm. A meta-regression was conducted to adjust the comparison between observational studies and RCTs on treatment type, study characteristics and average patient characteristics. 12 observational studies and 109 RCTs involving 6757 and 11035 patients in 12 and 149 arms were included in the main analysis. Meta-regression showed that the standardised treatment response in RCTs is greater by a magnitude of 4.59 (2.61 to 6.56). Study characteristics were related to standardised treatment response, positively (study duration, number of follow-up assessments, outpatients versus inpatients, per protocol analysis versus intention to treat analysis) or negatively (blinded design, placebo design). At patient level, response increased with baseline severity and decreased with age. Results of the second analysis were consistent with this.Response to antidepressants is greater in RCTs than in observational studies. Observational studies should be considered as a necessary complement to RCTs