Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in the BMJ and PLOS Medicine

Objectives To explore the effectiveness of data sharing by randomized controlled trials (RCTs) in journals with a full data sharing policy and to describe potential difficulties encountered in the process of performing reanalyses of the primary outcomes. Design Survey of published RCTs. Setting PubMed/Medline. Eligibility criteria RCTs that had been submitted and published by The BMJ and PLOS Medicine subsequent to the adoption of data sharing policies by these journals. Main outcome measure The primary outcome was data availability, defined as the eventual receipt of complete data with clear labelling. Primary outcomes were reanalyzed to assess to what extent studies were reproduced. Difficulties encountered were described. Results 37 RCTs (21 from The BMJ and 16 from PLOS Medicine) published between 2013 and 2016 met the eligibility criteria. 17/37 (46%, 95% confidence interval 30% to 62%) satisfied the definition of data availability and 14 of the 17 (82%, 59% to 94%) were fully reproduced on all their primary outcomes. Of the remaining RCTs, errors were identified in two but reached similar conclusions and one paper did not provide enough information in the Methods section to reproduce the analyses. Difficulties identified included problems in contacting corresponding authors and lack of resources on their behalf in preparing the datasets. In addition, there was a range of different data sharing practices across study groups. Conclusions Data availability was not optimal in two journals with a strong policy for data sharing. When investigators shared data, most reanalyses largely reproduced the original results. Data sharing practices need to become more widespread and streamlined to allow meaningful reanalyses and reuse of data

Validity of observational evidence on putative risk and protective factors:appraisal of 3744 meta-analyses on 57 topics

BACKGROUND: The validity of observational studies and their meta-analyses is contested. Here, we aimed to appraise thousands of meta-analyses of observational studies using a pre-specified set of quantitative criteria that assess the significance, amount, consistency, and bias of the evidence. We also aimed to compare results from meta-analyses of observational studies against meta-analyses of randomized controlled trials (RCTs) and Mendelian randomization (MR) studies. METHODS: We retrieved from PubMed (last update, November 19, 2020) umbrella reviews including meta-analyses of observational studies assessing putative risk or protective factors, regardless of the nature of the exposure and health outcome. We extracted information on 7 quantitative criteria that reflect the level of statistical support, the amount of data, the consistency across different studies, and hints pointing to potential bias. These criteria were level of statistical significance (pre-categorized according to 10-6, 0.001, and 0.05 p-value thresholds), sample size, statistical significance for the largest study, 95% prediction intervals, between-study heterogeneity, and the results of tests for small study effects and for excess significance. RESULTS: 3744 associations (in 57 umbrella reviews) assessed by a median number of 7 (interquartile range 4 to 11) observational studies were eligible. Most associations were statistically significant at P < 0.05 (61.1%, 2289/3744). Only 2.6% of associations had P < 10-6, ≥1000 cases (or ≥20,000 participants for continuous factors), P < 0.05 in the largest study, 95% prediction interval excluding the null, and no large between-study heterogeneity, small study effects, or excess significance. Across the 57 topics, large heterogeneity was observed in the proportion of associations fulfilling various quantitative criteria. The quantitative criteria were mostly independent from one another. Across 62 associations assessed in both RCTs and in observational studies, 37.1% had effect estimates in opposite directions and 43.5% had effect estimates differing beyond chance in the two designs. Across 94 comparisons assessed in both MR and observational studies, such discrepancies occurred in 30.8% and 54.7%, respectively. CONCLUSIONS: Acknowledging that no gold-standard exists to judge whether an observational association is genuine, statistically significant results are common in observational studies, but they are rarely convincing or corroborated by randomized evidence

The worldwide clinical trial research response to the COVID-19 pandemic - the first 100 days

Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited <20% of the targeted sample; 27 between 20-50%; and 30 over 50% (median 14.8% [IQR 2.0-62.0%]). Conclusions: The size and speed of the COVID-19 clinical trials agenda is unprecedented. However, most trials were small investigating a small fraction of treatment options. The feasibility of this research agenda is questionable, and many trials may end in futility, wasting research resources. Much better coordination is needed to respond to global health threats

Association between convalescent plasma treatment and mortality in COVID-19: a collaborative systematic review and meta-analysis of randomized clinical trials.

Funder: laura and john arnold foundationBACKGROUND: Convalescent plasma has been widely used to treat COVID-19 and is under investigation in numerous randomized clinical trials, but results are publicly available only for a small number of trials. The objective of this study was to assess the benefits of convalescent plasma treatment compared to placebo or no treatment and all-cause mortality in patients with COVID-19, using data from all available randomized clinical trials, including unpublished and ongoing trials (Open Science Framework, https://doi.org/10.17605/OSF.IO/GEHFX ). METHODS: In this collaborative systematic review and meta-analysis, clinical trial registries (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform), the Cochrane COVID-19 register, the LOVE database, and PubMed were searched until April 8, 2021. Investigators of trials registered by March 1, 2021, without published results were contacted via email. Eligible were ongoing, discontinued and completed randomized clinical trials that compared convalescent plasma with placebo or no treatment in COVID-19 patients, regardless of setting or treatment schedule. Aggregated mortality data were extracted from publications or provided by investigators of unpublished trials and combined using the Hartung-Knapp-Sidik-Jonkman random effects model. We investigated the contribution of unpublished trials to the overall evidence. RESULTS: A total of 16,477 patients were included in 33 trials (20 unpublished with 3190 patients, 13 published with 13,287 patients). 32 trials enrolled only hospitalized patients (including 3 with only intensive care unit patients). Risk of bias was low for 29/33 trials. Of 8495 patients who received convalescent plasma, 1997 died (23%), and of 7982 control patients, 1952 died (24%). The combined risk ratio for all-cause mortality was 0.97 (95% confidence interval: 0.92; 1.02) with between-study heterogeneity not beyond chance (I2 = 0%). The RECOVERY trial had 69.8% and the unpublished evidence 25.3% of the weight in the meta-analysis. CONCLUSIONS: Convalescent plasma treatment of patients with COVID-19 did not reduce all-cause mortality. These results provide strong evidence that convalescent plasma treatment for patients with COVID-19 should not be used outside of randomized trials. Evidence synthesis from collaborations among trial investigators can inform both evidence generation and evidence application in patient care

L'utilisation rationnelle des données disponibles avant d'extrapoler la balance bénéfice risque de l’adulte à l’enfant

Drug interventions are evaluated and receive a Marketing Authorization (MA) before being prescribed. They are generally evaluated in adult patients and then prescribed to children by extrapolating the treatment effect observed in adults. The extrapolation of the benefit risk ratio from adults to children occurs during drug development and when prescribing drugs (within the MA or off-label, which is frequent in children). This is due to the specific constraints of pediatric clinical research leading to a lack of data in children. A framework for extrapolation is currently being finalized by the European Medicines Agency. Using a meta-epidemiological approach, we explored the similarities and differences of the benefit, the benefit risk ratio and the perceived placebo effect between adults and children from meta-analysis including randomized double-blinded placebo-controlled trials evaluating a drug intervention in an indication in adults and children with separate data for both populations. We then built the effect model using adult data to predict the treatment effect in children and calibrate future pediatric clinical trials. Our research highlights the importance of using all available evidence before extrapolating the benefit risk ratio from adults to children and to justify new studies in the context of existing evidence. This approach allows to reduce unnecessary repetitions of clinical trials, to better allocate resources, to identify gaps in knowledge and thus optimize pediatric clinical research. More generally, it applies to any research allowing to avoid a waste in the time and resources investedLes médicaments sont évalués et reçoivent une autorisation de mise sur le marché (AMM) avant d'être prescrits. Ils sont généralement évalués chez des patients adultes, et utilisés chez les enfants en extrapolant les résultats obtenus chez les adultes. L'extrapolation de la balance bénéfice risque de l'adulte à l'enfant intervient lors du développement clinique du médicament et lorsqu'il est prescrit (dans l'AMM ou hors AMM, ce qui est fréquent chez l'enfant). Ceci est dû aux contraintes de la recherche clinique en pédiatrie, qui conduit à un manque de données chez l'enfant. Une recommandation sur l'extrapolation est en cours de finalisation par l'Agence Européenne du Médicament. En utilisant une approche méta-épidémiologique, nous avons exploré les similitudes ou différences du bénéfice, de la balance bénéfice risque et de l'évolution sous placebo entre adultes et enfants à partir de méta-analyses d'essais randomisés en double aveugle contre placebo, ayant inclus des adultes et des enfants dans des indications et avec des médicaments identiques, et présentant des données séparées chez l'adulte et l'enfant. Par la suite, nous avons construit le modèle d'effet à partir des données adultes et l'avons utilisé pour prédire l'effet du traitement et calibrer la taille de l'essai clinique pédiatrique. Ces travaux mettent en avant l'importance d'utiliser toutes les données disponibles avant d'extrapoler la balance bénéfice risque de l'adulte à l'enfant et de justifier les nouvelles études au regard des connaissances existantes. Cette démarche permet de réduire les répétitions inutiles d'essais cliniques, de mieux affecter les ressources destinées à la recherche, d'identifier les domaines pour lesquels les connaissances sont insuffisantes et ainsi optimiser la recherche clinique en pédiatrie. De manière plus globale, cela s'applique à tous types de recherche et permet d'éviter le gâchis au niveau du temps et des ressources investi

Umbrella Review of Umbrella Reviews

An umbrella review of umbrella reviews for non-randomized observational evidence on putative risk and protective factor

PragMS: Pragmatic trials in multiple sclerosis

Overview of pragmatic trials in multiple sclerosis

Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic?

Abstract Introduction Pragmatic randomized controlled trials (RCTs) mimic usual clinical practice and they are critical to inform decision-making by patients, clinicians and policy-makers in real-world settings. Pragmatic RCTs assess effectiveness of available medicines, while explanatory RCTs assess efficacy of investigational medicines. Explanatory and pragmatic are the extremes of a continuum. This debate article seeks to evaluate and provide recommendation on how to characterize pragmatic RCTs in light of the current landscape of RCTs. It is supported by findings from a PubMed search conducted in August 2017, which retrieved 615 RCTs self-labeled in their titles as “pragmatic” or “naturalistic”. We focused on 89 of these trials that assessed medicines (drugs or biologics). Discussion 36% of these 89 trials were placebo-controlled, performed before licensing of the medicine, or done in a single-center. In our opinion, such RCTs overtly deviate from usual care and pragmatism. It follows, that the use of the term ‘pragmatic’ to describe them, conveys a misleading message to patients and clinicians. Furthermore, many other trials among the 615 coined as ‘pragmatic’ and assessing other types of intervention are plausibly not very pragmatic; however, this is impossible for a reader to tell without access to the full protocol and insider knowledge of the trial conduct. The degree of pragmatism should be evaluated by the trial investigators themselves using the PRECIS-2 tool, a tool that comprises 9 domains, each scored from 1 (very explanatory) to 5 (very pragmatic). Conclusions To allow for a more appropriate characterization of the degree of pragmatism in clinical research, submissions of RCTs to funders, research ethics committees and to peer-reviewed journals should include a PRECIS-2 tool assessment done by the trial investigators. Clarity and accuracy on the extent to which a RCT is pragmatic will help understand how much it is relevant to real-world practice