Core outcome sets and systematic reviews

Systematic reviews seek to bring together research evidence to answer the question for the review. The reviewers usually wish to compare, contrast and, if appropriate, combine the findings of the existing research studies. However, these intentions are often thwarted by inconsistencies in the outcomes that were measured and reported in the individual studies. This, in turn, makes it difficult for readers of the review to use it to make informed decisions and choices about health and social care. One solution is for trials in a particular topic area to measure and report a standardised set of outcomes, which would then be used in the review. Core outcome sets are a means of doing this, providing an agreed standardised collection of outcomes for measuring and reporting for a specific area of health. In this commentary, we argue for greater involvement of systematic reviewers in the development and implementation of core outcome sets. This might help with, for example, the selection of outcomes to include in the Summary of findings tables that provide users of the review with the key quantitative findings. Consideration of core outcome sets when reviewers register their topics with Cochrane Review Groups or in PROSPERO would also help reviewers to plan their reviews. A greater uptake of core outcome sets across research, including systematic reviews, would help towards the ultimate aim of improving health and well-being through improving health and social care

The need for core outcome sets in urological cancer research

Peer reviewedPublisher PD

Nonadherence to treatment protocol in published randomised controlled trials: a review.

RIGHTS : This article is licensed under the BioMed Central licence at http://www.biomedcentral.com/about/license which is similar to the 'Creative Commons Attribution Licence'. In brief you may : copy, distribute, and display the work; make derivative works; or make commercial use of the work - under the following conditions: the original author must be given credit; for any reuse or distribution, it must be made clear to others what the license terms of this work are.This review aimed to ascertain the extent to which nonadherence to treatment protocol is reported and addressed in a cohort of published analyses of randomised controlled trials (RCTs). One hundred publications of RCTs, randomly selected from those published in BMJ, New England Journal of Medicine, the Journal of the American Medical Association and The Lancet during 2008, were reviewed to determine the extent and nature of reported nonadherence to treatment protocol, and whether statistical methods were used to examine the effect of such nonadherence on both benefit and harms analyses. We also assessed the quality of trial reporting of treatment protocol nonadherence and the quality of reporting of the statistical analysis methods used to investigate such nonadherence. Nonadherence to treatment protocol was reported in 98 of the 100 trials, but reporting on such nonadherence was often vague or incomplete. Forty-two publications did not state how many participants started their randomised treatment. Reporting of treatment initiation and completeness was judged to be inadequate in 64% of trials with short-term interventions and 89% of trials with long-term interventions. More than half (51) of the 98 trials with treatment protocol nonadherence implemented some statistical method to address this issue, most commonly based on per protocol analysis (46) but often labelled as intention to treat (ITT) or modified ITT (23 analyses in 22 trials). The composition of analysis sets for their benefit outcomes were not explained in 57% of trials, and 62% of trials that presented harms analyses did not define harms analysis populations. The majority of defined harms analysis populations (18 out of 26 trials, 69%) were based on actual treatment received, while the majority of trials with undefined harms analysis populations (31 out of 43 trials, 72%) appeared to analyse harms using the ITT approach. Adherence to randomised intervention is poorly considered in the reporting and analysis of published RCTs. The majority of trials are subject to various forms of nonadherence to treatment protocol, and though trialists deal with this nonadherence using a variety of statistical methods and analysis populations, they rarely consider the potential for bias introduced. There is a need for increased awareness of more appropriate causal methods to adjust for departures from treatment protocol, as well as guidance on the appropriate analysis population to use for harms outcomes in the presence of such nonadherence

A framework for the design, conduct and interpretation of randomised controlled trials in the presence of treatment changes.

BACKGROUND: When a randomised trial is subject to deviations from randomised treatment, analysis according to intention-to-treat does not estimate two important quantities: relative treatment efficacy and effectiveness in a setting different from that in the trial. Even in trials of a predominantly pragmatic nature, there may be numerous reasons to consider the extent, and impact on analysis, of such deviations from protocol. Simple methods such as per-protocol or as-treated analyses, which exclude or censor patients on the basis of their adherence, usually introduce selection and confounding biases. However, there exist appropriate causal estimation methods which seek to overcome these inherent biases, but these methods remain relatively unfamiliar and are rarely implemented in trials. METHODS: This paper demonstrates when it may be of interest to look beyond intention-to-treat analysis for answers to alternative causal research questions through illustrative case studies. We seek to guide trialists on how to handle treatment changes in the design, conduct and planning the analysis of a trial; these changes may be planned or unplanned, and may or may not be permitted in the protocol. We highlight issues that must be considered at the trial planning stage relating to: the definition of nonadherence and the causal research question of interest, trial design, data collection, monitoring, statistical analysis and sample size. RESULTS AND CONCLUSIONS: During trial planning, trialists should define their causal research questions of interest, anticipate the likely extent of treatment changes and use these to inform trial design, including the extent of data collection and data monitoring. A series of concise recommendations is presented to guide trialists when considering undertaking causal analyses

Evaluation of interventions for informed consent for randomised controlled trials (ELICIT) : protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey

Acknowledgements This work was supported by personal fellowship award (to KG) from the Medical Research Council’s Strategic Skills Methodology programme. The Health Services Research Unit is supported by a core grant from the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. PW is funded by a UK Medical Research Council Hub for Trials Methodology Research Network grant G0800792. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Chief Scientist Office, MRC or the Department of Health.Peer reviewedPublisher PD

A systematic review finds core outcome set uptake varies widely across different areas of health.

The aim of our review was to bring together studies that had assessed the uptake of core outcome sets (COS) to explore the level of uptake across different COS and areas of health. We examined the citations of 337 COS reports to identify studies that had assessed the uptake of a particular COS in randomised controlled trials (RCTs) or systematic reviews (SRs). We identified 24 studies that had assessed uptake in RCTs and two studies that has assessed uptake in SRs. The studies covered a total of 17/337 (5%) COS. Uptake rates reported for RCTs varied from 0% of RCTs (gout) to 82% RCTs (rheumatoid arthritis) measuring the full COS. Studies that assessed uptake of individual core outcomes showed wide variation in uptake between the outcomes. Suggested barriers to uptake included lack of validated measures, lack of patient and other key stakeholder involvement in COS development and lack of awareness of the COS. Few studies have been undertaken to assess the uptake of COS in RCTs and SRs. Further studies are needed to assess whether COS have been implemented across a wider range of disease categories and to explore the barriers and facilitators to COS uptake

Using the Delphi Technique to Determine Which Outcomes to Measure in Clinical Trials: Recommendations for the Future Based on a Systematic Review of Existing Studies

Ian Sinha and colleagues advise that when using the Delphi process to develop core outcome sets for clinical trials, patients and clinicians be involved, researchers and facilitators avoid imposing their views on participants, and attrition of participants be minimized

“Vicarious thinking” is a key driver of score change in Delphi surveys for COS development and is facilitated by feedback of results

Acknowledgements: We would like to thank and acknowledge the participants in the Delphi surveys used here. We also thank individuals who contributed significantly to the wider COS development projects: Professor Andrew G. Renehan and Professor Caroline Sanders (CORMAC); Dr. Thomas B. L. Lam (COMPACTERS); Professor Iain A Bruce and Professor Jane M Blazeby (GASTROS). Funding: This work was supported by a National Institute for Health Research (NIHR) Senior Investigator Award (NF-SI_0513-10025). The views expressed in this article are those of the authors and not necessarily those of the NIHR, or the Department of Health and Social Care.Peer reviewedPublisher PD

A randomised trial comparing three Delphi feedback strategies found no evidence of a difference in a setting with high initial agreement

Peer reviewedPostprintPostprintPostprintPostprintPostprintPostprintPostprintPostprintPostprintPostprintPostprin