Dwan, Kerry

Flemyng, Ella

Higgins, Julian Pt

Moore, Theresa Hm

Page, Matthew J

English

Explore Bristol Research

                          Flemyng, E., Dwan, K., Moore, T. H., Page, M. J., & Higgins, J. P.(2020). Risk of Bias 2 in Cochrane Reviews: a phased approach forthe introduction of new methodology. Cochrane Database ofSystematic Reviews, 10, ED000148.https://doi.org/10.1002/14651858.ED000148Publisher's PDF, also known as Version of recordLicense (if available):UnspecifiedLink to published version (if available):10.1002/14651858.ED000148Link to publication record in Explore Bristol ResearchPDF-documentThis is the final published version of the article (version of record). It first appeared online via Wiley athttps://doi.org/10.1002/14651858.ED000148.Please refer to any applicable terms of use of the publisherUniversity of Bristol - Explore Bristol ResearchGeneral rightsThis document is made available in accordance with publisher policies. Please cite only thepublished version using the reference above. Full terms of use are available:http://www.bristol.ac.uk/red/research-policy/pure/user-guides/ebr-terms/Trusted evidence.Informed decisions.Better health.  Cochrane Database of Systematic ReviewsEDITORIALRisk of Bias 2 in Cochrane Reviews: a phasedapproach for the introduction of newmethodologyElla Flemyng, Kerry Dwan, Theresa HM Moore, Matthew J Page, Julian PT HigginsCochrane Database of Systematic Reviews 2020;(10):ED000148 https://doi.org/10.1002/14651858.ED000148Publication date: 19 November 2020 Our confidence in the results of research studies is a.ectedby how they are designed, conducted and reported. In idealcircumstances, all studies would accurately determine thetrue e.ect of a health care intervention. However, studiesdo not take place in an ideal world, and flaws or problemsin the way they are conducted could lead to them eitheroverestimating or underestimating the true e.ect of theintervention. Understanding the potential flaws or problems instudies included in systematic reviews is important as they candistort or bias the results, which means the findings might notaccurately represent the truth. Assessing the design, conduct andreporting of studies included in Cochrane Reviews is thereforean important step in understanding whether there is any riskof bias in the results, supporting our confidence in the review'sconclusions.Since 2008, authors of Cochrane Reviews have been expected touse the Cochrane risk of bias tool to assess the risk of bias in anyrandomized trials that they identify, to help users understand thetrustworthiness of the review findings. Formal evaluations of userpractice over the last decade revealed a need to improve the tool.For example, some bias domains were considered challengingfor users (particularly incomplete outcome data and selectiveoutcome reporting), the tool was frequently implemented innon-standard ways (e.g. with important domains omitted orinappropriate domains added), and the tool was not well suitedto cluster randomized trials and crossover trials.[1][2] To addressthese limitations and to reflect current understanding of how thecauses of bias can influence study results, a new version of thetool, Risk of Bias 2 (RoB 2), was launched in 2019 and includedin version 6 of the Cochrane Handbook of Systematic Reviews ofInterventions.[3][4]The revised tool is structured into five domains of bias, accordingto the stages of a trial in which problems may arise: (1) therandomization process; (2) deviations from intended intervention;(3) missing outcome data; (4) measurement of the outcome;and (5) selection of the reported result. The judgement for eachdomain is ‘low risk of bias’, ‘some concerns’, or ‘high risk of bias’.In addition, the same three judgement options are available foroverall risk of bias. The assessments apply to a specific resultfrom the trial, rather than to the study as a whole. An importantinnovation is to specify whether the result being assessed is beinginterpreted as the e.ect of assigning participants to interventions(the intention-to-treat e.ect) or to the e.ect of participantsadhering to their assigned intervention according to the trialprotocol (the per-protocol e.ect). The implications of deviationsfrom the intended intervention are di.erent between these,particularly for trials in which participants are aware of theirintervention.The implementation of RoB 2 has been a strategic priorityfor Cochrane and was included in the Strategy to 2020(www.cochrane.org/about-us/strategy-to-2020) as part of the goalto develop and continue to produce high-quality, relevant, up-to-date systematic reviews. The first Cochrane Review that usesRoB 2 has recently been published.[5] It follows new guidancefor reporting RoB 2 assessments in Protocols and Reviews, andillustrates new designs for interactive, results-level tables forthese assessments.The review by Williams and colleagues examined the e.ectsof physical activity interventions for people with congenitalheart disease. The RoB 2 tool was used to examine resultsof the included trials in relation to five outcomes: maximalcardiorespiratory fitness; submaximal cardiorespiratory fitness;health-related quality of life; physical activity; and muscularstrength. The risks of bias were judged by the review team tobe similar across these outcomes and in most cases the resultswere rated overall as having ‘some concerns’. The review teamstated that trialists had not reported well the information theyneeded to judge bias in measurement of the outcome (Domain4), which includes questions concerning blinding of outcomeassessors. Nor had most trials presented su.icient detail in anypre-specified statistical analysis plans to allow assessment ofbias in selection of the reported result (Domain 5). Health-relatedquality of life was, however, judged to be at high risk of bias acrossall randomized trials that reported it: it was self-reported with alack of blinding of outcome assessors, and this was thought tohave the potential to a.ect how it was reported. This led to anassessment of ‘very low’ certainty in the health-related quality oflife outcome using GRADE, which was downgraded twice due torisk of bias (also referred to as study limitations). Other outcomesin the review were not downgraded for risk of bias.Risk of Bias 2 in Cochrane Reviews: a phased approach for the introduction of new methodology (Editorial) 1Copyright © 2020 The Cochrane Collaboration | Published by John Wiley & Sons, Ltd. | https://doi.org/10.1002/14651858.ED000148  www.cochranelibrary.com  Trusted evidence.Informed decisions.Better health.  Cochrane Database of Systematic ReviewsIntroducing new methods and tools for Cochrane Reviews isnot without its challenges. Cochrane has 52 Cochrane ReviewGroups (CRGs), with editors, sta., and authors who need to betrained and supported in building the confidence and expertisewithin their Groups. Updates to technology are generally needed,including amendments to Cochrane-owned technology (suchas RevMan) as well as to those with which we collaborate andthat authors frequently use, including Covidence, DistillerSR, andGRADEPro. Other updates to guidance, tools, and training arealso required, along with plans to ensure the method or tool isbeing used appropriately. For RoB 2, we decided that a phasedimplementation approach would be used, beginning with a pilotphase period during which authors and CRGs may opt in to usethe tool (methods.cochrane.org/news/implementation-risk-bias-2-cochrane). The aims were for a supported and gradual roll-out of RoB 2 in Cochrane Reviews to observe common issues asthey arose and to put training and support in place before scalingup to wider implementation.Led by members of the Cochrane Editorial and MethodsDepartment (EF and KD), the pilot phase is helping us understandand overcome obstacles, gather evidence on the usability of thetool, and understand the impact it has on technology, publishing,and presentation. Fundamentally, the pilot is helping us worktowards a streamlined and e.icient process for authors andeditors, from writing the protocol to publishing the full systematicreview using the RoB 2 tool (community.cochrane.org/news/what-you-need-know-about-risk-bias-2-rob-2-cochrane). As of23 October 2020, there were over 80 Cochrane Reviews across alleight of the CRG Networks using or planning to use RoB 2, with thenumber steadily increasing over time.As part of the pilot phase, the Cochrane Methods Support Unit(methods.cochrane.org/about-us/cochrane-central-executive-methods-team/methods-support-unit) will work with CRGs toensure RoB 2 is being used accurately in protocols and reviews.As well as providing feedback directly on protocols and reviews,the Methods Support Unit are coaching editors and sta. on whatto look for, helping build methodological expertise within theCRGs. Updates on implementation and access to key resources forusing RoB 2 in Cochrane Reviews, such as an introductory leaflet,starter pack, and FAQs, can be found on the Cochrane Methodswebsite (methods.cochrane.org/risk-bias-2). Hosting in-personand virtual training has also been important, including a RoB 2Cochrane Learning Live Webinar Series (training.cochrane.org/rob-2-learning-live-webinar-series). The pilot phase allows theEditorial and Methods Department and the Bias Methods Group tounderstand gaps in guidance and support in real time so that wecan ensure they are addressed as soon as possible.RoB 2 provides a more appropriate way to assess risk of bias inrandomized trials and should help review authors draw moreappropriate conclusions about the included evidence. Thephased implementation of RoB 2 is ensuring we understand whatchallenges the improved tool may pose to authors and editorsso that the necessary guidance and infrastructure can be putin place to alleviate them. Review teams who plan to use RoB 2in a Cochrane Review are encouraged to contact the CochraneMethods team (methods@cochrane.org) so that they can benefitfrom additional guidance and support.Author InformationElla Flemyng1, Kerry Dwan1, Theresa HM Moore1,2, Matthew JPage3, Julian PT Higgins21Cochrane, UK. 2Population Health Sciences, University of Bristol,UK. 3School of Public Health and Preventive Medicine, MonashUniversity, AustraliaDeclarations of interestEF, KD and TM are employed and receive a salary from Cochrane.EF and KD are leading the RoB 2 implementation project 2 inCochrane, and TM is heavily involved in the project. JH and MP areConvenors of the Cochrane Bias Methods Group. JH is one of thelead developers of the Risk of Bias 2 tool.Provenance and peer reviewThis editorial was commissioned based on a proposal by theCochrane RoB research group, and it was not externally peer-reviewed.AcknowledgementsRoB 2 research group. Core group: Julian Higgins, JelenaSavović, Matthew Page, Asbjørn Hróbjartsson, Isabelle Boutron,Barney Reeves, Roy Elbers, Jonathan Sterne. Working Groupmembers: Doug Altman, Natalie Blencowe, Mike Campbell,Christopher Cates, Rachel Churchill, Mark Corbett, Nicky Cullum,Francois Curtin, Amy Drahota, Sandra Eldridge, JonathanEmberson, Bruno Giraudeau, Jeremy Grimshaw, Sharea Ijaz,Miguel Hernán, Sally Hopewell, Asbjørn Hróbjartsson, Peter Jüni,Jamie Kirkham, Toby Lasserson, Tianjing Li, Stephen Senn, SashaShepperd, Ian Shrier, Nandi Siegfried, Lesley Stewart, Kate Tilling,Ian White, Penny Whiting. And: Henning Keinke Andersen, VincentCheng, Mike Clarke, Jon Deeks, Daniela Junqueira, AlexandraMcAleenan, Geraldine Macdonald, Richard Morris, Mona Nasser,Nishith Patel, Jani Ruotsalainen, Holger Schünemann, JayneTierneyThe authors would like to thank Kayleigh Kew for her commentson an early draQ of this Editorial.References1. Savović J, Weeks L, Sterne JAC, Turner L, Altman DG,Moher D et al. Evaluation of the Cochrane Collaboration'stool for assessing the risk of bias in randomized trials: focusgroups, online survey, proposed recommendations andtheir implementation. Systematic Reviews 2014;3:37. https://doi.org/10.1186/2046-4053-3-372. Jørgensen L, Paludan-Müller AS, Laursen DR, Savović J,Boutron I, Sterne JAC et al. Evaluation of the Cochrane tool forassessing risk of bias in randomized clinical trials: overview ofpublished comments and analysis of user practice in Cochraneand non-Cochrane reviews. Systematic Reviews 2016;5:80. https://doi.org/10.1186/s13643-016-0259-83. Sterne JAC, Savovič J, Page MJ, Elbers RG, Blencowe NS,Boutron I, et al. RoB 2: a revised tool for assessing risk of bias inrandomised trials. BMJ 2019;366:I4898. https://www.bmj.com/content/366/bmj.l4898Risk of Bias 2 in Cochrane Reviews: a phased approach for the introduction of new methodology (Editorial) 2Copyright © 2020 The Cochrane Collaboration | Published by John Wiley & Sons, Ltd. | https://doi.org/10.1002/14651858.ED000148  www.cochranelibrary.com  Trusted evidence.Informed decisions.Better health.  Cochrane Database of Systematic Reviews4. Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC.Chapter 8: Assessing risk of bias in a randomized trial. In: HigginsJPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, WelchVA (editors). Cochrane Handbook for Systematic Reviews ofInterventions version 6.0 (updated August 2019). Cochrane, 2019.Available from www.training.cochrane.org/handbook.5. Williams CA, Wadey C, Pieles G, Stuart G, Taylor RS, LongL. Physical activity interventions for people with congenitalheart disease. Cochrane Database of Systematic Reviews 2020;(10):CD013400. https://doi.org/10.1002/14651858.CD013400.pub2 Risk of Bias 2 in Cochrane Reviews: a phased approach for the introduction of new methodology (Editorial) 3Copyright © 2020 The Cochrane Collaboration | Published by John Wiley & Sons, Ltd. | https://doi.org/10.1002/14651858.ED000148  www.cochranelibrary.com  

Risk of Bias 2 in Cochrane Reviews:a phased approach for the introduction of new methodology

http://hdl.handle.net/1983/93aade17-8545-4e00-861f-e9f4d81376c4

Risk of Bias 2 in Cochrane Reviews:a phased approach for the introduction of new methodology

Abstract

Similar works

Full text

Available Versions

Explore Bristol Research