Eccleston, Christopher

Fisher, Emma

Hearn, Leslie

Moore, R. Andrew

O'Connell, Neil E.

Stewart, Gavin

Williams, Amanda C.de C.

English

OPUS

        Citation for published version:Williams, ACDC, Hearn, L, Moore, RA, Stewart, G, Fisher, E, Eccleston, C & O'Connell, NE 2023, 'Effectivequality control in the medical literature: investigation and retraction vs inaction', Journal of Clinical Epidemiology,vol. 157, pp. 156-157. https://doi.org/10.1016/j.jclinepi.2023.02.022DOI:10.1016/j.jclinepi.2023.02.022Publication date:2023Document VersionPeer reviewed versionLink to publicationPublisher RightsCC BY-NC-NDhttps://doi.org/10.1016/j.jclinepi.2023.02.022University of BathAlternative formatsIf you require this document in an alternative format, please contact:openaccess@bath.ac.ukGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Download date: 08. Jul. 2024Effective quality control in the medical literature: investigation and retraction vs inaction  Williams ACdeC, Hearn L, Moore RA, Stewart G, Fisher E, Eccleston C, O’Connell NE  Authors: Amanda C de C Williams, PhD,  Research Dept of Clinical, Educational & Health Psychology, University College London, London, UK  ORCID: 0000-0003-3761-8704 Leslie Hearn, MSc,  Cochrane Pain, Palliative and Supportive Care Review Group, Oxford University Hospitals, Oxford, UK   ORCID: 0000-0003-4695-1744 R Andrew Moore, DSc,  Court Road, Newton Ferrers, Plymouth, UK      ORCID: 0000-0002-7932-5136 Gavin Stewart, PhD,  School of Natural and Environmental Sciences, University of Newcastle upon Tyne, Newcastle, UK        ORCID: 0000-0001-5684-1544 Emma Fisher, PhD,  Centre for Pain Research, University of Bath, Claverton Down, Bath, UK   ORCID: 0000-0001-8980-3181     Christopher Eccleston, PhD,  Centre for Pain Research, University of Bath, Claverton Down, Bath, UK  ORCID: 0000-0003-0698-1543 Neil E O’Connell, PhD,  Centre for Health and Wellbeing Across the Lifecourse, Department of Health Sciences, Brunel University London, Uxbridge, UK ORCID: 0000-0003-1989-4537 Corresponding Author: Amanda C de C Williams  Research Dept of Clinical, Educational & Health Psychology, University College London, Gower St, London WC1E 6BT, UK   amanda.williams@ucl.ac.uk   tel: 02076791608  Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Disclosures and competing interests: None of the authors has any financial interest in this work. Several authors are on the editorial boards of pain journals referred to in this paper (ACdeCW, CE) and of other pain journals (ACdeCW, NO’C, GS, EF, CE). Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research. Word count: 735 Tables and figures: 0  References: 13        Keywords Ethics; publishing; chronic pain  Authors’ contributions: AW Conceptualization, investigation, writing - original draft LH Conceptualization, investigation, writing – review and editing AM Conceptualization, investigation, writing – review and editing GS Conceptualization, investigation, writing – review and editing EF Conceptualization, writing – review and editing CE Conceptualization, writing – review and editing NO’C Conceptualization, writing – review and editing    Dear Editor Effective quality control in the medical literature: investigation and retraction vs inaction How should editors and publishers act on concerns raised about the trustworthiness of published papers? Despite the Committee on Publication Ethics (COPE) [1] guidelines, journal responses can be slow, inadequate, and opaque [2]. The potential harms of failure to retract untrustworthy studies - to clinical practice and public confidence - are considerable. We describe here our experience with six journal editors with whom we raised concerns. In 2022 we published our doubts about the integrity of several trials [3,4]. We had noticed three divergent trials with the same first author in a large systematic review and meta-analysis of psychological treatments for chronic pain [5]. The first author was unable to explain the extraordinarily positive results; after sensitivity analyses, we excluded them from meta-analyses. In a subsequent study, we used the Cochrane risk of bias tool [6] and the Cochrane Pregnancy and Childbirth Review Group Trustworthiness Screening Tool (CPC-TST) to evaluate trustworthiness [7] of trials of spinal pain by this author group. For eight of the 10 trials in six journals, from 2012 to 2021 [4], applying the CPC-TST identified concerns about trustworthiness. None was prospectively registered; protocols, ethical approval and data were not provided on request; baseline differences appeared inconsistent with randomization (see also Bolland et al. [8] on anomalous baseline data by the same first author in 17 trials); some data were identical or highly similar across nominally independent studies; results of all were judged implausible. Given our findings, we emailed the relevant editors, attaching our paper, asking that they consider retraction.  Three editors acted consistently with COPE guidelines [1]. Two immediately initiated investigations, informed us of progress, and ultimately retracted the trials (one each). The third journal editor, with two published trials (one of major concern), also investigated, resulting in retraction by the authors of the problematic trial for errors in data collection; this journal added a link to our trustworthiness paper to the trial of less concern and published an editorial on ways to improve research integrity [9]. The other three editors seemed reluctant to act.  The editor of the journal with four trials contacted the first author, receiving notice of his long-term sickness. Since this author had subsequently responded to other editors, we recommended further approaches, but heard nothing more.  Another editor advised us to address our concerns to the first author and his employer. Having previously been unsuccessful in this, we reminded him of the duties of editors outlined by COPE, and updated him on other editors’ initiatives. Following this, he wrote to the first author, receiving some data from the trial concerned, with complaints about our “unjustifiable attack”. These data, forwarded to us, confirmed our concerns of implausibility.  The sixth editor was reassured by his journal’s “fair review process by three experts”, so that “the results are likely to be useful to other researchers in the field”, and professed to being ”against academic intimidation”. We responded that unreliable results, which we already knew were included in multiple systematic reviews and clinical guidelines [10], were misleading rather than useful, while peer review of single papers would not identify duplicated data across trials.  After further correspondence with the first author, this editor decided against retraction but invited us to write to the journal expressing our concerns, for publication with a response from the first author: “this may ensure full transparency and promote critical debate.” We declined the offer, explaining that this was not an intellectual difference but an editorial responsibility. We have no evidence of any formal investigation. This journal's ethical policy includes the statement: “Despite vigorous peer-review, it is possible that a paper that is fraudulent in some manner may be published. If this is discovered, it will be retracted” [11]. What have we learned? That open science practices, including publishing data, cannot come too soon; that we need better tools to detect problematic data within and across studies; that while error and fraud are acknowledged as widespread, some editors appear disinclined to believe that their journals are affected; that whistleblowers may still need to pursue concerns that are editors’ responsibilities [12]. We know data can be flawed, futile, or fabricated, with documented examples in pain research [13]. Left unaddressed by those tasked with tackling them, such data go on to influence evidence reviews and guidelines, and increase the risk of harm to patients. We no longer worry about being too sceptical – we worry that we are not sceptical enough.  Williams ACdeC, Hearn L, Moore RA, Stewart G, Fisher E, Eccleston C, O’Connell NE  1 Committee on Publication Ethics. COPE flowcharts: Handling of post-publication critiques v1. 2021. https://publicationethics.org/sites/default/files/handling-post-publication-critiques-cope-flowchart.pdf. Accessed 31 Dec 2022. 2 Holbeach N, Freckelton I, Mol BW. Journal editors and publishers’ legal obligations with respect to medical research misconduct. Res Ethics 2022, 25 December. DOI: 10.1177/17470161221147440 3 O’Connell NE, Moore RA, Stewart G, Fisher E, Erskine E, Hearn L, Eccleston C, Williams ACdeC. Investigating the veracity of a sample of published trials with divergent results in spinal pain. OSF Registries https://osf.io/345vq 4 O’Connell NE, Moore RA, Stewart G, Fisher E, Hearn L, Eccleston C, Williams ACdeC. Investigating the veracity of a sample of divergent published trial data in spinal pain. PAIN 2023;164:72–83. Doi:10.1097/j.pain.0000000000002659 5 Williams ACDC, Fisher E, Hearn L, Eccleston C. Psychological therapies for the management of chronic pain (excluding headache) in adults. Cochrane Database Syst Rev 2020, Issue 8. Art. No.: CD007407. doi: 10.1002/14651858.CD007407.pub4 6 Higgins JPT, Altman DG, Sterne JAC. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration, updated March 2011. 7 Alfirevic Z, Kellie FJ, Stewart F, Jones L, Hampson L, on behalf of Pregnancy & Childbirth Editorial Group. Identifying and handling potentially untrustworthy trials in Pregnancy & Childbirth Cochrane Reviews. https://pregnancy.cochrane.org/sites/pregnancy.cochrane.org/files/public/uploads/identifying_and_handling_potentially_untrustworthy_trials_v_2.4_-_20_july_2021.pdf. Accessed January 5 2023.  8 Bolland MJ, Gamble GD, Avenell A, Cooper DJ, Grey A. Distributions of baseline categorical variables were different from the expected distributions in randomized trials with integrity concerns. J Clin Epidemiol 2023, in press. doi: https://doi.org/10.1016/j.jclinepi.2022.12.018. 9 Negrini S, Ceravolo MG, Ferriero G. Trust/untrust is not the same as true/false. Lessons learned and ethical questions on the application of untrustworthiness scales to judge individuals. Eur J Phys Rehab Med 2022;Nov 03. Doi:10.23736/S1973-9087.22.07767-X  10 O’Connell NE, Moore RA, Stewart G, Fisher E, Hearn L, Eccleston C, Wewege M, Williams ACdeC. Trials we cannot trust: investigating their impact on systematic reviews and clinical guidelines in spinal pain. Submitted. OSF record https://osf.io/m92ax/ 11 European Journal of Pain Ethical Policies. https://onlinelibrary.wiley.com/page/journal/15322149/homepage/forauthors.html. Accessed 5th January 2023.  12 Bolland MJ, Grey A, Avenell A, Klein AA. Correcting the scientific record – A broken system? Accountability in Research 2021;28(5):265-79. DOI: 10.1080/08989621.2020.1852938 13 Moore A, Fisher E, Eccleston C. Flawed, futile, and fabricated – features that limit confidence in clinical research in pain and anaesthesia: a narrative review. Brit J Anaesth Nov 8 2022. https://doi.org/10.1016/j.bja.2022.09.030  

Effective quality control in the medical literature:investigation and retraction vs inaction

https://purehost.bath.ac.uk/ws/files/270870261/Letter_subm_JCE_journals_retraction_Williams_et_al.pdf

Effective quality control in the medical literature:investigation and retraction vs inaction

Abstract

Similar works

Full text

Available Versions

OPUS