Eccleston, C

Fisher, E

Hearn, L

Moore, RA

O'Connell, N

Stewart, G

Williams, ACdeC

Pain

Brunel University Research Archive

1 Title: Untrustworthy trials should be challenged if no satisfactory response is forthcoming:  a response to Monticone Neil Edward O’Connell, Centre for Health and Wellbeing Across the Lifecourse, Department of Health Sciences, Brunel University London, Uxbridge, UK R. Andrew Moore, Court Road, Newton Ferrers, Plymouth, UKGavin Stewart, School of Natural and Environmental Sciences, University of Newcastle upon Tyne,Newcastle, UKEmma Fisher, Centre for Pain Research, University of Bath, Claverton Down, Bath, UKLeslie Hearn, Cochrane Pain, Palliative and Supportive Care Review Group, Oxford UniversityHospitals, Oxford, UKChristopher Eccleston, Centre for Pain Research, University of Bath, Claverton Down, Bath, UKAmanda C de C Williams, Research Dept of Clinical, Educational & Health Psychology, UniversityCollege London, London, UKCorresponding author: Neil O’Connell, email neil.oconnell@brunel.ac.uk  Address: Department of Health Sciences, Brunel University, Kingston Lane, Uxbridge, Middlesex, UB83PH, UK Word Count: 733 words Number of text pages: 3 Number of tables: 0 Number of figures: 0 This is a non-final version of an article published in final form in O'Connell, Neil Edwarda,*; Moore, R. Andrewb; Stewart, Gavinc; Fisher, Emmad; Hearn, Lesliee; Eccleston, Christopherd; Williams, Amanda C. de C.f. Reply to Monticone. PAIN 164(4):p e239-e241, April 2023. | DOI: 10.1097/j.pain.00000000000028732     We acknowledge Dr Monticone’s letter in response to our paper [7] which investigated the trustworthiness of a group of studies published by his group. Contrary to Dr Monticone’s suggestion, we made no accusation of data fabrication or misconduct. Rather, we raise valid concerns regarding the trustworthiness of this group of trials that we judge to be important, in particular because their inclusion in reviews and clinical guidance risks over-confidence and error. What caused these trial reports to be submitted and accepted for publication is beyond the scope of our investigation.   To address Dr Monticone’s specific points:  a) We acknowledged the limitations of Stouffer’s method [2,9] in our paper. We ran and reported our analyses of baseline variables with and without categorical variables. Even in the latter analysis, omnibus p values diverged from what might be expected with randomisation in five of the ten included trials. b) We recognise and have corrected an error regarding the description of the normative data cohort [6]. Importantly, this does not alter our findings as we did not make any definitive risk judgements regarding the feasibility of participant characteristics. c) We are not aware of a clear theoretical or evidential basis that would predict much greater effectiveness of cognitive behavioural therapy (CBT) when added to exercise in the Italian context, compared with “Anglo-Saxon countries” and do not accept that the Italian setting of these trials offers a compelling explanation for the divergence of their results.  d) The apparent suggestion that CBT added to rehabilitation may have delivered greater benefits when constituting a smaller proportion of the overall intervention is interesting. One potential reason we can acknowledge is the potential for additive effects when CBT is added to rehabilitation. However, we note that the results of the trials authored by Monticone and colleagues also diverge substantially from those of a review [8] that specifically focused on the addition of psychological treatments to physical treatments versus physical treatments alone. e) It is common to analyse the included outcomes as though they were from interval scales. Had we analysed the data as ordinal, we would have observed the same degree of divergence. Our findings are robust to any reasonable analytical approach. f) Though the Trustworthiness Screening Tool [6] has not been formally validated, it allowed us to take a consistent, systematic approach across trials and probes a range of reasonable questions regarding research governance and data authenticity. g) We raised concerns in our paper regarding the low levels of attrition in some of the studies, which are striking. Trials commonly employ methods to improve retention similar to those described by Dr Monticone, but often with markedly lower success. h) As for the points regarding ethics, we requested evidence of ethical approvals, study protocols and individual patient data. This correspondence was included in our submitted draft but was removed during the editorial process. Dr Monticone declined to share the requested information with us. i) Dr Monticone states in his letter that protocols were developed. This contrasts with his earlier correspondence which informed us that there were no trial protocols because “the interventions belonged to our clinical practice”. This raises the question of what the Institutional Review Board actually approved. j) The information Dr Monticone reports regarding randomisation processes is useful and may explain the equal groups observed in each trial. This was present neither in the published trials nor in Dr Monticone’s earlier responses to our queries. 3  k) We are pleased to see incorrect data retracted from the literature. In earlier correspondence with Dr Monticone where we raised these anomalies, he expressed surprise at the duplication but indicated that he had checked the data and that they were what the staff had collected. l) In the examples of impossible p values that we raised in our paper, the means and standard deviations across the treatment groups were either identical or differed by one decimal place. On that basis, the reported p values cannot be correct.  Our investigation raised important concerns regarding the trustworthiness of these trials. The points raised in Dr Monticone’s letter, and in his responses to our prior correspondence, do not adequately address those concerns, nor satisfactorily explain the extremely divergent results in this group of trials. At the time of writing, three of the studies had been withdrawn [3-5]; we have invited the editors of three remaining journals to consider retracting the five further studies which we judged untrustworthy.  Acknowledgments: none.  Conflicts of interest: The authors have no conflicts of interest to declare.    References  [1] Cochrane Pregnancy and Childbirth Review Group. Identifying and handling potentially untrustworthy trials in Pregnancy and Childbirth Cochrane Reviews. https://pregnancy.cochrane.org/news/identifying-and-handling-potentially-untrustworthy-trialspregnancy- and-childbirth-cochrane. Accessed 16/9/21  [2] Heard NA, Rubin-Delanchy P. Choosing between methods of combining p-values  Biometrika 2018; 105: 1: 239–246  [3] Monticone M, Ambrosini E, Rocca B, Cazzaniga D, Liquori V, Pedrocchi A, et al. Group-based multimodal exercises integrated with cognitive-behavioural therapy improve disability, pain and quality of life of subjects with chronic neck pain: a randomized controlled trial with one-year follow-up. Clin Rehabil 2017;31(6):742-52. [CRSREF: 14279864]  [4] Monticone M, Ambrosini E, Rocca B, Cazzaniga D, Liquori V, Lovi A, et al. Multimodal exercises integrated with cognitive-behavioural therapy improve disability of patients with failed back surgery syndrome: a randomized controlled trial with one-year follow-up. Disabil Rehabil 2020;27:1-8.  [5] Monticone M, Ambrosini E, Portoghese I, Rocca B. Multidisciplinary program based on early management of psychological factors reduces disability of patients with subacute low back pain. Results of a randomised controlled study with one year follow-up. Eur J Phys Rehabil Med 2021;57(6):959-67.  [6] Nicholas MK, Asghari A, Blyth FM. What do the numbers mean? Normative data in chronic pain measures. Pain. 2008;134(1-2):158-7.  [7] O'Connell NE, Moore RA, Stewart G, Fisher E, Hearn L, Eccleston C, Williams ACC. Investigating the veracity of a sample of divergent published trial data in spinal pain. Pain (2022 Apr 25 online ahead of print). 4   [8] O’Keeffe M, Purtill H, Kennedy N, Conneely M, Hurley J, O'Sullivan P, Dankaerts W, O'Sullivan K. Comparative Effectiveness of Conservative Interventions for Nonspecific Chronic Spinal Pain: Physical, Behavioral/ Psychologically Informed, or Combined? A Systematic Review and Meta- Analysis. J Pain 2016;17(7):755-74. doi: 10.1016/j.jpain.2016.01.473   [9] Stouffer SA, Suchman EA, DeVinney LC, Star SA, Williams RM Jr (1949) The American Soldier, Vol. 1 - Adjustment during Army Life. Princeton, Princeton University Press 

Reply to Monticone

http://bura.brunel.ac.uk/bitstream/2438/26277/3/FullText.pdf

Reply to Monticone

Abstract

Similar works

Full text

Available Versions

Brunel University Research Archive