Treasure, T

English

UCL Discovery

1  The risk of editorial complicity in publishing claims for unproven treatments  Tom Treasure MS MD FRCS FRCP  Professor of Cardiothoracic Surgery University College London, UK tom.treasure@gmail.com   2  Conducting and publishing a survey of top cited authors in science, Gregory Patience, an academic chemical engineer, drew attention to areas of disquiet.(1) At the subsequent World Forum of Principles of Scientific Publication, Patience taught us in his lectures that there is a gradation in the nature of proof across different scientific disciplines. Among mathematicians, hypotheses appear to be amenable to absolute and ultimately incontestable proof. Cautiously I have inserted “appear to be” because, for example, Andrew Wiles’ proof of Fermat’s Last Theorem is beyond my comprehension(2) but I take it on trust that, due to this paper, mathematicians see the theorem as proven. Patience further suggest that physics comes next with its theories amenable to reasoned proof. More open to interpretation, and thus prone to uncertainty are conclusions from Chemistry research findings—his own field—but when you come to biology he says “it’s all over the place”.   In biology the interplay of so many factors, and natural variation means that we rely very often on empirical evidence. Conclusions may be debatable and are often contested. But objectivity should remain. Medicine is further down the gradient of provability. Objectivity is easily lost because we are all at one time or another sufferers from disease and we place our hope and trust in treatments. Subjectivity intrudes on judgement.   For surgical procedures when the patient’s problem and the surgeon’s solution are mechanistically and temporarily linked the efficacy of treatment may be self-evident. Just as a carpenter, plumber, or car mechanic takes something broken or worn out and fixes it, the problem is obvious and the surgeon sorts it out. Operations such as removing a cataract or fixation of a hip fracture brought biblical miracles—“The blind receive their sight, and the lame walk”—into everyday life.(3) The immediate and sustained benefits of vision and mobility were all the proof that was needed.   Nowadays in surgery we are looking for much smaller differences such as the relative benefits of one strategy, procedure or device over another, and the timescale over which we need to assess the outcome is measured in years for really worthwhile operations. The immediate effect may be seen instantly. That might be called “efficacy” in the terminology of Evidence Based Medicine to distinguish these short-term effects from “clinical effectiveness”. With hip replacement, pain is relieved and mobility is restored but clear the difference between a prosthetic hip which will last five, ten, fifteen of twenty years is of great importance to the seventy-year-old who receives it. The same applies to heart valves and breast implants. Such wear and tear results are probably best assessed by registries and post marketing surveillance.  In surgery one individual’s outcome is in the hands of another. The expectation of being made better by an operation is an expression of belief and trust in the knowledge, judgement and expertise of the surgeon. In turn, for a surgeon to agree to operate on a fellow human being requires a firm belief that the operation will help. The surgeon must also be self-assured and have a sense of entitlement to take up the knife. Ethos and Pathos come to the fore and Logos may drop out of sight. The same occurs in other treatments where the therapist and therapy are indivisible, such as psychotherapy or homeopathy. Although the balance of consequences—benefit versus harm—may be less dramatic, subjectivity on the part of both the therapist and the patient impair objectivity about judging effectiveness. It is inescapable that in presenting the results of their own work, authors are able to weave in unconscious bias at every step from the design of the study, through the measurement of outcomes to the interpretation of their results.  3   I will give examples where there was a conflict between new research evidence and those involved with an existing practice which the evidence called into question.   Mesothelioma and Radical Surgery: “MARS” randomised controlled trial It is well known that asbestos is bad for the lungs. The worst of its harmful effects is a cancer called malignant pleural mesothelioma. It typically makes its presence known about 40 years after the worst of the exposure.(4) Mesothelioma spreads along the pleura—the membrane enclosing the lung—getting its claws into the lung beneath and the chest wall covering it. Cure by surgery remains improbable but a US surgeon revived in the 1990s(5) an operation abandoned as futile in the 1970s.(6) The operation called extrapleural pneumonectomy entails removal of a whole lung, the membranes surrounding it, the adjacent part of the membrane around the heart, and the diaphragm. Chemotherapy and post-operative radiotherapy are added to make “trimodality therapy”.(7) It is a brutal treatment occupying 6 to 9 months with a high death rate. Fewer than four out of ten patients were alive after two years. None were cured. The surgeon travelled the world promoting his method of treatment and patients were mortgaging their houses to go to a private hospital in hope of cure. So grim was this treatment that the suffering of a patient and impoverishment of her family became the subject of a novel “So Much for That”.(8)  In Britain from 2005-2008 we ran a randomised controlled trial (RCT) to find out if the operation improved survival. When we had 25 patients in each arm the independent Data Monitoring Committee stopped the trial because the patients assigned to operation died significantly sooner and with worse quality of life than those in the unoperated control group.(9) Undaunted, a group of adopters of the radical operation, self-styled as the International Mesothelioma Interest Group, wrote an article finding fault with the trial. This article was peer reviewed and rejected by a leading cancer journal. Nevertheless it was published in a thoracic surgical journal under the banner headline ‘Clinical Guidelines’.(10) In my view it was a distorted and erroneous contradiction to the best available form of evidence, an RCT. The late David Sugarbaker worked in the same hospital as the editor of the journal. His riposte bypassed the reviewing process and the editor. Sugarbaker’s faith in his surgery was deep and personal and he resented it being questioned writing on one occasion “Treasure doesn’t do the operation”. Quite so. I didn’t share his conviction and he resented my seeking evidence.  Pulmonary Metastasectomy in Colorectal Cancer: the PulMiCC Trial Colorectal cancer is curable by surgery. By “cure” I mean that the patient remains free of cancer and lives to die of something else without evident cancer. Cancers may be indolent and continue insidiously to become evident later. For that reason, those who choose their words carefully, talk about the cancer being in remission. Colorectal cancer patients have about a 60% chance of living free of cancer to die of something else.   In perhaps a third of patients it has already spread in the blood stream. The easiest place to see the secondary growths—called metastases—is in the lung and the practice of removing them—metastasectomy—became established in the 1960s.(11) This was encouraged by a self-designated group of practitioners who collected uncontrolled observational data in the International Registry of Lung Metastases.(12) Only about one in thirty in 30 patients with lung metastases actually have a metastasectomy. These are patients with the most favourable predictive factors. We questioned whether the apparently better-than-expected survival which 4  encouraged this practice was in fact due to selection of those more likely to live longer rather than an effect of the operation. Our randomised trial showed no survival difference between the operated patients and the controls. We sent it to a thoracic surgery journal who held it for a couple of months before rejecting it in February 2019. The journal is the official organ of the US based Society of Thoracic Surgeons (STS). In the same month the journal published a Consensus Statement from the STS Work Force of Evidence Based Surgery. Its first page makes two assertions. First, that there had not been a single randomised controlled trial on lung metastasectomy and second, that it could be assumed that survival without the operation would have been zero.(13) The journal had just rejected the report of a randomised controlled trial that falsified both statements.(14)  If for example, you are in the business of flying people around the world, you expect the metallurgy, chemistry of the fuel, physics of tyres and hydraulics, the oxygen, cabin pressure and the filtration systems, critical to your passengers’ and employees' safety, to be evidence based and open to scrutiny. "Science" in medicine falls far short of that principle. The surgical specialist journals are the most vulnerable because they are edited by a fellow specialist, who gets reviews from other specialists, and all their opinions are coloured by their beliefs, prejudices and cognitive biases. In this case ego, status and the marketplace seemed to rank above both plausibility, rational reasoning and randomised controlled trial evidence. I cannot imagine that a consensus "assumption" would be okay in any other supposedly science-based discipline. "Show us the measurements and your calculations" they would say. Cessation of an established practice when it has been put to the test of a randomised trial has been seen repeatedly.  A well-known example is the RCT that stopped radical mastectomy for breast cancer after 90 years of mutilating surgery. Much less radical surgery had no worse outcomes.(15) From 2001 to 2010 a study found that 146 controlled trials had contradicted established practices.(16) And yet cancer teams resist putting surgical treatments to the test of randomised trials.(17) Drugs cannot be introduced into practice without randomised trials and we have seen during the pandemic that the delays and obstacles to doing randomised trials were overcome and the time-consuming processes can be concertinaed.  Radiotherapy rather than surgery for lung cancer A third example features specialists factions competing for business, and in private practice, particularly in the US model, that is what it is. Radiotherapy is an alternative to cutting cancer out. It has the advantage of avoiding a surgical wound and with modern stereotactic methods (SABR/SBRT) the therapist can concentrate an optimum dose on a tumour within the lung while limiting radiation damage to the surrounding lung tissue that the patient needs to breathe with. But surgery is the gold standard and only patients unfit or unsuitable for operation are advised by cancer multidisciplinary teams (MDTs) to be treated in that way by specialist Clinical (UK) or Radiation (US) oncologists.  Attempts were made to enter patients in to RCTs but it was very difficult for MDTs to declare surgery and radiotherapy as equivalent and so encourage a patient to allow random allocation. Two trials were too small but when the data were pooled it looked as though there was no detriment in allowing a patient to have radiotherapy rather than an operation (18) Invited to write a commentary for Lancet Oncology, I asked experts in lung cancer, a pulmonologist and an oncologist to join me in welcoming research seeking to spare patients surgery when it had no advantage over a less severe treatment.(19) The specialist editors awarded themself two articles occupying six pages to pour scorn on the pooled trial and our commentary.(20, 21) The seven authors were all thoracic surgeons, writing in a specialist journal to be read almost exclusively by other than thoracic surgeons. 5   Concerns about failings in impartiality in editorial decisions In the Montreal Forum scientists presented for discussion examples where they had been unfairly treated by journals. We discussed in breakout sessions how we as researchers should deal with such instances. It is not a level playing field. The peer reviewers are usually anonymous and frequently have undeclared financial and other vested interests. The editors of specialty journals, may also have prejudices and similar vested interests and has the final say. Authors have to be cautious about challenging editorial decisions, particularly young researchers building their careers. Junior colleagues are usually sagely advised to accept the rejection and try elsewhere. After all, they may want to send their next and better paper to the same journal. This is the diplomatic course of action but one consequence is that a flawed or unjust editorial decision goes uncorrected.  Comic opera In my three examples patients were disadvantaged and harmed and some lives were probably shortened by pointless operations. In 1996 Richard Horton, the editor of The Lancet challenged surgeons to get better evidence for what they do rather than relying on descriptive reports attributing benefit to their work without reliable controls. Usually without any controls at all. Horton dubbed it “comic opera”.(22) In my examples surgeons involved in these practices disputed the need for the trials, undermined them during their progress, and countermanded the findings of the peer reviewed publications. They were aided and abetted by specialist surgical journals publishing non-evidence based and unscientific articles along the way. Scientists contributing to the Montreal Forum “Principles of Scientific Publication” were aghast at these examples of editorial behaviour and urged me to go public.      6  References 1. Patience GS, Galli F, Patience PA, Boffito DC. Intellectual contributions meriting authorship: Survey results from the top cited authors across all science categories. PLoS One. 2019;14(1):e0198117. 2. Wiles A. Modullar ellipical curves and Fermat's last theorem. Annals of Mathematics. 1995;141:443-551. 3. The Gospel according to Matthew 5.11.  The King James Bible. 111611. p. 5. 4. Peto J, Hodgson JT, Matthews FE, Jones JR. Continuing increase in mesothelioma mortality in Britain. Lancet. 1995;345(8949):535-9. 5. Sugarbaker DJ, Garcia JP, Richards WG, Harpole DH, Jr., Healy-Baldini E, DeCamp MM, Jr., et al. Extrapleural pneumonectomy in the multimodality therapy of malignant pleural mesothelioma. Results in 120 consecutive patients. Ann Surg. 1996;224(3):288-94. 6. Butchart EG, Ashcroft T, Barnsley WC, Holden MP. Pleuropneumonectomy in the management of diffuse malignant mesothelioma of the pleura. Experience with 29 patients. Thorax. 1976;31(1):15-24. 7. Sugarbaker DJ, Flores RM, Jaklitsch MT, Richards WG, Strauss GM, Corson JM, et al. Resection margins, extrapleural nodal status, and cell type determine postoperative long-term survival in trimodality therapy of malignant pleural mesothelioma: results in 183 patients. J Thorac Cardiovasc Surg. 1999;117(1):54-63. 8. Shriver L. So much for that: Harper; 2011 2011. 9. Treasure T, Lang-Lazdunski L, Waller D, Bliss JM, Tan C, Entwisle J, et al. Extra-pleural pneumonectomy versus no extra-pleural pneumonectomy for patients with malignant pleural mesothelioma: clinical outcomes of the Mesothelioma and Radical Surgery (MARS) randomised feasibility study. Lancet Oncol. 2011;12(8):763-72. 10. Rusch V, Baldini EH, Bueno R, De Perrot M, Flores R, Hasegawa S, et al. The role of surgical cytoreduction in the treatment of malignant pleural mesothelioma: Meeting summary of the International Mesothelioma Interest Group Congress, September 11-14, 2012, Boston, Mass. J Thorac Cardiovasc Surg. 2013;145(4):909-10. 11. Thomford NR, Woolner L, Clagett O. The surgical treatment of metastatic tumours in the lung. J Thorac Cardiovasc Surg. 1965;49:357-63. 12. Pastorino U, McCormack PM, Ginsberg RJ. A new staging proposal for pulmonary metastases. The results of analysis of 5206 cases of resected pulmonary metastases. Chest Surg Clin N Am. 1998;8(1):197-202. 13. Handy JR, Bremner RM, Crocenzi TS, Detterbeck FC, Fernando HC, Fidias PM, et al. Expert Consensus Document on Pulmonary Metastasectomy. Ann Thorac Surg. 2019;107(2):631-49. 14. Milosevic M, Edwards J, Tsang D, Dunning J, Shackcloth M, Batchelor T, et al. Pulmonary Metastasectomy in Colorectal Cancer: updated analysis of 93 randomized patients - control survival is much better than previously assumed. Colorectal Dis. 2020;22(10):1314-24. 15. Mukherjee S. A Radical Idea in "The Emperor of All Maladies".  The Emperor of the Maladies. New York: Scribner; 2010. p. 60-72. 16. Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S, et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc. 2013;88(8):790-8. 17. Treasure T, F M, Baum M, M S, J E, T B. Randomising patients into trials of thoracic cancer surgery: an analysis of patient and cancer team behaviour in practice. 2019;Posters: 2.16-02:903. 18. Chang JY, Liu H, Balter P, Komaki R, Liao Z, Welsh J, et al. Clinical outcome and predictors of survival and pneumonitis after stereotactic ablative radiotherapy for stage I non-small cell lung cancer. Radiat Oncol. 2012;7:152. 19. Treasure T, Rintoul RC, Macbeth F. SABR in early operable lung cancer: time for evidence. Lancet Oncol. 2015;16(6):597-8. 7  20. Meyers BF, Puri V, Broderick SR, Samson P, Keogan K, Crabtree TD. Lobectomy versus stereotactic body radiotherapy for stage I non-small cell lung cancer: Post hoc analysis dressed up as level-1 evidence? J Thorac Cardiovasc Surg. 2015;150(3):468-71. 21. Jones DR. Do we know bad science when we see it? J Thorac Cardiovasc Surg. 2015;150(3):472-3. 22. Horton R. Surgical research or comic opera: questions, but few answers. Lancet. 1996;347(9007):984-5.  

The risk of editorial complicity in publishing claims for unproven treatments

https://discovery.ucl.ac.uk/10126511/1/Treasure_Tom%20Treasure%20Specialist%20Journal%20complicity%20210421.pdf

The risk of editorial complicity in publishing claims for unproven treatments

Abstract

Similar works

Full text

Available Versions

UCL Discovery