Bock, T

Cheng, G

Edwards, SJL

Palm, U

Pitts, P

Wang, L

Wang, S

English

UCL Discovery

 The Case for Methodological Pluralism in Medical Science Sarah JL Edwardsa, Thomas Bockb, Ulo Palmb, Sally Wangb, Glen Chengc, Lixia Wangd, and Peter Pittse a University College London b APANDEMIC c Harvard University d CTI BioPharma Corp e Center for Medicine in the Public Interest  CONTACT Sarah JL Edwards sarah.edwards@ucl.ac.uk Science and Technology Studies, University College London, London WC1H 0AW, UK.  The received view for integrating clinical science during epidemics of emerging and reemerging infectious diseases is that socially important knowledge is gained only through evaluating the most promising drug candidates in large so-called “definitive” clinical trials (National Academies of Sciences, Engineering, and Medicine 2017). Webb et al. (2020) state that priority should be given to recruiting eligible patients to such trials. Our position is that well-designed observational studies promote both patient autonomy and generation of real-world evidence more quickly than traditional randomized clinical trials (RCTs) and that regulators should permit the broader use of real-world observational studies as a complementary tool in establishing a medicine?s safety and efficacy profile. Freedom of individuals to buy medicines is limited by regulatory restrictions which often require those developing new medicines to expose the minimum number of humans to minimal harm unless and until there is good evidence of sufficient safety and efficacy from research. With such evidence, new medicines can be marketed more or less widely. It is thus a regulatory rather than straightforward scientific question whether clinical evidence is sufficient to allow more patients to access a new medicine, whether it should be marketed, and whether longer term surveillance measures might be necessary given existing data. It should not then be too surprising that there are circumstances in which regulatory standards of sufficiency differ and come apart from a wholly scientific endeavor. The article by Webb et al. (2020) assumes that clinical trials provide the maximally efficient methods for gaining important scientific knowledge and that a strong precautionary approach to regulation should support such endeavours by restricting access only to clinical trials. Financial investment and treatment of patients in anything other than well-designed clinical trials is considered a waste. However, this view does not acknowledge that such an approach leaves several important gaps in the overall accumulated knowledge on which individual patients can later be treated. Firstly, an understanding of statistics shows that there is never a so-called definitive answer and we can only be confident—to different degrees—that the observed data (from whichever design of clinical trials) is enough to reject the null hypothesis. “Properly” designed clinical trials can include anything from dose escalation with few volunteers to using concurrent comparators to reduce confounding variables, and to randomizing patients to reduce selection biases. These methodological features may be more or less scientifically important depending on the type of investigational treatment, its stage of development, similarity with existing medicines, proposed mechanism of action, understanding of the disease pathology, observational evidence from other sources, and the amount of the effect size to be considered clinically worthwhile. Secondly, there is a supposed crisis of reproducibility in science, with clinical trials included, emphasizing the importance of “triangulation” where results can be corroborated using different methods. Thirdly, once large clinical trials have been completed, it is not always clear how those statistical results apply to individual patients under routine, real-world conditions as opposed to a classically-controlled clinical experiment. A negative trial result could mask important beneficial effects on sub-groups of patients which might only be identified through closer observational work. Hypotheses regarding mechanisms of action can be generated by therapeutic observations and there is often little room for such activities which might now be considered old school except for when existing medicines might be repurposed. Conversely, a positive trial result by itself is typically constrained to a highly selected patient population that often is not reflective of the population a prospective treatment needs to serve in actual medical practice (the “real world”). Thus, it does not offer a complete causal picture (Deaton and Cartwright 2018). Indeed, there may be relatively strong evidence of safety and efficacy in a narrow, homogenous population of patients on which basis to file a market application with the regulator yet, once on sale, the medicine can be prescribed off-label used by others beyond the approved narrowly indicated group. The rise of implementation science can be seen as an attempt to answer some of these knowledge gaps often requiring later studies, sometimes further clinical trials of a more pragmatic nature, to establish generalisability of results to routine settings. While randomized clinical trials and traditional Fisher statistics are one robust and established way to evaluate results about a medicine’s safety and efficacy, it also stipulates certain trade-offs and does not enable black-and-white conclusions at a single timepoint (typically at the unblinding of a trial). We suggest that there is a need to assess the programme of scientific work related to investigational products which, when combined with modern analytical methods, enable equally robust conclusions, but with their own (different) trade-offs. In particular, real-world observational evidence collected in parallel to randomized clinical trials can offer complementary rather than competing datasets when reviewed in the round. By contrast to the position described by Webb et al. (2020) which seeks to prioritise recruitment to clinical trials, the US FDA has recently issued guidance on analyses of real world data for Covid-19 research (FDA 2020) thereby acknowledging the critical role of real-world observation as an important scientific pathway to complement clinical trials of the sort given priority in the Article by Webb et al. (2020). To provide the technological platforms for gathering and analysing such real world data, a coalition of healthcare leaders including physicians, medical researchers, therapeutics developers, data scientists, engineers and corporate executives who joined forces to help end the COVID-19 crisis. Called APANDEMIC (Accelerate Patient Access & New Therapy Development in the COVID-19 Crisis) (2020), the coalition provides expertise and experience of research especially in life threatening rare diseases where traditional precautionary approaches to regulating new medicines are often self-defeating. The traditional approach to drug development via “gold standard” RCTs, often result in delaying patient access to new and potentially life-saving medicines, putting patients at risk and delaying (in the case of COVID-19) a nation’s ability to reopen their economy. It is often thought that to have a preference for an investigational medicine and to provide it on compassionate grounds through observational studies assumes it is safe. Not so. Depending on theory of the specific mechanism of action and any preclinical data, a new medicine may simply be worth the gamble for an individual. The patient may be said to have a rational preference for trying something knowing that there is potential benefit but also potential costs relative to doing nothing at all. To say that there is a state of epistemic uncertainty between these two options misses the point. In some cases, without treatment the patient will likely die (for example, treatment with Remdesivir, originally developed–unsuccessfully–to treat Ebola Grein et al. 2020). While there is radical uncertainty concerning the safety and benefits of an intervention, it is this very uncertainty which can offer hope if not any expectation of benefit. Hope can be entirely rational because without it there may be little rationale for testing a new medicine at all (through any method, scientifically robust or otherwise). As Alex John London rightly points out, private interests of patients are not the only salient consideration (London 2019). Yet accepting the scientific value of observational studies of investigational medicines does not imply sacrificing social value from research. The regulator must balance the expected burdens and benefits to decide when it is important to have strong evidence of safety and efficacy before permitting wider use or not. A market approval does not mark at definitive point when science is done and routine practice can take over. Promoting real-world observational studies will emphasize longer term surveillance and research in a board sense to track the ‘life’ of a new product. Here, we are not committed to a particular position on the extent to which research during epidemics should primarily serve the individual patients affected, the population affected by the current outbreak, or the next one. Settling those issues would, however, help us clarify issues over whose interests should be of primary concern, how efficient science should be, and how science constrains the interests and freedoms of the patients who participate in it. Regulators might try to rebalance the interests of clinical trials and of the individual patient by accepting different study designs such a cross over or adaptive designs as a methodological compromise in the sense that a result—to be equivalent in terms of statistical power and internal validity—might require more patients or longer follow-up to compensate. Our point here is simply to show that clinical science does not advance only through clinical trials. As the battle against COVID-19 demonstrates, a global health challenge requires a multitude of methodologies to rapidly identify, develop and evaluate investigational and existing medicinal products. Legitimate claims of causation rest on plausible hypotheses, evidence of mechanisms of action and clinical evidence from a variety of sources. Promoting the use of real-world observational studies whose research designs could be run in comprehensive and parallel programs alongside RCTs will establish safety and effectiveness under real-world settings, further patient autonomy and lead to the swifter development of therapies to prevent loss of life and human suffering.  References APANDEMIC (To Accelerate Patient Access & New Therapy Development in the COVID-19 Crisis). 2020. www.APANDEMIC.org (accessed July 6, 2020). Deaton, A., and N. Cartwright. 2018. Understanding and misunderstanding randomized controlled trials. Social Science & Medicine (1982) 210: 2–21. doi  FDA. 2020. Coronavirus (COVID-19) update: FDA collaborations promote rigorous analyses of real-world data to inform pandemic response. https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-collaborations-promote-rigorous-analyses-real-world-data-inform (accessed July 6, 2020). Grein, J., N. Ohmagari, D. Shin, et al. 2020. Compassionate use of remdesivir for patients with severe COVID-19. The New England Journal of Medicine 382(24): 2327–2336. doi:  London, A. J. 2019. Social value, clinical equipoise, and research in a public health emergency. Bioethics 33(3): 326–334. doi:   National Academies of Sciences, Engineering, and Medicine. 2017. Integrating Clinical Research into Epidemic Response: The Ebola Experience. Washington, DC: The National Academies Press.   Webb, J., L. D. Shah, and H. Fernandez Lynch. 2020. Ethically allocating COVID-19 drugs via pre-approval access and emergency use authorization. The American Journal of Bioethics 20(9): 4–17.  

The Case for Methodological Pluralism in Medical Science

https://discovery.ucl.ac.uk/10124169/1/Edwards_AJOB%20Methodological%20Pluralism%20in%20Medical%20Science.pdf

The Case for Methodological Pluralism in Medical Science

Abstract

Similar works

Full text

Available Versions

UCL Discovery