This reply refers to the comment available at: https://doi.org/10.1007/s40273-017-0576-

C Bell

D Husereau

I Bermejo

Inigo Bermejo

Ji-Hee Youn

M Versteegh

P Tappenden

Paul Tappenden

Z Philips

Crossref

PharmacoEconomics

Response to ‘Comment on “Replicating Health Economic Models: Firm Foundations or a House of Cards?”’

Bermejo, Inigo

Tappenden, Paul

Youn, Ji-Hee

The University of Manchester - Institutional Repository

Response to 'Comment on "Replicating Health Economic Models:Firm Foundations or a House of Cards?"'

Bermejo, I.

Tappenden, P.

Youn, J.H.

White Rose Research Online

This is a repository copy of Response to ‘Comment on “Replicating Health Economic Models: Firm Foundations or a House of Cards?”’.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/124580/Version: Accepted VersionArticle:Bermejo, I., Tappenden, P. orcid.org/0000-0001-6612-2332 and Youn, J.H. (2017) Response to ‘Comment on “Replicating Health Economic Models: Firm Foundations or a House of Cards?”’. PharmacoEconomics, 35 (11). pp. 1189-1190. ISSN 1170-7690 https://doi.org/10.1007/s40273-017-0574-5eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version. This is indicated by the licence information on the White Rose Research Online record for the item. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. Title: Response to ‘Comment on “Replicating Health Economic Models: Firm Foundations or a House of Cards?”’  Authors Bermejo I, Research Associate, School of Health and Related Research, University of Sheffield Tappenden P, Reader in Health Economic Modelling, School of Health and Related Research, University of Sheffield Youn J, Research Fellow, Division of Population Health, Health Services Research & Primary Care, University of Manchester  Dear Editor, We welcome the debate around our article “Replicating Health Economic Models: Firm Foundations or a House of Cards?” [1] put forward by McManus and Sach [2]. In their commentary, McManus et al. criticise the original article on the grounds that it did not include a comprehensive review of the advantages and disadvantages of model replication. As such, it is argued that there are other benefits to model replication which are not discussed in the paper, making particular reference to increasing research transparency. Our original article did not purport to include a comprehensive review; rather, it is an opinion piece written largely from the experience and perspective of the authors. We agree that there may be other benefits to model replication and that further exploration of these may be valuable. However, we consider that model replication is neither an efficient nor effective means of promoting research transparency in modelling studies – in our view, this goal is more likely to be achieved by subjecting original model analyses to more rigorous reporting standards, and ideally, by establishing processes in which original models are made openly accessible to other potential model users, thereby bypassing the need for replication entirely. We believe that the responsibility for increasing research transparency lies with the original model developer, not the model replicator.  McManus et al. also dispute one of the caveats of model replication described in our article: namely, wrongfully assuming that the existing model structure is suitable to address the current decision problem at hand. It is suggested that this may be more of a failure on the part of the modeller rather than a failure of model replication process. Regardless of whether this issue rests with the model replicator, or the process by which the candidate model is selected, this remains an important caveat that should be considered before pursuing the replication of any model. McManus et al. also point out that the selection criteria for our pilot study were not sufficiently clear: we included the latest five economic analyses published in Pharmacoeconomics at the time we started working on our pilot study. Whilst it is not possible to have a fully random sample of modelling studies, excluding Tappenden et al. [3] and selecting a different study in its place would have introduced an additional non-random element into our sample. The key finding of the pilot study was that most of the models attempted could not be replicated by the authors, and for those which could be replicated, discrepancies in the results were evident. The specific models which could be replicated, and those that could not, are irrelevant to this conclusion. It is also suggested that we might have harmed the validity of our pilot study by influencing the replication of the Tappenden et al. UC model. As stated in the article, this model was replicated in isolation by a different author. We agree however that other modellers might produce more or less discrepant results when attempting to replicate the models reported by Tappenden et al. [3] and Versteegh et al [4]. McManus et al. comment that the MS pilot study by Versteegh et al.[4] was replicated from a previous model (Bell et al. [5]) and that the difficulties in replication encountered in their replication exercise were not explored. Our objective was to replicate the analysis reported by Versteegh et al., not the earlier model by Bell et al., hence this criticism is not relevant to our pilot study.  We agree that the notion of ‘successful replication’ was not explicitly defined. In our pilot study, we considered that the replication was successful if: (a) all the necessary information to replicate the model was available, and (b) if the results were not significantly dissimilar from the original reported model results. McManus et al. also criticise the paper for failing to provide the results of the replicated models versus the original study results. As indicated in the manuscript, the replicated case study models are available upon request and readers are welcome to make their own judgements about the significance of the discrepancies found in our results. For clarity, we report here the percentage differences between the results of our replicated models and the original reported analyses. For the UC model, the difference between the original and replicated QALYs for the five interventions was less than 0.36%, whilst the difference between original and replicated costs was 1-4%. Excluding colectomy, which dominated all other options, the deterministic ICERs were 9.6% and 1% lower than the reported ICERs for adalimumab and infliximab, respectively. For the MS model, all seven replicated ICERs indicated a deviation in ICERs of around 17% compared with the original reported results. We agree that the Philips et al. checklist [7] represents a more comprehensive basis for critiquing health economic models and that its use may further improve transparency in model reporting, particularly in terms of the adopted methodology, but note that adopting multiple checklists would increase the burden for publishers. Despite some points of disagreement, overall, we agree with McManus et al. that there are few publications specifically on the role and value of model replication and that there is scope to further explore this topic.   Compliance with Ethical Standards IB, PT and JY did not receive any funding to support the preparation of this commentary response  and have no conflicts of interest to declare.    References: [1] Bermejo, I., Tappenden, P., Youn, J., 2017. Replicating Health Economic Models: Firm Foundations or a House of Cards? Pharmacoeconomics, doi: 10.1007/s40273-017-0553-x. [Epub ahead of print] [2] McManus, E., Sach, T., 2016. Comment on “Replicating Health Economic Models: Firm Foundations or a House of Cards?” Pharmacoeconomics. [3] Tappenden P, Ren S, Archer R, Harvey R, Martyn St-James M, Basarir H, et al. A model-based economic evaluation of biologic and non-biologic options for the treatment of adults with moderately-to-severely active ulcerative colitis after the failure of conventional therapy. Pharmacoeconomics 2016;34(10):1023–1038. [4] Versteegh M. Impact on the incremental cost-effectiveness ratio of using alternatives to EQ-5D in a Markov model for multiple sclerosis. Pharmacoeconomics 2016;34(11):1133–1144.  [5] Bell C, Graham J, Earnshaw S, Oleen-Burkey M, Castelli-Haley J, Johnson K. Cost-effectiveness of four immunomodulatory therapies for relapsing-remitting multiple sclerosis: a Markov model based on long-term clinical data. J Manag Care Pharm 2007;13(3):245–261. [6] Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D. Consolidated health economic evaluation reporting standards (CHEERS)—explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health 2013;16(2):231–250 [7] Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technology Assessment, 2004;8(36):1-158. 

https://eprints.whiterose.ac.uk/124580/4/ReplicationResponse_WR.pdf

Response to ‘Comment on “Replicating Health Economic Models: Firm Foundations or a House of Cards?”’

Abstract

Similar works

Full text

Available Versions

Crossref

The University of Manchester - Institutional Repository

White Rose Research Online