Here we discuss the report by Claassen and colleagues describing an indirect treatment comparison between tetrabenazine and deutetrabenazine for chorea in Huntington’s disease using individual patient data. We note the potential for discrepancies in apparently statistically significant findings, due to the rank reversal phenomenon. We provide some cautionary observations and suggestions concerning the limitations of indirect comparisons and the low likelihood that good quality evidence will become available to guide clinical decision comparing these two agents

Costa, J

Duarte, GS

Ferreira, JJ

Rodrigues, FB

Wild, EJ

UCL Discovery

LETTER TO THE EDITOR Open AccessMeta-research metrics matter: letterregarding article “indirect tolerabilitycomparison of Deutetrabenazine andTetrabenazine for Huntington disease”Filipe B. Rodrigues1,2,3*, Gonçalo S. Duarte2,3, João Costa2,3,4,5, Joaquim J. Ferreira2,3 and Edward J. Wild1AbstractHere we discuss the report by Claassen and colleagues describing an indirect treatment comparison betweentetrabenazine and deutetrabenazine for chorea in Huntington’s disease using individual patient data. We note thepotential for discrepancies in apparently statistically significant findings, due to the rank reversal phenomenon. Weprovide some cautionary observations and suggestions concerning the limitations of indirect comparisons and the lowlikelihood that good quality evidence will become available to guide clinical decision comparing these two agents.Keywords: Huntington’s disease, Tetrabenazine, Deutratrabenazine, Indirect treatment comparisonTo the Editor,We read with interest the report by Claassen andcolleagues describing an indirect treatment comparisonbetween tetrabenazine (TBZ) and deutetrabenazine (DEU)for chorea in Huntington’s disease (HD) using individualpatient data [1].DEU is a form of TBZ, chemically-modified to optimizeits pharmacokinetic properties. Both are vesicularmonoamine transporter 2 (VMAT2) inhibitors and eachwas tested successfully against placebo for chorea associ-ated with HD [2, 3]. No double-blinded head-to-headcomparison has been performed or is planned to comparethe efficacy and safety profiles of these compounds.Indirect treatment comparisons are useful meta-researchtools when little or no data directly comparing treatmentare available [4]. In meta-research, the use of individualpatient data instead of aggregate data has many potentialadvantages, such as the power to study subgroups and tocontrol for confounding factors. To some extent Claassenet al. used individual patient data, since raw patient datafrom the FIRST-HD trial testing DEU was incorporatedinto the analysis. This partially overcomes possible report-ing bias from the literature, such as adverse events frequen-cies not recorded in primary reports, and provides theopportunity to use more complex and complete statisticalmodels adjusted to important covariates. It is a shame thatindividual patient data from the TETRA-HD trial [3] werenot included, especially since both trials were performed bythe same study consortium (the Huntington Study Group).In addition, it would have been both possible and interest-ing to undertake exploratory analyses to find out whethersubgroups of patients with different genders, CAG repeatlengths, baseline levels of functional ability, motor symp-toms and quality of life differed in regards to safety profile.We also think it unfortunate that the authors limitedtheir report to safety data, while it would be extremelyrelevant to learn how the efficacy profiles of TBZ andDEU compared using the individual patient data availableto them. We recently performed an indirect treatmentcomparison using all published aggregate data from thesame trials, and found no difference between the primaryefficacy outcomes of both trials: the total maximal choreascore change from baseline mean difference was betweenTBZ and DEU was −1.00 (95% confidence interval: −3.04to 1.04) [5].* Correspondence: f.rodrigues@ucl.ac.uk1Huntington’s Disease Centre, Institute of Neurology, University CollegeLondon, Russell Square 10-12, London WC1B 5EH, UK2Laboratory of Clinical Pharmacology and Therapeutics, Faculty of Medicine,University of Lisbon, Lisbon, PortugalFull list of author information is available at the end of the article© The Author(s). 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Rodrigues et al. Journal of Clinical Movement Disorders  (2017) 4:19 DOI 10.1186/s40734-017-0067-xInterestingly, in contrast to Claasen and co-authors’findings, our safety outcomes did not demonstrate anydifference between odds ratios of TBZ over DEU(Table 1). However, when converting our raw dataset torisk differences instead of odds ratios, as presented byClaassen et al., our results were in line with theirs, inregards to direction and magnitude of effect. This is awell-described statistical phenomenon called rank rever-sal. It stems from the fact that different measures (e.g.risk differences, risk ratios, and odds ratios) are affecteddifferently by dissimilar baseline risks [6]. Bucher’smodel of indirect treatment comparisons was originallydesigned for odds ratios, but others have applied it torisk ratios and risk differences, with proper adjustment.When indirectly comparing treatments, the choice ofpresented metric matters. We believe it is important forreaders to be made aware of the potential for apparentlydiscrepant findings that may arise from the rank reversalphenomenon [6]. Undoubtedly, risk differences increase in-terpretability, but odds ratios are the only measure thatguarantees the avoidance of impossible predicted eventrates when extrapolating the results for real populations(for example, applying a risk difference of 0.1 to a popula-tion with a risk of 0.05 would give rise to an apparent popu-lation risk with a value less than zero, which is implausible)and relative measures are known to be more consistentthan absolute measures [7–9].Choice of data and outcomes aside, an overarchingconcern is that none of the included clinical trials has beenappropriately powered to investigate the safety of thesecompounds, and an indirect treatment comparison of onetrial per agent (i.e. TBZ versus placebo, and DEU versusplacebo), cannot improve the precision of the results.Therefore, the results of these comparisons [1, 5] should beinterpreted with caution; and although regarded as bestavailable evidence, they are nonetheless of low qualityaccording to the Grading of Recommendations Assessment,Development and Evaluation (GRADE) framework [10].Given all this, only a direct comparison between TBZand DEU would be able to rigorously test whether theefficacy profiles of these compounds significantly differ, andour sample size calculation (over 600 participants) suggeststhat such a trial is unlikely to take place [5]. Further post-authorisation safety studies or other observational studieswill be required to provide robust evidence on the safetyprofile of DEU to inform safe prescribing decisions.AbbreviationsDEU: deutetrabenazine; GRADE: Grading of Recommendations Assessment,Development and Evaluation; HD: Huntington’s disease; TBZ: tetrabenazine;VMAT2: vesicular monoamine transporter 2AcknowledgmentsNone.FundingNone.Availability of data and materialsNot applicable.Authors’ contributionsFBR conceptualized, designed, interpreted, wrote the first draft and revisedthe manuscript. GSD, JC and JJF interpreted and revised the manuscript forintellectual content. EJW conceptualized, designed, interpreted, and revisedthe manuscript. All authors read and approved the final manuscript.Ethics approval and consent to participateNot applicable.Consent for publicationNot applicable.Competing interestsAll the authors of this manuscript performed and published an aggregatedata indirect treatment comparison between tetrabenazine anddeutetrabenazide for chorea in Huntington’s disease. The authors declarethat they have no other competing interests.Table 1 Indirect treatment comparison between TBZ and DEU (as reported by us in Rodrigues et al. [5]), both in odds ratios asinitially reported, and converted to risk differences; and as reported by Claassen et al. [1] as risk differencesAdverseeventIndirect comparisonRodrigues et al. [5] Claassen et al. [1]OR (95% CI)(TBZ vs DEU)RD (95% CI)(DEU vs TBZ)RD (95% CI)(DEU vs TBZ)SAE 5.44 (0.09 to 322.08) −0.07 (−0.17 to 0.03) −0.074 (−0.167 to 0.019)Somnolence 4.95 (0.34 to 72.37) −0.21 (−0.39 to −0.03)* −0.215 (−0.392 to −0.037)*Diarrhoea 0.07 (0.03 to 2.06) 0.12 (−0.03 to 0.27) 0.115 (−0.038 to 0.268)Insomnia 14.18 (0.47 to 426.77) −0.24 (−0.40 to −0.04)* −0.237 (−0.387 to −0.087)*Fatigue 1.21 (0.31 to 11.14) −0.07 (−0.26 to 0.12) −0.067 (−0.256 to 0.123)Falls 2.71 (0.31 to 23.98) −0.07 (−0.26 to 0.12) −0.078 (−0.265 to 0.110)Depression 17.15 (0.55 to 531.90) −0.17 (−0.31 to −0.28)* −0.170 (−0.304 to −0.037)*Presentation as risk differences produces statistically significant differences between DEU and TBZ that are not seen when presented as odds ratios; neitherapproach is intrinsically more accurate and an awareness of the difference is important. TBZ, tetrabenazine; DEU, deutetrabenazine; OR, odds ratio; 95% CI, 95%confidence interval; RD, risk difference; SAE, severe adverse events; *p-value < 0.05Rodrigues et al. Journal of Clinical Movement Disorders  (2017) 4:19 Page 2 of 3Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.Author details1Huntington’s Disease Centre, Institute of Neurology, University CollegeLondon, Russell Square 10-12, London WC1B 5EH, UK. 2Laboratory of ClinicalPharmacology and Therapeutics, Faculty of Medicine, University of Lisbon,Lisbon, Portugal. 3Clinical Pharmacology Unit, Instituto de MedicinaMolecular, Lisbon, Portugal. 4Cochrane Movement Disorders Group, Facultyof Medicine, University of Lisbon, Lisbon, Portugal. 5Center forEvidence-Based Medicine, Faculty of Medicine, University of Lisbon, Lisbon,Portugal.Received: 18 April 2017 Accepted: 17 October 2017References1. Claassen DO, Carroll B, De Boer LM, Wu E, Ayyagari R, Gandhi S, Stamler D.Indirect tolerability comparison of Deutetrabenazine and Tetrabenazine forHuntington disease. J Clin Mov Disord. 2017;4:3.2. Huntington Study Group. Tetrabenazine as antichorea therapy in Huntingtondisease: a randomized controlled trial. Neurology. 2006;66(3):366–72.3. Huntington Study Group. Effect of Deutetrabenazine on chorea among patientswith Huntington disease: a randomized clinical trial. JAMA. 2016;316(1):40–50.4. Bucher HC, Guyatt GH, Griffith LE, Walter SD. The results of direct andindirect treatment comparisons in meta-analysis of randomized controlledtrials. J Clin Epidemiol. 1997;50(6):683–91.5. Rodrigues FB, Duarte GS, Costa J, Ferreira JJ, Wild EJ: Tetrabenazine versusdeutetrabenazine for Huntington's disease: twins or distant cousins? MovDisord Clin Pract 2017:n/a-n/a.6. Norton EC, Miller MM, Wang JJ, Coyne K, Kleinman LC. Rank reversal inindirect comparisons. Value Health. 2012;15(8):1137–40.7. Walter SD. Choice of effect measure for epidemiological data. J ClinEpidemiol. 2000;53(9):931–9.8. Deeks JJ. Issues in the selection of a summary statistic for meta-analysis ofclinical trials with binary outcomes. Stat Med. 2002;21(11):1575–600.9. Engels EA, Schmid CH, Terrin N, Olkin I, Lau J. Heterogeneity and statisticalsignificance in meta-analysis: an empirical study of 125 meta-analyses. StatMed. 2000;19(13):1707–28.10. Guyatt GH, Oxman AD, Schunemann HJ, Tugwell P, Knottnerus A. GRADEguidelines: a new series of articles in the journal of clinical epidemiology. JClin Epidemiol. 2011;64(4):380–2.•  We accept pre-submission inquiries •  Our selector tool helps you to find the most relevant journal•  We provide round the clock customer support •  Convenient online submission•  Thorough peer review•  Inclusion in PubMed and all major indexing services •  Maximum visibility for your researchSubmit your manuscript atwww.biomedcentral.com/submitSubmit your next manuscript to BioMed Central and we will help you at every step:Rodrigues et al. Journal of Clinical Movement Disorders  (2017) 4:19 Page 3 of 3

Meta-research metrics matter: letter regarding article “indirect tolerability comparison of Deutetrabenazine and Tetrabenazine for Huntington disease”

http://discovery.ucl.ac.uk/10038320/1/2017%2011%20Meta-research%20metrics%20matter%20letter%20regarding%20article%20%E2%80%9CIndirect%20tolerability%20comparison%20of%20Deutetrabenazine%20and%20Tetrabenazine%20for%20Huntington%20disease.pdf

Meta-research metrics matter: letter regarding article “indirect tolerability comparison of Deutetrabenazine and Tetrabenazine for Huntington disease”

Abstract

Similar works

Full text

Available Versions

UCL Discovery