Camelo Castillo, Wendy

Delaney, Joseph A. C.

Stürmer, Til

Carolina Digital Repository

The challenges of comparing results between placebo controlledrandomized trials and non-experimental new user, activecomparator cohort studies: the example of OlmesartanWendy Camelo Castillo1,2, Joseph A C Delaney3,§, and Til Stürmer41Center for Health Research, Geisinger Health System, Danville, PA, USA2Department of Environmental Health Sciences, Johns Hopkins Bloomberg School of PublicHealth, Baltimore, MD, USA3Department of Epidemiology, University of Washington, Seattle, WA, USA4Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USAOne of the challenges of pharmacoepidemiology that is both exciting and frustrating is thereplication of findings. It is often easy to frame the question as “does drug X cause sideeffect Y”, but this ignores subtle differences in the actual contrasts that are being made. Thisissue is well highlighted by three recent studies of the cardiovascular safety of themedication Olmesartan published in this issue of the Journal 1–3. These non-experimental(observational) studies were motivated by safety signals from two randomized, placebocontrolled trials that suggested that Olmesartan increased all-cause mortality in patients withdiabetes4–5. Although the absolute number of deaths observed in the randomized trials wassmall, the possibility of such an effect is a legitimate cause for concern 1–3. The three non-experimental studies are all well designed and placed in independent populations in twodifferent countries (the United States of America and the United Kingdom).However, right from the beginning, there are intrinsic challenges with replicating arandomized, placebo controlled trial using a non-experimental study design. Drugs are givenfor specific indications and one of the primary indications of angiotensin receptor blockers(ARBs), including Olmestartan, is the treatment of hypertension. Unfortunately,hypertension itself is a risk factor for major cardiovascular events. So any analysis of the§Corresponding author: Please address correspondence to: Joseph Delaney, Collaborative Health Studies Coordinating Center,Department of Biostatistics, University of Washington, Bldg.29, Suite 310, 6200 NE 74th Street, Seattle, WA 98115, Phone: (206)897-1918, jacd@uw.edu.CONFLICT OF INTERESTDr. Camelo Castillo has no conflicts of interest to declare.Dr. Delaney receives investigator-initiated research funding and support as a principal investigator (R21HL120394-01) from NHLBI.Dr. Stürmer receives investigator-initiated research funding and support as Principal Investigator (R01 AG023178) and Co-Investigator (R01 AG042845) from the National Institute on Aging (NIA), and as Co-Investigator (R01 CA174453) from the NationalCancer Institute (NCI) at the National Institutes of Health (NIH), and as Principal Investigator of a Pilot Project from the PatientCentered Outcomes Research Institute (PCORI). He also received research funding as Principal Investigator of the UNC-DEcIDEcenter from the Agency for Healthcare Research and Quality. Dr. Stürmer does not accept personal compensation of any kind fromany pharmaceutical company, though he receives salary support from the Center for Pharmacoepidemiology (current members:GlaxoSmithKline, UCB BioSciences, Merck) and research support from pharmaceutical companies (Amgen, Genentech, Merck,Sanofi) to the Department of Epidemiology, University of North Carolina at Chapel Hill.NIH Public AccessAuthor ManuscriptPharmacoepidemiol Drug Saf. Author manuscript; available in PMC 2015 April 01.Published in final edited form as:Pharmacoepidemiol Drug Saf. 2014 April ; 23(4): 357–360. doi:10.1002/pds.3602.NIH-PA Author ManuscriptNIH-PA Author ManuscriptNIH-PA Author Manuscriptcardiovascular safety of Olmesartan has to deal with confounding by indication6, which canbe tricky to overcome.This concern about confounding by indication makes it inadvisable to do a non-experimentalabsolute safety comparison in this particular context. Instead, the non-experimental studiesask a closely related question: is initiating patients on Olmesartan risky relative to initiatingpatients on other ARBs and angiotensin converting enzyme inhibitors (ACEI)? This newquestion restricts the potential for confounding by indication insofar as the indication forthese drugs and drug classes is similar. It is also sensible, since medications in this class areknown to have positive effects on all-cause and cardiovascular mortality7 when used to treathypertension (a major reason for prescribing these medications). So it would be unlikely thatparticipants who are prescribed this drug in the general population would ever be leftuntreated, but concerns about cardiovascular risk would likely lead to the substitution withanother anti-hypertensive medication. Designing these non-experimental studies ascomparative safety studies refocuses the inference from the specific comparison in the trials(Olmesartan versus placebo) to what is clearly the most concerning possibility - that there isa previously undetected excess risk of mortality and/or cardiovascular events withOlmesartan compared with readily available clinical alternatives.When comparing results between these studies, we need to also understand the differencesbetween the study populations and choice of subgroup analyses. In the non-experimentalstudies the study populations are more representative of the general population ofOlmesartan initiators with subgroup analyses conducted on diabetics. In all patientsinitiating ARBs/ACEIs, none of the studies found an association between initiation ofOlmesartan versus other ARBs/ACEIs and all-cause or cardiovascular disease mortality. Inboth trials, the study populations were diabetics either with risk factors for cardiovasculardisease (ROADMAP) or overt nephropathy (ORIENT). For all-cause mortality, whileROADMAP reported an elevated risk (HR 1.70), in ORIENT the estimate was close to thenull (HR 0.99). In diabetics, Zhou et al. also found a slightly elevated risk which was evenhigher among those treated with high versus low dose of Olmesartan (HR 1.35). Walker etal. found no relative increase in risk of all-cause mortality among diabetics. Regardingcardiovascular death, both trials reported an elevated risk in users of Olmesartan (HR 4.94ROADMAP; 2.81 ORIENT). Only the study by Walker et al reported on cardiovascularrelated death among diabetics (HR 2.10), consistent with the trials. The exception was indiabetics with prior use of ACEIs in whom no association was found (HR 0.80). Graham etal. used an outcome definition enriched for cardiovascular deaths and conducted analysisamong diabetics treated with high dose, comparing short versus long term use (<6 vs ≥6months). They found an elevated risk among those who had longer use (HR 2.03).Apart from the active comparator design decision, there are more general issues with thecomparison of non-experimental and randomized studies, given the concern aboutgeneralizability, namely sub-group effects. In the presence of treatment effect heterogeneityit becomes problematic to generalize from the trials. In diabetics, prior cardiovasculardisease is a potential modifier of the effect of Olmesartan on mortality. Only ROADMAPconducted a subgroup analysis stratifying by coronary heart disease history. In this trialparticipants with prior coronary heart disease had an elevated risk for all-cause mortalityCastillo et al. Page 2Pharmacoepidemiol Drug Saf. Author manuscript; available in PMC 2015 April 01.NIH-PA Author ManuscriptNIH-PA Author ManuscriptNIH-PA Author Manuscript(HR 4.16) and cardiovascular death (HR 10.6) compared with participants without coronaryheart disease (HR 1.08, and HR 2.00, respectively). It is known that there are also strongassociations between age, sex and both all-cause and cardiovascular mortality. Howeverdifferences by sex or age were not explored in the trials or the observational studies.Another important consideration is the scale in which results are reported. From a clinicalstandpoint, reporting both the relative and absolute difference in risk of death acrosssubgroups could be useful to assess the change in individual risk associated with Olmesartantreatment.Another key difference between the randomized controlled trials and the non-experimentalstudies is the shorter duration of treatment in the non-experimental cohorts. Both of therandomized trials had a mean duration of treatment greater than 3 years 4–5, whereas thecohorts had mean duration of treatment of 1.2 years or less1–3. As an illustration, in theGraham et al. study, there were only 10% of participants who were still taking the samedrug at one year. This paucity of long term use mostly reflects the reality of how the drugsare taken in real clinical practice. It could have been alleviated to some extent by allowingpatients to switch to another drug of the same class (at the cost of some exposuremisclassification). While the lack of data on long term use may be seen as a limitation ofany non-experimental new user study, it also speaks to the remoteness of RCTs with respectto actual treatment patterns. It also clearly limits the scale of the public health impact of anypotential increase in risk with long term treatment, although knowing about such risksremains important.Given the legitimate worry about confounding in non-experimental studies, all of the studiesused some form of propensity score based approach as a part of their inference whichenabled them to broadly adjust for the information on potential confounders available inlarge automated healthcare databases. The use of a large number of covariates in thepropensity score is becoming increasingly common in pharmacoepidemiology as evidencedby the use of high dimensional propensity score8. One advantage of propensity scoreapproaches is that patients always treated with one of the drugs compared based on theircovariates can be excluded. After exclusion of such patients, in whom the treatment effectcannot be estimated, estimates apply to participants who could have received eithertreatment, i.e., resembling the target population of patients in whom a treatment decisionneeds to be made. In addition, trimming the tails of the overlapping propensity scoredistribution can reduce the potential for bias caused by poorly captured confounders10 thatmay be present in those patients treated contrary to prediction. The small loss ofinterpretability (estimate does not apply to all treated) is almost certainly worth it for theincrease in internal validity, even if it does complicate direct comparisons. Additionalconsiderations could also be made on the use of the overall versus a subgroup specificpropensity score especially in when suspecting the presence of a strong confounder such asprior history of coronary heart disease14.So what is the current evidence on Olmesartan and mortality? Combined, the three non-experimental studies provide current best evidence for the comparative safety of Olmesartanas currently used in the US and UK. None of these studies sheds a lot of light on the specificquestion asked by the trials, as the data that would be required to precisely replicate theCastillo et al. Page 3Pharmacoepidemiol Drug Saf. Author manuscript; available in PMC 2015 April 01.NIH-PA Author ManuscriptNIH-PA Author ManuscriptNIH-PA Author Manuscriptprecise causal question asked in the trials was simply not available. Furthermore, evenattempting a precise replication would have been highly inadvisable, due to concerns aboutconfounding by indication6 and underlying differences in participant propensity to adhere tomedication assignment (as seen by the low persistence on therapy in the non-experimentalstudies). The results of these studies do, however, all appear to agree that there is littleevidence that the excess cardiovascular mortality detected in the ROADMAP and ORIENTtrials4 applies to the majority of users of Olmesartan. Among diabetics, however, theassociation between Olmesartan and cardiovascular mortality is less clear and may require acloser look. Small trials and subgroup analyses are vulnerable to chance findings, andregression to the mean suggests that the initial size of large effects may well shrink uponreplication15. In the absence of additional data on Olmesartan in patients with diabetes,physicians and patients should weigh the potential benefits and harms versus availableclinical alternatives before initiating Olmesartan in patients with diabetes.References1. Zhou EH, Gelperin K, Levenson MS, et al. Risk of acute myocardial infarction, stroke, or death inpatients initiating olmesartan or other angiotensin receptor blockers – a cohort study using theClinical Practice Research Datalink. Pharmacoepidemiol Drug Saf. (details to follow; commentaryto be published in the same issue).2. Walker AM, Liang C, Clifford CR, et al. Cardiac Mortality in Users of Olmesartan, OtherAngiotensin Receptor Blockers and Angiotensin Converting Enzyme Inhibitors.Pharmacoepidemiol Drug Saf. (details to follow; commentary to be published in the same issue).3. Graham DJ, Zhou E, McKean S, et al. Cardiovascular and mortality risk in elderly Medicarebeneficiaries treated with olmesartan vs. other angiotensin receptor blockers. PharmacoepidemiolDrug Saf. (details to follow; commentary to be published in the same issue).4. Haller H, Ito S, Izzo JL, et al. Olmesartan for the delay or prevention of microalbuminuria in Type 2Diabetes. N Engl J Med. 2011; 364:907–17. [PubMed: 21388309]5. Imai E, Chan JCN, Ito S, et al. Effects of olmesartan on renal and cardiovascular outcomes in type 2diabetes with overt nephropathy: a multicentre, randomised, placebo-controlled trial. Diabetologia.2011; 54:2978–86. [PubMed: 21993710]6. Psaty BM, Siscovick DS. Minimizing bias due to confounding by indication in comparativeeffectiveness research: the importance of restriction. JAMA. 2010; 304(8):897–8. [PubMed:20736474]7. van Vark LC, Bertrand M, Akkerhuis KM, et al. Angiotensin-converting enzyme inhibitors reducemortality in hypertension: a meta-analysis of randomized clinical trials of renin-angiotensin-aldosterone system inhibitors involving 158,998 patients. Eur Heart J. 2012; 33(16):2088–97.[PubMed: 22511654]8. Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensionalpropensity score adjustment in studies of treatment effects using health care claims data.Epidemiology. 2009; 20(4):512–22. [PubMed: 19487948]9. Stürmer T, Joshi M, Glynn RJ, Avorn J, Rothman KJ, Schneeweiss S. A review of the application ofpropensity score methods yielded increasing use, advantages in specific settings, but notsubstantially different estimates compared with conventional multivariable methods. J ClinEpidemiol. 2006; 59(5):437–47. [PubMed: 16632131]10. Stürmer T, Rothman KJ, Avorn J, Glynn RJ. Treatment effects in the presence of unmeasuredconfounding: dealing with observations in the tails of the propensity score distribution--asimulation study. Am J Epidemiol. 2010; 172(7):843–54. [PubMed: 20716704]11. Patrick AR, Schneeweiss S, Brookhart MA, Glynn RJ, Rothman KJ, Avorn J, Stürmer T. Theimplications of propensity score variable selection strategies in pharmacoepidemiology: anempirical illustration. Pharmacoepidemiol Drug Saf. 2011; 20(6):551–9. [PubMed: 21394812]Castillo et al. Page 4Pharmacoepidemiol Drug Saf. Author manuscript; available in PMC 2015 April 01.NIH-PA Author ManuscriptNIH-PA Author ManuscriptNIH-PA Author Manuscript12. Brookhart MA, Schneeweiss S, Rothman KJ, Glynn RJ, Avorn J, Stürmer T. Variable selection forpropensity score models. Am J Epidemiol. 2006; 163(12):1149–56. [PubMed: 16624967]13. Lefebvre G, Delaney JA, Platt RW. Impact of mis-specification of the treatment model onestimates from a Marginal Structural Model. Stat Med. 2008; 27(18):3629–3642. [PubMed:18254127]14. Girman CJ, Gokhale M, Kou TD, Brodovicz KG, Wyss R, Stürmer T. Assessing the Impact ofPropensity Score Estimation and Implementation on Covariate Balance and Confounding ControlWithin and Across Important Subgroups in Comparative Effectiveness Research. Med Care. 2013Dec 26.15. Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baselinedata in clinical trials. Lancet. 2000 Mar 25; 355(9209):1064–9. [PubMed: 10744093]Castillo et al. Page 5Pharmacoepidemiol Drug Saf. Author manuscript; available in PMC 2015 April 01.NIH-PA Author ManuscriptNIH-PA Author ManuscriptNIH-PA Author ManuscriptFigure 1.Forest Plot summarizing the results from the randomized controlled trials and non-experimental studies discussed in thecommentary. Cardiovascular Disease (CVD) mortality was defined as sudden cardiac death in the study by Walker AM et al.Results from Graham et al. are not reported in some of the subgroup analysis due to lack of comparability.Castillo et al. Page 6Pharmacoepidemiol Drug Saf. Author manuscript; available in PMC 2015 April 01.NIH-PA Author ManuscriptNIH-PA Author ManuscriptNIH-PA Author Manuscript

The challenges of comparing results between placebo controlled randomized trials and non-experimental new user, active comparator cohort studies: the example of olmesartan: COMPARING RESULTS FOR OLMESARTAN

https://cdr.lib.unc.edu/downloads/2514nt344

The challenges of comparing results between placebo controlled randomized trials and non-experimental new user, active comparator cohort studies: the example of olmesartan: COMPARING RESULTS FOR OLMESARTAN

Abstract

Similar works

Full text

Available Versions

Carolina Digital Repository