Challenges and methodologies in using progression free survival as a surrogate for overall survival In oncology

Objectives: A primary outcome in oncology trials is overall survival (OS). However, to estimate OS accurately requires a sufficient number of patients to have died, which may take a long time. If an alternative end point is sufficiently highly correlated with OS, it can be used as a surrogate. Progression-free survival (PFS) is the surrogate most often used in oncology, but does not always satisfy the correlation conditions for surrogacy. We analyze the methodologies used when extrapolating from PFS to OS. Methods: Davis et al. previously reviewed the use of surrogate end points in oncology, using papers published between 2001 and 2011. We extend this, reviewing papers published between 2012 and 2016. We also examine the reporting of statistical methods to assess the strength of surrogacy. Results: The findings from 2012 to 2016 do not differ substantially from those of 2001 to 2011: the same factors are shown to affect the relationship between PFS and OS. The proportion of papers reporting individual patient data (IPD), strongly recommended for full assessment of surrogacy, remains low: 33 percent. A wide range of methods has been used to determine the appropriateness of surrogates. While usually adhering to reporting standards, the standard of scholarship appears sometimes to be questionable and the reporting of results often haphazard. Conclusions: Standards of analysis and reporting PFS to OS surrogate studies should be improved by increasing the rigor of statistical reporting and by agreeing to a minimum set of reporting guidelines. Moreover, the use of IPD to assess surrogacy should increase

Survival extrapolation in cancer immunotherapy: a validation-based case study

Background: Immune-checkpoint inhibitors may provide long-term survival benefits via a cured proportion, yet data are usually insufficient to prove this upon submission to health technology assessment bodies. Objective: We revisited the National Institute for Health and Care Excellence assessment of ipilimumab in melanoma (TA319). We used updated data from the pivotal trial to assess the accuracy of the extrapolation methods used and compared these to previously unused techniques to establish whether an alternative extrapolation may have provided more accurate survival projections. Methods: We compared projections from the piecewise survival model used in TA319 and those produced by alternative models (fit to trial data with minimum follow-up of 3 years) to a longer-term data cut (5-year follow-up). We also compared projections to external data to help assess validity. Alternative approaches considered were parametric, spline-based, mixture, and mixture-cure models. Results: Only the survival model used in TA319 and a mixture-cure model provided 5-year survival predictions close to those observed in the 5-year follow-up data set. Standard parametric, spline, and non–curative-mixture models substantially underestimated 5-year survival. Survival estimates from the TA319 model and the mixture-cure model diverge considerably after 5 years and remain unvalidated. Conclusions: In our case study, only models that incorporated an element of external information (through a cure fraction combined with background mortality rates or using registry data) provided accurate estimates of 5-year survival. Flexible models that were able to capture the complex hazard functions observed during the trial, but which did not incorporate external information, extrapolated poorly

Mixture and non-mixture cure models for health technology assessment: what you need to know

There is increasing interest in the use of cure modelling to inform health technology assessment (HTA) due to the development of new treatments that appear to offer the potential for cure in some patients. However, cure models are often not included in evidence dossiers submitted to HTA agencies, and they are relatively rarely relied upon to inform decision-making. This is likely due to a lack of understanding of how cure models work, what they assume, and how reliable they are. In this tutorial we explain why and when cure models may be useful for HTA, describe the key characteristics of mixture and non-mixture cure models, and demonstrate their use in a range of scenarios, providing Stata code. We highlight key issues that must be taken into account by analysts when fitting these models and by reviewers and decision-makers when interpreting their predictions. In particular, we note that flexible parametric non-mixture cure models have not been used in HTA, but they offer advantages that make them well suited to an HTA context when a cure assumption is valid but follow-up is limited

The Cost of Costing Treatments Incorrectly: Errors in the Application of Drug Prices in Economic Evaluation Due to Failing to Account for the Distribution of Patient Weight

AbstractBackgroundThe cost of pharmaceuticals dosed by weight or body surface area (BSA) can be estimated in several ways for economic evaluations. A review of 20 recent National Institute for Health and Care Excellence appraisals showed that 17 of them took the mean weight or BSA of patients, 2 costed the individual patient data from trials, and 2 fitted a distribution to patient-level data.ObjectivesTo investigate the estimated drug costs using different methodologies to account for patient characteristics for pharmaceuticals with a weight- or BSA-based posology. The secondary objective was to explore the suitability of general population data as a proxy for patient-level data.MethodsPatient-level data were pooled from three clinical trials and used to calculate a hypothetical cost per administration of eight licensed pharmaceuticals, applying the three methods used in recent National Institute for Health and Care Excellence appraisals. The same analysis was performed using data from the Health Survey for England (in place of patient-level data) to investigate the validity of using general population data as a substitute for patient-level data.ResultsCompared with using patient-level data from clinical trials, the mean patient characteristics (weight or BSA) led to an underestimation of drug cost by 6.1% (range +1.5% to −25.5%). Fitting a distribution to patient-level data led to a mean difference of +0.04%. All estimates were consistent using general population data.ConclusionsEstimation of drug costs in health economic evaluation should account for the distribution in weight or BSA to produce accurate results. When patient data are not available, general population data may be used as an alternative

A cautionary tale: an evaluation of the performance of treatment switching adjustment methods in a real world case study

Background Treatment switching in randomised controlled trials (RCTs) is a problem for health technology assessment when substantial proportions of patients switch onto effective treatments that would not be available in standard clinical practice. Often statistical methods are used to adjust for switching: these can be applied in different ways, and performance has been assessed in simulation studies, but not in real-world case studies. We assessed the performance of adjustment methods described in National Institute for Health and Care Excellence Decision Support Unit Technical Support Document 16, applying them to an RCT comparing panitumumab to best supportive care (BSC) in colorectal cancer, in which 76% of patients randomised to BSC switched onto panitumumab. The RCT resulted in intention-to-treat hazard ratios (HR) for overall survival (OS) of 1.00 (95% confidence interval [CI] 0.82–1.22) for all patients, and 0.99 (95% CI 0.75–1.29) for patients with wild-type KRAS (Kirsten rat sarcoma virus). Methods We tested several applications of inverse probability of censoring weights (IPCW), rank preserving structural failure time models (RPSFTM) and simple and complex two-stage estimation (TSE) to estimate treatment effects that would have been observed if BSC patients had not switched onto panitumumab. To assess the performance of these analyses we ascertained the true effectiveness of panitumumab based on: (i) subsequent RCTs of panitumumab that disallowed treatment switching; (ii) studies of cetuximab that disallowed treatment switching, (iii) analyses demonstrating that only patients with wild-type KRAS benefit from panitumumab. These sources suggest the true OS HR for panitumumab is 0.76–0.77 (95% CI 0.60–0.98) for all patients, and 0.55–0.73 (95% CI 0.41–0.93) for patients with wild-type KRAS. Results Some applications of IPCW and TSE provided treatment effect estimates that closely matched the point-estimates and CIs of the expected truths. However, other applications produced estimates towards the boundaries of the expected truths, with some TSE applications producing estimates that lay outside the expected true confidence intervals. The RPSFTM performed relatively poorly, with all applications providing treatment effect estimates close to 1, often with extremely wide confidence intervals. Conclusions Adjustment analyses may provide unreliable results. How each method is applied must be scrutinised to assess reliability

Adjusting Overall Survival Estimates after Treatment Switching: a Case Study in Metastatic Castration-Resistant Prostate Cancer

Background If patients in oncology trials receive subsequent therapy, standard intention-to-treat (ITT) analyses may inaccurately estimate the overall survival (OS) effect of the investigational product. In this context, a post-hoc analysis of the phase 3 PREVAIL study was performed with the aim to compare enzalutamide with placebo in terms of OS, adjusting for potential confounding from switching to antineoplastic therapies that are not part of standard metastatic castration-resistant prostate cancer (mCRPC) treatment pathways in some jurisdictions. Methods The PREVAIL study, which included 1717 chemotherapy-naïve men with mCRPC randomized to treatment with enzalutamide 160 mg/day or placebo, was stopped after a planned interim survival analysis revealed a benefit in favor of enzalutamide. Data from this cutoff point were confounded by switching from both arms and so were evaluated in terms of OS using two switching adjustment methods: the two-stage accelerated failure time model (two-stage method) and inverse probability of censoring weights (IPCW). Results Following adjustment for switching to nonstandard antineoplastic therapies by 14.8 (129/872 patients) and 21.3% (180/845 patients) of patients initially randomized to enzalutamide and placebo, respectively, the two-stage and IPCW methods both resulted in numerical reductions in the hazard ratio (HR) for OS [HR 0.66, 95% confidence interval (CI) 0.57–0.81 and HR 0.63, 95% CI 0.52–0.75, respectively] for enzalutamide compared to placebo versus the unadjusted ITT analysis (HR 0.71, 95% CI 0.60–0.84). These results suggest a slightly greater effect of enzalutamide on OS than originally reported. Conclusion In the PREVAIL study, switching to nonstandard antineoplastic mCRPC therapies resulted in the ITT analysis of primary data underestimating the benefit of enzalutamide on OS

Leveraging real-world data to assess treatment sequences in health economic evaluations: a study protocol for emulating target trials using the English Cancer Registry and US Electronic Health Records-Derived Database

Background Considering the sequence of treatments is vital for optimising healthcare resource allocation, especially in cancer care, where sequence changes can affect patients’ overall survival and associated costs. A key challenge in evaluating treatment sequences in health technology assessments (HTA) is the scarce evidence on effectiveness, leading to uncertainties in decision making. While randomised controlled trials (RCTs) and meta-analyses are viewed as the gold standards for evidence, applying them to determine the effectiveness of treatment sequences in economic models often necessitates making arbitrary assumptions due to insufficient information on patients' treatment histories and subsequent therapies. In contrast, real-world data (RWD) presents a promising alternative source of evidence, often encompassing details across treatment lines. However, due to its non-randomised nature, estimates of the treatment effectiveness based on RWD analyses can be susceptible to biases if not properly adjusted for confounding factors. To date, several international initiatives have been investigating methods to derive reliable treatment effects from RWD — by emulating Target Trials that replicate existing RCTs (i.e. benchmarks) and comparing the emulated results against the benchmarks. These studies primarily seek to determine the viability of obtaining trial-equivalent results through deploying specific analytical methodologies and study designs within the Target Trial emulation framework, using a given database. Adopting the Target Trial emulation framework facilitates the analyses to be operated under causal inference principles. Upon validation in a particular database, these techniques can be applied to address similar questions (e.g., same disease area, same outcome type), but in populations lacking clinical trial evidence, leveraging the same RWD source. Studies to date, however, have predominantly focused on the comparison of individual treatments rather than treatment sequences. Moreover, the majority of these investigations have been undertaken in non-English contexts. Consequently, the use of RWD in evaluating treatment sequences for HTA, especially in an English setting, remains largely unexplored. Objectives The goal of this project is to investigate the feasibility of leveraging RWD to produce reliable, trial-like effectiveness estimates for treatment sequences. We aim to assess the capability of two oncology databases: the US-based Flatiron electronic health record and the National Cancer Registration and Analysis Service (NCRAS) database of England. To achieve this, we plan to harness the Target Trial Emulation (TTE) framework for replicating two existing oncology RCTs that compared treatment sequences, with the intent of benchmarking our results against the original studies. Further, we aim to detail the practicalities involved with implementing TTE in diverse databases and outline the challenges encountered. Methods 1. We aim to emulate existing RCTs that compare the effect of different treatment sequences by constructing the study design and analysis plan following the TTE framework. Specifically, the following case studies are planned: (1) Prostate cancer case study 1 (PC1) - US direct proof-of-concept study (method direct validation): replicating the GUTG-001 trial using Flatiron data (2) Prostate cancer case study 2 (PC2) - US-England bridging study (method extension): emulating Target Trials that compare treatment sequences that have been common in England using Flatiron data (3) Prostate cancer case study 3 (PC3) - English indirect proof-of-concept study (method indirect validation): emulating the same Target Trial in PC2 using English NCRAS data (4) Renal cell carcinoma case study (RCC) - method direct validation in a single-arm setting: emulating the sunitinib followed by everolimus arm in the RECORD-3 trial using English NCRAS data 2. We will compare results of the emulated Target Trials with those from the benchmark trials. 3. We plan to compare different advanced causal inference methods (e.g. marginal structural models using IPW and other g-methods) in estimating the effect of treatment sequences in RWD. Expected results This study will provide evidence on whether it is feasible to obtain reliable estimates of the (comparative) effectiveness of treatment sequences using Flatiron data and English NCRAS data. If applicable, we intend to develop a framework that provides a systematic way of obtaining the (comparative) effectiveness of treatment sequences using RWD. It is possible that the data quality is insufficient to emulate the planned Target Trials. In this case, we will report reasons for the implausibility of data analysis. If applicable, we will make suggestions to whether the national health data collection may be enhanced to make the analyses possible. The results of this study will be submitted to peer-reviewed journals and international conferences

A systematic review of methods to incorporate external evidence into trial-based survival extrapolations for health technology assessment

Background External evidence is commonly used to inform survival modeling for health technology assessment (HTA). While there are a range of methodological approaches that have been proposed, it is unclear which methods could be used and how they compare. Purpose This review aims to identify, describe, and categorize established methods to incorporate external evidence into survival extrapolation for HTA. Data Sources Embase, MEDLINE, EconLit, and Web of Science databases were searched to identify published methodological studies, supplemented by hand searching and citation tracking. Study Selection Eligible studies were required to present a novel extrapolation approach incorporating external evidence (i.e., data or information) within survival model estimation. Data Extraction Studies were classified according to how the external evidence was integrated as a part of model fitting. Information was extracted concerning the model-fitting process, key requirements, assumptions, software, application contexts, and presentation of comparisons with, or validation against, other methods. Data Synthesis Across 18 methods identified from 22 studies, themes included use of informative prior(s) (n = 5), piecewise (n = 7), and general population adjustment (n = 9), plus a variety of “other” (n = 8) approaches. Most methods were applied in cancer populations (n = 13). No studies compared or validated their method against another method that also incorporated external evidence. Limitations As only studies with a specific methodological objective were included, methods proposed as part of another study type (e.g., an economic evaluation) were excluded from this review. Conclusions Several methods were identified in this review, with common themes based on typical data sources and analytical approaches. Of note, no evidence was found comparing the identified methods to one another, and so an assessment of different methods would be a useful area for further research. Highlights This review aims to identify methods that have been used to incorporate external evidence into survival extrapolations, focusing on those that may be used to inform health technology assessment. We found a range of different approaches, including piecewise methods, Bayesian methods using informative priors, and general population adjustment methods, as well as a variety of “other” approaches. No studies attempted to compare the performance of alternative methods for incorporating external evidence with respect to the accuracy of survival predictions. Further research investigating this would be valuable

Assessing methods for dealing with treatment crossover in clinical trials: A follow-up simulation study

Background: Treatment switching commonly occurs in clinical trials of novel interventions, particularly in the advanced or metastatic cancer setting, which causes important problems for health technology assessment. Previous research has demonstrated which adjustment methods are suitable in specific scenarios, but scenarios considered have been limited. Objectives: We aimed to assess statistical approaches for adjusting survival estimates in the presence of treatment switching in order to determine which methods are most appropriate in a new range of realistic scenarios, building upon previous research. In particular we consider smaller sample sizes, reduced switching proportions, increased levels of censoring, and alternative data generating models. Methods: We conducted a simulation study to assess the bias, mean squared error and coverage associated with alternative switching adjustment methods across a wide range of realistic scenarios. Results: Our results generally supported those found in previous research, but the novel scenarios considered meant that we could make conclusions based upon a more robust evidence base. Simple methods such as censoring or excluding patients that switch again resulted in high levels of bias. More complex randomisation-based methods (e.g. Rank Preserving Structural Failure Time Models (RPSFTM)) were unbiased when the “common treatment effect” held. Observational-based methods (e.g. inverse probability of censoring weights (IPCW)) coped better with time-dependent treatment effects but are heavily data reliant, and generally led to higher levels of bias in our simulations. Novel “two stage” methods produced relatively low bias across all simulated scenarios. All methods generally produced higher bias when the simulated sample size was smaller and when the censoring proportion was higher. All methods generally produced lower bias when switching proportions were lower. We find that the size of the treatment effect in terms of an acceleration factor has an important bearing on the levels of bias associated with the adjustment methods. Conclusions: Randomisation-based methods can accurately adjust for treatment switching when the treatment effect received by patients that switch is the same as that received by patients randomised to the experimental group. When this is not the case observational-based methods or simple twostage methods should be considered, although the IPCW is prone to substantial bias when the proportion of patients that switch is greater than approximately 90%. Simple methods such as censoring or excluding patients that switch should not be used