Applying Matching Procedures in the Generation and Synthesis of Evidence

The gold standard for clinical studies are blinded randomized trials, but such a design is not always feasible due to ethical or practical reasons. Using an external historical control group out of an earlier conducted trial or registry might be an option. When using historical controls, one often faces the situation of non-comparable study populations. Matching procedures may help to build balanced samples for comparison. In this thesis an adaptive matched case-control trial design is established, which allows for a sample size recalculation at a planned interim analysis with the goal to enhance the matching rate at final analysis. The recalculation is based on the lower confidence interval limit of the matching rate observed at interim analysis. The newly developed resampling CI method estimates the 1:1 matching rate using a bootstrap like procedure (without replacement) and equal-sized groups for matching at interim. A naïve approach would be to use all patients for estimating the matching rate and directly reflect this value for recalculating the sample size. The new approach shows good performance in terms of power and type I error rate but needs more newly recruited patients than the naïve approach. Additionally, investigations for the time point of interim analysis are done. Simulations result in a number of 1/2 to 2/3 of the control patients, however, it seems that the time point is more depending on the actual number of patients used for matching than on the proportion. However, if the historical control group is large and for example only a small phase II trial is feasible the before described method might not be a good choice. Rather, each intervention patient may find more than one matching partner. Therefore, an iterative procedure to determining the number of matching partners is developed. The idea is an interim analysis, which includes an iterative increase in the number of matching partners and a parallel calculation of the matching rate. The number will be increased as long as the 1:M matching rate is higher than the 1:1 matching rate including a potential tolerance. The 1:M matching rate at interim analysis can then be used for recalculating the sample size. This procedure is easy to implement and can be combined with many study designs, such as two-stage designs. One has to note that the number of matching partners highly depends on the overlap of patient populations, meaning a small overlap leads to a low number of matching partners and vice versa. To conclude, by involving the trial-specific matching rate in the sample size recalculation one is able to enhance power in a matched case-control trial. Not only in the generation of evidence unbalanced patient cohorts arise, but also in evidence synthesis this poses a problem. A common situation in evidence synthesis is an indirect comparison, where the comparison of interest, assume treatment A versus C, is not examined in a direct comparison. But there are trials comparing A with treatment B and another trial comparing C and B. using those trials to calculate a treatment effect for A versus C is called indirect comparison. It is likely that the independent trials AB and CB do not have the same underlying population. A special case, where individual patient data is available for one of the trials is assumed. Then a matching-like procedure can help to balance the cohorts, this method is called matching adjusted indirect comparison which is not sufficiently examined, yet. Another widely used method for indirect comparisons is the method of Bucher. A method comparison between those two methods is conducted for clinically relevant scenarios where assumptions of the methods are violated. Simulations lead to the conjecture that indirect comparisons are considerably underpowered. The method of Bucher and the matching adjusted indirect comparison show similar performance in scenarios without cross-trial differences. The matching approach leads to higher coverage and power when populations differ, effect modifiers are present, and regression models are not sufficiently adjusted. But matching confounders which do not modify the effect leads to increased bias. Until now, indirect comparisons are applied using one study per treatment comparison because the matching adjusted indirect comparison is designed for this setting. Nevertheless, it is likely that there are two or even more studies comparing the same treatments. When synthesizing evidence, one should always aim to include all appropriate evidence. Therefore, approaches to include multiple studies in indirect comparisons are introduced and compared. All include a step for combining treatment effects and one for calculating indirect treatment effects. The main difference between the approaches is the order of those two steps. An increasing number of studies can enhance power to desired regions above 80%, but it was not possible to identify one best performing method over all considered scenarios. In conclusion, when applying matching procedures in evidence synthesis the underlying situation needs to be checked carefully, and matching variables need to be chosen carefully because adjusting for confounders influences the precision of the indirect comparison

Methods for Population Adjustment with Limited Access to Individual Patient Data: A Review and Simulation Study

Population-adjusted indirect comparisons estimate treatment effects when access to individual patient data is limited and there are cross-trial differences in effect modifiers. Popular methods include matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC). There is limited formal evaluation of these methods and whether they can be used to accurately compare treatments. Thus, we undertake a comprehensive simulation study to compare standard unadjusted indirect comparisons, MAIC and STC across 162 scenarios. This simulation study assumes that the trials are investigating survival outcomes and measure continuous covariates, with the log hazard ratio as the measure of effect. MAIC yields unbiased treatment effect estimates under no failures of assumptions. The typical usage of STC produces bias because it targets a conditional treatment effect where the target estimand should be a marginal treatment effect. The incompatibility of estimates in the indirect comparison leads to bias as the measure of effect is non-collapsible. Standard indirect comparisons are systematically biased, particularly under stronger covariate imbalance and interaction effects. Standard errors and coverage rates are often valid in MAIC but the robust sandwich variance estimator underestimates variability where effective sample sizes are small. Interval estimates for the standard indirect comparison are too narrow and STC suffers from bias-induced undercoverage. MAIC provides the most accurate estimates and, with lower degrees of covariate overlap, its bias reduction outweighs the loss in effective sample size and precision under no failures of assumptions. An important future objective is the development of an alternative formulation to STC that targets a marginal treatment effect.Comment: 73 pages (34 are supplementary appendices and references), 8 figures, 2 tables. Full article (following Round 4 of minor revisions). arXiv admin note: text overlap with arXiv:2008.0595

Secukinumab versus adalimumab for psoriatic arthritis: comparative effectiveness up to 48 weeks using a matching-adjusted indirect comparison

Secukinumab and adalimumab are approved for adults with active psoriatic arthritis (PsA). In the absence of direct randomized controlled trial (RCT) data, matching-adjusted indirect comparison can estimate the comparative effectiveness in anti-tumor necrosis factor (TNF)-naïve populations. Individual patient data from the FUTURE 2 RCT (secukinumab vs. placebo; N = 299) were adjusted to match baseline characteristics of the ADEPT RCT (adalimumab vs. placebo; N = 313). Logistic regression determined adjustment weights for age, body weight, sex, race, methotrexate use, psoriasis affecting ≥ 3% of body surface area, Psoriasis Area and Severity Index score, Health Assessment Questionnaire Disability Index score, presence of dactylitis and enthesitis, and previous anti-TNF therapy. Recalculated secukinumab outcomes were compared with adalimumab outcomes at weeks 12 (placebo-adjusted), 16, 24, and 48 (nonplacebo-adjusted). After matching, the effective sample size for FUTURE 2 was 101. Week 12 American College of Rheumatology (ACR) response rates were not significantly different between secukinumab and adalimumab. Week 16 ACR 20 and 50 response rates were higher for secukinumab 150 mg than for adalimumab (P = 0.017, P = 0.033), as was ACR 50 for secukinumab 300 mg (P = 0.030). Week 24 ACR 20 and 50 were higher for secukinumab 150 mg than for adalimumab (P = 0.001, P = 0.019), as was ACR 20 for secukinumab 300 mg (P = 0.048). Week 48 ACR 20 was higher for secukinumab 150 and 300 mg than for adalimumab (P = 0.002, P = 0.027), as was ACR 50 for secukinumab 300 mg (P = 0.032). In our analysis, patients with PsA receiving secukinumab were more likely to achieve higher ACR responses through 1 year (weeks 16-48) than those treated with adalimumab. Although informative, these observations rely on a subgroup of patients from FUTURE 2 and thus should be considered interim until the ongoing head-to-head RCT EXCEED can validate these findings. Novartis Pharma AG

Population-adjusted treatment comparisons:estimates based on MAIC (Matching-Adjusted Indirect Comparisons) and STC (Simulated Treatment Comparisons)

OBJECTIVES: To review the properties and assumptions of methods for population-adjusted treatment comparison, including Matching-Adjusted Indirect Comparison (MAIC) and Simulated Treatment Comparison (STC), and to provide guidance on their use in health technology appraisal.METHODS: Standard methods for indirect comparisons and network meta-analysis are based on aggregate data, with the key assumption that there is no difference between trials in the distribution of effect-modifying variables. Two methods which relax this assumption, MAIC and STC, are becoming increasingly common in industry-sponsored treatment comparisons, where a company has access to individual patient data (IPD) from its own trials but only aggregate information from competitor trials. Both methods use IPD to adjust for between-trial differences in covariate distributions. We review the properties of these methods in light of the wider literature on standardisation and calibration based on propensity score reweighting and covariate adjustment, which are the foundation for MAIC and STC respectively, and identify the key assumptions in the context of indirect comparisons.RESULTS: There is a lack of clarity about how and when the methods should be applied in practice, and both MAIC and STC as currently applied can only produce population-adjusted estimates that are valid for the populations in the competitor trials, rather than the target population for the decision. In addition, the fundamental distinction between “anchored” and “unanchored” forms of indirect comparison – where a common comparator arm is or is not utilised to control for between-trial differences in prognostic variables – is under-emphasised, with the unanchored comparison making assumptions that are infeasibly strong.CONCLUSIONS: We provide recommendations on how and when population adjustment methods of this type should be used in order to provide statistically valid, clinically meaningful, transparent and consistent results for any given target population, and set out the additional analyses that should be presented to support their use

Estimating marginal treatment effects from observational studies and indirect treatment comparisons: When are standardization-based methods preferable to those based on propensity score weighting?

In light of newly developed standardization methods, we evaluate, via simulation study, how propensity score weighting and standardization -based approaches compare for obtaining estimates of the marginal odds ratio and the marginal hazard ratio. Specifically, we consider how the two approaches compare in two different scenarios: (1) in a single observational study, and (2) in an anchored indirect treatment comparison (ITC) of randomized controlled trials. We present the material in such a way so that the matching-adjusted indirect comparison (MAIC) and the (novel) simulated treatment comparison (STC) methods in the ITC setting may be viewed as analogous to the propensity score weighting and standardization methods in the single observational study setting. Our results suggest that current recommendations for conducting ITCs can be improved and underscore the importance of adjusting for purely prognostic factors.Comment: 33 page

Effect modification in anchored indirect treatment comparisons: Comments on "Matching-adjusted indirect comparisons: Application to time-to-event data"

This commentary regards a recent simulation study conducted by Aouni, Gaudel-Dedieu and Sebastien, evaluating the performance of different versions of matching-adjusted indirect comparison (MAIC) in an anchored scenario with a common comparator. The simulation study uses survival outcomes and the Cox proportional hazards regression as the outcome model. It concludes that using the LASSO for variable selection is preferable to balancing a maximal set of covariates. However, there are no treatment effect modifiers in imbalance in the study. The LASSO is more efficient because it selects a subset of the maximal set of covariates but there are no cross-study imbalances in effect modifiers inducing bias. We highlight the following points: (1) in the anchored setting, MAIC is necessary where there are cross-trial imbalances in effect modifiers; (2) the standard indirect comparison provides greater precision and accuracy than MAIC if there are no effect modifiers in imbalance; (3) while the target estimand of the simulation study is a conditional treatment effect, MAIC targets a marginal or population-average treatment effect; (4) in MAIC, variable selection is a problem of low dimensionality and sparsity-inducing methods like the LASSO may be problematic. Finally, data-driven approaches do not obviate the necessity for subject matter knowledge when selecting effect modifiers. R code is provided in the Appendix to replicate the analyses and illustrate our points.Comment: 14 pages, minor changes after conditional acceptance by Statistics in Medicine. This is a response to `Matching-adjusted indirect comparisons: Application to time-to-event data' by Aouni, Gaudel-Dedieu and Sebastien (2020

The Effects of Model Misspecification in Unanchored Matching-Adjusted Indirect Comparison (MAIC): Results of a Simulation Study

OBJECTIVES: To assess the performance of unanchored matching-adjusted indirect comparison (MAIC) by matching on first moments or higher moments in a cross-study comparisons under a variety of conditions. A secondary objective was to gauge the performance of the method relative to propensity score weighting (PSW). METHODS: A simulation study was designed based on an oncology example, where MAIC was used to account for differences between a contemporary trial in which patients had more favorable characteristics and a historical control. A variety of scenarios were then tested varying the setup of the simulation study, including violating the implicit or explicit assumptions of MAIC. RESULTS: Under ideal conditions and under a variety of scenarios, MAIC performed well (shown by a low mean absolute error [MAE]) and was unbiased (shown by a mean error [ME] of about zero). The performance of the method deteriorated where the matched characteristics had low explanatory power or there was poor overlap between studies. Only when important characteristics are not included in the matching did the method become biased (nonzero ME). Where the method showed poor performance, this was exaggerated if matching was also performed on the variance (ie, higher moments). Relative to PSW, MAIC provided similar results in most circumstances, although it exhibited slightly higher MAE and a higher chance of exaggerating bias. CONCLUSIONS: MAIC appears well suited to adjust for cross-trial comparisons provided the assumptions underpinning the model are met, with relatively little efficiency loss compared with PSW

Comparison of Nusinersen and Evrysdi in the Treatment of Spinal Muscular Atrophy

Spinal Muscular Atrophy (SMA) is a genetic neuromuscular disease that commonly affects children, and usually worsens with age that often leads to permanent disability and death for many of the SMA patients. Recently, two drugs are developed to improving the quality of life of SMA sufferers: Evrysdi and Nusinersen. This study is identified by a systematic literature review to compare two treatments. The comparison attempts to focus on mechanism, administration and clinical trials. The trials include the ENDEAR study for Nusinersen, and the FIREFISH study for Evrysdi. Due to the different baselines of two trials, matching-adjusted indirect comparison (MAIC) is used to “weighted” baseline characteristics to match each other across all the studies. Each of the trials highlighted the effectiveness for comparison. Both Nusinersen and Evrysdi have had a major and positive impact on improving the quality of life of SMA, and both therapies have been shown to be highly effective. Moreover, the indirect comparison with Matching Adjustment Indirect Comparison shows that Risdiplam is more effective as compared to Nusinersen. Nonetheless, the comparison is still inaccurate due to lack of real-world evidence from patients