Comparing paired vs non-paired statistical methods of analyses when making inferences about absolute risk reductions in propensity-score matched samples

Propensity-score matching allows one to reduce the effects of treatment-selection bias or confounding when estimating the effects of treatments when using observational data. Some authors have suggested that methods of inference appropriate for independent samples can be used for assessing the statistical significance of treatment effects when using propensity-score matching. Indeed, many authors in the applied medical literature use methods for independent samples when making inferences about treatment effects using propensity-score matched samples. Dichotomous outcomes are common in healthcare research. In this study, we used Monte Carlo simulations to examine the effect on inferences about risk differences (or absolute risk reductions) when statistical methods for independent samples are used compared with when statistical methods for paired samples are used in propensity-score matched samples. We found that compared with using methods for independent samples, the use of methods for paired samples resulted in: (i) empirical type I error rates that were closer to the advertised rate; (ii) empirical coverage rates of 95 per cent confidence intervals that were closer to the advertised rate; (iii) narrower 95 per cent confidence intervals; and (iv) estimated standard errors that more closely reflected the sampling variability of the estimated risk difference. Differences between the empirical and advertised performance of methods for independent samples were greater when the treatment-selection process was stronger compared with when treatment-selection process was weaker. We recommend using statistical methods for paired samples when using propensity-score matched samples for making inferences on the effect of treatment on the reduction in the probability of an event occurring. Copyright © 2011 John Wiley & Sons, Ltd

Practical recommendations for reporting Fine-Gray model analyses for competing risk data

In survival analysis, a competing risk is an event whose occurrence precludes the occurrence of the primary event of interest. Outcomes in medical research are frequently subject to competing risks. In survival analysis, there are 2 key questions that can be addressed using competing risk regression models: first, which covariates affect the rate at which events occur, and second, which covariates affect the probability of an event occurring over time. The cause‐specific hazard model estimates the effect of covariates on the rate at which events occur in subjects who are currently event‐free. Subdistribution hazard ratios obtained from the Fine‐Gray model describe the relative effect of covariates on the subdistribution hazard function. Hence, the covariates in this model can also be interpreted as having an effect on the cumulative incidence function or on the probability of events occurring over time. We conducted a review of the use and interpretation of the Fine‐Gray subdistribution hazard model in articles published in the medical literature in 2015. We found that many authors provided an unclear or incorrect interpretation of the regression coefficients associated with this model. An incorrect and inconsistent interpretation of regression coefficients may lead to confusion when comparing results across different studies. Furthermore, an incorrect interpretation of estimated regression coefficients can result in an incorrect understanding about the magnitude of the association between exposure and the incidence of the outcome. The objective of this article is to clarify how these regression coefficients should be reported and to propose suggestions for interpreting these coefficients

Alkali metal cation interactions with 12-crown-4 in the gas phase: revisited

pre-printQuantitative interactions of alkali metal cations with the cyclic 12-crown-4 polyether ligand (12C4) are studied. Experimentally, Rb+(12C4) and Cs+(12C4) complexes are formed using electrospray ionization and their bond dissociation energies (BDEs) determined using threshold collision-induced dissociation of these complexes with xenon in a guided ion beam tandem mass spectrometer. The energy-dependent cross sections thus obtained are interpreted using an analysis that includes consideration of unimolecular decay rates, internal energy of the reactant ions, and multiple ion-neutral collisions. 0 K BDEs of 151.5 9.7 and 137.0 8.7 kJ/mol, respectively, are determined and exceed those previously measured by 60 and 54 kJ/mol, respectively, consistent with the hypothesis proposed there that excited conformers had been studied. In order to provide comparable thermochemical results for the Na+(12C4) and K+(12C4) systems, the published data for these systems are reinterpreted using the same analysis techniques, which have advanced since the original data were acquired. Revised BDEs for these systems are obtained as 243.9 12.6 and 182.0 17.3 kJ/mol, respectively, which are within experimental uncertainty of the previously reported values. In addition, quantum chemical calculations are conducted at the B3LYP and MP2(full) levels of theory with geometries and zero point energies calculated at the B3LYP level using both HW*/6-311+G(2d,2p) and def2-TZVPPD basis sets. The theoretical results are in reasonable agreement with experiment, with B3LYP/def2-TZVPPD values being in particularly good agreement. Computations also allow the potential energy surfaces for dissociation of the M+(12C4) complexes to be elucidated. These are used to help explain why the previous studies formed excited conformers of Rb+(12C4) and Cs+(12C4) but apparently not of Na+(12C4) and K+(12C4)

The Use of Bootstrapping when Using Propensity-Score Matching without Replacement: A Simulation Study

Propensity‐score matching is frequently used to estimate the effect of treatments, exposures, and interventions when using observational data. An important issue when using propensity‐score matching is how to estimate the standard error of the estimated treatment effect. Accurate variance estimation permits construction of confidence intervals that have the advertised coverage rates and tests of statistical significance that have the correct type I error rates. There is disagreement in the literature as to how standard errors should be estimated. The bootstrap is a commonly used resampling method that permits estimation of the sampling variability of estimated parameters. Bootstrap methods are rarely used in conjunction with propensity‐score matching. We propose two different bootstrap methods for use when using propensity‐score matching without replacement and examined their performance with a series of Monte Carlo simulations. The first method involved drawing bootstrap samples from the matched pairs in the propensity‐score‐matched sample. The second method involved drawing bootstrap samples from the original sample and estimating the propensity score separately in each bootstrap sample and creating a matched sample within each of these bootstrap samples. The former approach was found to result in estimates of the standard error that were closer to the empirical standard deviation of the sampling distribution of estimated effects

Accounting for competing risks in randomized controlled trials: a review and recommendations for improvement

In studies with survival or time-to-event outcomes, a competing risk is an event whose occurrence precludes the occurrence of the primary event of interest. Specialized statistical methods must be used to analyze survival data in the presence of competing risks. We conducted a review of randomized controlled trials with survival outcomes that were published in high-impact general medical journals. Of 40 studies that we identified, 31 (77.5%) were potentially susceptible to competing risks. However, in the majority of these studies, the potential presence of competing risks was not accounted for in the statistical analyses that were described. Of the 31 studies potentially susceptible to competing risks, 24 (77.4%) reported the results of a Kaplan-Meier survival analysis, while only five (16.1%) reported using cumulative incidence functions to estimate the incidence of the outcome over time in the presence of competing risks. The former approach will tend to result in an overestimate of the incidence of the outcome over time, while the latter approach will result in unbiased estimation of the incidence of the primary outcome over time. We provide recommendations on the analysis and reporting of randomized controlled trials with survival outcomes in the presence of competing risks. © 2017 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd

The number of primary events per variable affects estimation of the subdistribution hazard competing risks model

AbstractObjectivesTo examine the effect of the number of events per variable (EPV) on the accuracy of estimated regression coefficients, standard errors, empirical coverage rates of estimated confidence intervals, and empirical estimates of statistical power when using the Fine–Gray subdistribution hazard regression model to assess the effect of covariates on the incidence of events that occur over time in the presence of competing risks.Study Design and SettingMonte Carlo simulations were used. We considered two different definitions of the number of EPV. One included events of any type that occurred (both primary events and competing events), whereas the other included only the number of primary events that occurred.ResultsThe definition of EPV that included only the number of primary events was preferable to the alternative definition, as the number of competing events had minimal impact on estimation. In general, 40–50 EPV were necessary to ensure accurate estimation of regression coefficients and associated quantities. However, if all of the covariates are continuous or are binary with moderate prevalence, then 10 EPV are sufficient to ensure accurate estimation.ConclusionAnalysts must base the number of EPV on the number of primary events that occurred

Covariate balance in a Bayesian propensity score analysis of beta blocker therapy in heart failure patients

Regression adjustment for the propensity score is a statistical method that reduces confounding from measured variables in observational data. A Bayesian propensity score analysis extends this idea by using simultaneous estimation of the propensity scores and the treatment effect. In this article, we conduct an empirical investigation of the performance of Bayesian propensity scores in the context of an observational study of the effectiveness of beta-blocker therapy in heart failure patients. We study the balancing properties of the estimated propensity scores. Traditional Frequentist propensity scores focus attention on balancing covariates that are strongly associated with treatment. In contrast, we demonstrate that Bayesian propensity scores can be used to balance the association between covariates and the outcome. This balancing property has the effect of reducing confounding bias because it reduces the degree to which covariates are outcome risk factors

Introduction to the Analysis of Survival Data in the Presence of Competing Risks

Competing risks occur frequently in the analysis of survival data. A competing risk is an event whose occurrence precludes the occurrence of the primary event of interest. In a study examining time to death attributable to cardiovascular causes, death attributable to noncardiovascular causes is a competing risk. When estimating the crude incidence of outcomes, analysts should use the cumulative incidence function, rather than the complement of the Kaplan-Meier survival function. The use of the Kaplan-Meier survival function results in estimates of incidence that are biased upward, regardless of whether the competing events are independent of one another. When fitting regression models in the presence of competing risks, researchers can choose from 2 different families of models: modeling the effect of covariates on the cause-specific hazard of the outcome or modeling the effect of covariates on the cumulative incidence function. The former allows one to estimate the effect of the covariates on the rate of occurrence of the outcome in those subjects who are currently event free. The latter allows one to estimate the effect of covariates on the absolute risk of the outcome over time. The former family of models may be better suited for addressing etiologic questions, whereas the latter model may be better suited for estimating a patient’s clinical prognosis. We illustrate the application of these methods by examining cause-specific mortality in patients hospitalized with heart failure. Statistical software code in both R and SAS is provided