Reporting of covariate selection and balance assessment in propensity score analysis is suboptimal: a systematic review

Abstract Objectives: To assess the current practice of propensity score (PS) analysis in the medical literature, particularly the assessment and reporting of balance on confounders. Study Design and Setting: A PubMed search identified studies using PS methods from December 2011 through May 2012. For each article included in the review, information was extracted on important aspects of the PS such as the type of PS method used, variable selection for PS model, and assessment of balance. Results: Among 296 articles that were included in the review, variable selection for PS model was explicitly reported in 102 studies (34.4%). Covariate balance was checked and reported in 177 studies (59.8%). P-values were the most commonly used statistical tools to report balance (125 of 177, 70.6%). The standardized difference and graphical displays were reported in 45 (25.4%) and 11 (6.2%) articles, respectively. Matching on the PS was the most commonly used approach to control for confounding (68.9%), followed by PS adjustment (20.9%), PS stratification (13.9%), and inverse probability of treatment weighting (IPTW, 7.1%). Balance was more often checked in articles using PS matching and IPTW, 70.6% and 71.4%, respectively. Conclusion: The execution and reporting of covariate selection and assessment of balance is far from optimal. Recommendations on reporting of PS analysis are provided to allow better appraisal of the validity of PS-based studies.

A constructive probabilistic proof of Choquet's theorem

Contains fulltext : 18718.pdf ( ) (Open Access)Report no. 99135 p

Creating and classifying measures of linear association by optimization techniques

The idea of measures of linear association, such as Pearson's correlation coefficient, can be put in a general framework by axiomization. Groups of linear transformations on $\mathsf{R}^n$ can be exploited to create new and classify existing measures according to their invariance properties. Thus invariance under the Euclidean transformation group leads to the class of so-called geometric measures. Similarly, a measure is called algebraic if it is invariant under scalings. Pearson's coefficient is an example of an algebraic measure; it is not geometric. It is proved that, generally, a measure of linear association cannot possibly be both geometric and algebraic. A procedure is developed to convert a geometric measure into an algebraic and vice versa. Thus a kind of a duality between algebraic and geometric measures arises. In this duality measures can be reflexive or not

A reverse to the Jeffreys–Lindley paradox

 In this paper the seminal Jeffreys–Lindley paradox is regarded from a mathematical point of view. We show that in certain scenarios the paradox may emerge in a reverse direction. In this paper the seminal Jeffreys–Lindley paradox is regarded from a mathematical point of view. We show that in certain scenarios the paradox may emerge in a reverse direction

A computational approach to compare regression modelling strategies in prediction research

BACKGROUND: It is often unclear which approach to fit, assess and adjust a model will yield the most accurate prediction model. We present an extension of an approach for comparing modelling strategies in linear regression to the setting of logistic regression and demonstrate its application in clinical prediction research. METHODS: A framework for comparing logistic regression modelling strategies by their likelihoods was formulated using a wrapper approach. Five different strategies for modelling, including simple shrinkage methods, were compared in four empirical data sets to illustrate the concept of a priori strategy comparison. Simulations were performed in both randomly generated data and empirical data to investigate the influence of data characteristics on strategy performance. We applied the comparison framework in a case study setting. Optimal strategies were selected based on the results of a priori comparisons in a clinical data set and the performance of models built according to each strategy was assessed using the Brier score and calibration plots. RESULTS: The performance of modelling strategies was highly dependent on the characteristics of the development data in both linear and logistic regression settings. A priori comparisons in four empirical data sets found that no strategy consistently outperformed the others. The percentage of times that a model adjustment strategy outperformed a logistic model ranged from 3.9 to 94.9 %, depending on the strategy and data set. However, in our case study setting the a priori selection of optimal methods did not result in detectable improvement in model performance when assessed in an external data set. CONCLUSION: The performance of prediction modelling strategies is a data-dependent process and can be highly variable between data sets within the same clinical domain. A priori strategy comparison can be used to determine an optimal logistic regression modelling strategy for a given data set before selecting a final modelling approach

Measuring balance and model selection in propensity score methods

Background: Propensity score (PS) methods focus on balancing confounders between groups to estimate an unbiased treatment or exposure effect. However, there is lack of attention in actually measuring, reporting and using the information on balance, for instance for model selection. Objectives: To describe and evaluate measures for balance in PS methods: the overlapping coefficient, the Kolmogorov- Smirnov distance, and the Lévy distance, and mean based measures for balance. Methods: We performed simulation studies to estimate the association between these three and several mean based measures for balance and bias (i.e., discrepancy between the true and the estimated treatment effect). Results: For large sample sizes (n=2000) the average Pearson's correlation coefficients between bias and Kolmogorov- Smirnov distance (r=0.89), the Lévy distance (r=0.89) and the absolute standardized mean difference (r=0.90) were similar, whereas this was lower for the overlapping coefficient (r= -0.42). When sample size decreased to 400, mean based measures of balance had stronger correlations with bias. Models including all confounding variables, their squares and interaction terms resulted in smaller bias than models that included only main terms for confounding variables. Conclusions: We conclude that measures for balance are useful for reporting the amount of balance reached in propensity score analysis and can be helpful in selecting the final PS model