Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines

Background: Multiple imputation (MI) provides an effective approach to handle missing covariate data within prognostic modelling studies, as it can properly account for the missing data uncertainty. The multiply imputed datasets are each analysed using standard prognostic modelling techniques to obtain the estimates of interest. The estimates from each imputed dataset are then combined into one overall estimate and variance, incorporating both the within and between imputation variability. Rubin's rules for combining these multiply imputed estimates are based on asymptotic theory. The resulting combined estimates may be more accurate if the posterior distribution of the population parameter of interest is better approximated by the normal distribution. However, the normality assumption may not be appropriate for all the parameters of interest when analysing prognostic modelling studies, such as predicted survival probabilities and model performance measures. Methods: Guidelines for combining the estimates of interest when analysing prognostic modelling studies are provided. A literature review is performed to identify current practice for combining such estimates in prognostic modelling studies. Results: Methods for combining all reported estimates after MI were not well reported in the current literature. Rubin's rules without applying any transformations were the standard approach used, when any method was stated. Conclusion: The proposed simple guidelines for combining estimates after MI may lead to a wider and more appropriate use of MI in future prognostic modelling studies

Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study

Background: There is no consensus on the most appropriate approach to handle missing covariate data within prognostic modelling studies. Therefore a simulation study was performed to assess the effects of different missing data techniques on the performance of a prognostic model. Methods: Datasets were generated to resemble the skewed distributions seen in a motivating breast cancer example. Multivariate missing data were imposed on four covariates using four different mechanisms; missing completely at random (MCAR), missing at random (MAR), missing not at random (MNAR) and a combination of all three mechanisms. Five amounts of incomplete cases from 5% to 75% were considered. Complete case analysis (CC), single imputation (SI) and five multiple imputation (MI) techniques available within the R statistical software were investigated: a) data augmentation (DA) approach assuming a multivariate normal distribution, b) DA assuming a general location model, c) regression switching imputation, d) regression switching with predictive mean matching (MICE-PMM) and e) flexible additive imputation models. A Cox proportional hazards model was fitted and appropriate estimates for the regression coefficients and model performance measures were obtained. Results: Performing a CC analysis produced unbiased regression estimates, but inflated standard errors, which affected the significance of the covariates in the model with 25% or more missingness. Using SI, underestimated the variability; resulting in poor coverage even with 10% missingness. Of the MI approaches, applying MICE-PMM produced, in general, the least biased estimates and better coverage for the incomplete covariates and better model performance for all mechanisms. However, this MI approach still produced biased regression coefficient estimates for the incomplete skewed continuous covariates when 50% or more cases had missing data imposed with a MCAR, MAR or combined mechanism. When the missingness depended on the incomplete covariates, i.e. MNAR, estimates were biased with more than 10% incomplete cases for all MI approaches. Conclusion: The results from this simulation study suggest that performing MICE-PMM may be the preferred MI approach provided that less than 50% of the cases have missing data and the missing data are not MNAR

Multiple imputation for estimating hazard ratios and predictive abilities in case-cohort surveys

<p>Abstract</p> <p>Background</p> <p>The weighted estimators generally used for analyzing case-cohort studies are not fully efficient and naive estimates of the predictive ability of a model from case-cohort data depend on the subcohort size. However, case-cohort studies represent a special type of incomplete data, and methods for analyzing incomplete data should be appropriate, in particular multiple imputation (MI).</p> <p>Methods</p> <p>We performed simulations to validate the MI approach for estimating hazard ratios and the predictive ability of a model or of an additional variable in case-cohort surveys. As an illustration, we analyzed a case-cohort survey from the Three-City study to estimate the predictive ability of D-dimer plasma concentration on coronary heart disease (CHD) and on vascular dementia (VaD) risks.</p> <p>Results</p> <p>When the imputation model of the phase-2 variable was correctly specified, MI estimates of hazard ratios and predictive abilities were similar to those obtained with full data. When the imputation model was misspecified, MI could provide biased estimates of hazard ratios and predictive abilities. In the Three-City case-cohort study, elevated D-dimer levels increased the risk of VaD (hazard ratio for two consecutive tertiles = 1.69, 95%CI: 1.63-1.74). However, D-dimer levels did not improve the predictive ability of the model.</p> <p>Conclusions</p> <p>MI is a simple approach for analyzing case-cohort data and provides an easy evaluation of the predictive ability of a model or of an additional variable.</p

Non-Hematopoietic Essential Functions of Bone Marrow Cells: A Review of Scientific and Clinical Literature and Rationale for Treating Bone Defects.

Hematopoiesis as the only essential function of bone marrow cells has been challenged for several decades through basic science (in vitro and in vivo) and clinical data. Such work has shed light on two other essential functions of bone marrow cells: osteopoiesis and angio-genesis/vasculogenesis. Clinical utility of autologous concentrated bone marrow aspirate (CBMA) has demonstrated both safety and efficacy in treating bone defects. Moreover, CBMA has been shown to be comparable to the gold standard of iliac crest bone graft (ICBG), or autograft, with regard to being osteogenic and osteoinductive. ICBG is not considered an advanced therapy medicinal product (ATMP), but CBMA may become regulated as an ATMP. The European Medicines Agency Committee for Advanced Therapies (EMA:CAT) has issued a reflection paper (20 June 2014) in which reversal of the 2013 ruling that CBMA is a non-ATMP has been proposed. We review bone marrow cell involvement in osteopoiesis and angiogenesis/vasculogenesis to examine EMA:CAT 2013 decision to use CBMA for treatment of osteonecrosis (e.g, of the femoral head) should be considered a non-ATMP. This paper is intended to provide discussion on the 20 June 2014 reflection paper by reviewing two non-hematopoietic essential functions of bone marrow cells. Additionally, we provide clinical and scientific rationale for treating osteonecrosis with CBMA

Comparison of imputation methods for handling missing covariate data when fitting a Cox proportional hazards model: a resampling study

<p>Abstract</p> <p>Background</p> <p>The appropriate handling of missing covariate data in prognostic modelling studies is yet to be conclusively determined. A resampling study was performed to investigate the effects of different missing data methods on the performance of a prognostic model.</p> <p>Methods</p> <p>Observed data for 1000 cases were sampled with replacement from a large complete dataset of 7507 patients to obtain 500 replications. Five levels of missingness (ranging from 5% to 75%) were imposed on three covariates using a missing at random (MAR) mechanism. Five missing data methods were applied; a) complete case analysis (CC) b) single imputation using regression switching with predictive mean matching (SI), c) multiple imputation using regression switching imputation, d) multiple imputation using regression switching with predictive mean matching (MICE-PMM) and e) multiple imputation using flexible additive imputation models. A Cox proportional hazards model was fitted to each dataset and estimates for the regression coefficients and model performance measures obtained.</p> <p>Results</p> <p>CC produced biased regression coefficient estimates and inflated standard errors (SEs) with 25% or more missingness. The underestimated SE after SI resulted in poor coverage with 25% or more missingness. Of the MI approaches investigated, MI using MICE-PMM produced the least biased estimates and better model performance measures. However, this MI approach still produced biased regression coefficient estimates with 75% missingness.</p> <p>Conclusions</p> <p>Very few differences were seen between the results from all missing data approaches with 5% missingness. However, performing MI using MICE-PMM may be the preferred missing data approach for handling between 10% and 50% MAR missingness.</p

Predicting developmental dysplasia of the hip in at-risk newborns.

BACKGROUND: The development of developmental dysplasia of the hip can be attributed to several risk factors and often in combination with each other. When predicting the likelihood of developing this condition, clinicians tend to over and underestimate its likelihood of occurring. Therefore, the study aim is to determine among at-risk newborns how to best predict developmental dysplasia of the hip (DDH) within 8 weeks post-partum. METHODS: Prospective cohort study in secondary care. Patient population included newborns at-risk for DDH - we assessed 13,276 consecutive newborns for the presence of DDH risk factors. Only newborns with at least one of the predefined risk factors and those showing an abnormal examination of the hip were enrolled (n = 2191). For the development of a risk prediction model we considered 9 candidate predictors and other variables readily available at childbirth. The main outcome measure was ultrasonography at a median age of 8 weeks using consensus diagnostic criteria; outcome assessors were blinded. RESULTS: The risk model includes four predictors: female sex (OR = 5.6; 95% CI: 2.9-10.9; P  4000 g (OR = 1.6; 95% CI: 0.6-4.2; P = 0.34), and abnormal examination of hip (OR = 58.8; 95% CI: 31.9, 108.5; P <  0.001). This model demonstrated excellent discrimination (C statistic = 0.9) and calibration of observed and predicted risk (P = 0.35). A model without the variable 'hip examination' demonstrated similar performance. CONCLUSION: The risk model quantifies absolute risk of DDH within 8 weeks postpartum in at-risk newborns. Based on clinical variables readily available at the point of childbirth, the model will enhance parental counselling and could serve as the basis for real time decisions prior to discharge from maternity wards

Variable selection under multiple imputation using the bootstrap in a prognostic study

Background: Missing data is a challenging problem in many prognostic studies. Multiple imputation (MI) accounts for imputation uncertainty that allows for adequate statistical testing. We developed and tested a methodology combining MI with bootstrapping techniques for studying prognostic variable selection. Method: In our prospective cohort study we merged data from three different randomized controlled trials (RCTs) to assess prognostic variables for chronicity of low back pain. Among the outcome and prognostic variables data were missing in the range of 0 and 48.1%. We used four methods to investigate the influence of respectively sampling and imputation variation: MI only, bootstrap only, and two methods that combine MI and bootstrapping. Variables were selected based on the inclusion frequency of each prognostic variable, i.e. the proportion of times that the variable appeared in the model. The discriminative and calibrative abilities of prognostic models developed by the four methods were assessed at different inclusion levels. Results: We found that the effect of imputation variation on the inclusion frequency was larger than the effect of sampling variation. When MI and bootstrapping were combined at the range of 0% (full model) to 90% of variable selection, bootstrap corrected c-index values of 0.70 to 0.71 and slope values of 0.64 to 0.86 were found. Conclusion: We recommend to account for both imputation and sampling variation in sets of missing data. The new procedure of combining MI with bootstrapping for variable selection, results in multivariable prognostic models with good performance and is therefore attractive to apply on data sets with missing values

Using quantile regression to investigate racial disparities in medication non-adherence

<p>Abstract</p> <p>Background</p> <p>Many studies have investigated racial/ethnic disparities in medication non-adherence in patients with type 2 diabetes using common measures such as medication possession ratio (MPR) or gaps between refills. All these measures including MPR are quasi-continuous and bounded and their distribution is usually skewed. Analysis of such measures using traditional regression methods that model mean changes in the dependent variable may fail to provide a full picture about differential patterns in non-adherence between groups.</p> <p>Methods</p> <p>A retrospective cohort of 11,272 veterans with type 2 diabetes was assembled from Veterans Administration datasets from April 1996 to May 2006. The main outcome measure was MPR with quantile cutoffs Q1-Q4 taking values of 0.4, 0.6, 0.8 and 0.9. Quantile-regression (QReg) was used to model the association between MPR and race/ethnicity after adjusting for covariates. Comparison was made with commonly used ordinary-least-squares (OLS) and generalized linear mixed models (GLMM).</p> <p>Results</p> <p>Quantile-regression showed that Non-Hispanic-Black (NHB) had statistically significantly lower MPR compared to Non-Hispanic-White (NHW) holding all other variables constant across all quantiles with estimates and p-values given as -3.4% (p = 0.11), -5.4% (p = 0.01), -3.1% (p = 0.001), and -2.00% (p = 0.001) for Q1 to Q4, respectively. Other racial/ethnic groups had lower adherence than NHW only in the lowest quantile (Q1) of about -6.3% (p = 0.003). In contrast, OLS and GLMM only showed differences in mean MPR between NHB and NHW while the mean MPR difference between other racial groups and NHW was not significant.</p> <p>Conclusion</p> <p>Quantile regression is recommended for analysis of data that are heterogeneous such that the tails and the central location of the conditional distributions vary differently with the covariates. QReg provides a comprehensive view of the relationships between independent and dependent variables (i.e. not just centrally but also in the tails of the conditional distribution of the dependent variable). Indeed, without performing QReg at different quantiles, an investigator would have no way of assessing whether a difference in these relationships might exist.</p