Validation of prediction models based on lasso regression with multiply imputed data

BACKGROUND: In prognostic studies, the lasso technique is attractive since it improves the quality of predictions by shrinking regression coefficients, compared to predictions based on a model fitted via unpenalized maximum likelihood. Since some coefficients are set to zero, parsimony is achieved as well. It is unclear whether the performance of a model fitted using the lasso still shows some optimism. Bootstrap methods have been advocated to quantify optimism and generalize model performance to new subjects. It is unclear how resampling should be performed in the presence of multiply imputed data. METHOD: The data were based on a cohort of Chronic Obstructive Pulmonary Disease patients. We constructed models to predict Chronic Respiratory Questionnaire dyspnea 6 months ahead. Optimism of the lasso model was investigated by comparing 4 approaches of handling multiply imputed data in the bootstrap procedure, using the study data and simulated data sets. In the first 3 approaches, data sets that had been completed via multiple imputation (MI) were resampled, while the fourth approach resampled the incomplete data set and then performed MI. RESULTS: The discriminative model performance of the lasso was optimistic. There was suboptimal calibration due to over-shrinkage. The estimate of optimism was sensitive to the choice of handling imputed data in the bootstrap resampling procedure. Resampling the completed data sets underestimates optimism, especially if, within a bootstrap step, selected individuals differ over the imputed data sets. Incorporating the MI procedure in the validation yields estimates of optimism that are closer to the true value, albeit slightly too larger. CONCLUSION: Performance of prognostic models constructed using the lasso technique can be optimistic as well. Results of the internal validation are sensitive to how bootstrap resampling is performed

Single-arm studies involving patient-reported outcome data in oncology: a literature review on current practice

Patient-reported outcomes (PROs) are increasingly used in single-arm cancer studies. We reviewed 60 recent publications of single-arm studies of cancer treatment involving PRO data for current practice on design, analysis, reporting, and interpretation. We further examined their handling of potential bias and how they informed decision-making. Most studies (97%) analyzed PROs without stating a predefined research hypothesis. Thirteen studies (22%) used a PRO as a (co)primary endpoint. Definitions of PRO objectives, study population, endpoints, and strategies of handling missing data varied widely. Twenty-three studies (38%) compared the PRO data to external information, most often by using a clinically important difference value; one study used a historical control group. Appropriateness of methods to handle missingness and intercurrent events including death were seldom discussed. Most studies (85%) concluded that PRO results supported treatment. Conducting and reporting of PROs in cancer single-arm studies lacks standards, and a critical discussion of statistical methods and possible biases. These findings will guide the Setting International Standards in Analysing Patient-Reported Outcomes and Quality of Life Data in Cancer Clinical Trials-Innovative Medicines Initiative (SISAQOL-IMI) in developing recommendations for the use of PRO-measures in single arm studies

Minimally important differences for interpreting EORTC QLQ-C30 scores in patients with advanced breast cancer

Background We aimed to estimate the minimally important difference (MID) for interpreting group-level change over time, both within a group and between groups, for European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire core 30 (EORTC QLQ-C30) scores in patients with advanced breast cancer. Patients and Methods Data were derived from two published EORTC trials. Clinical anchors, e.g. performance status, were selected using correlation strength and clinical plausibility of their association with a particular QLQ-C30 scale. Three change status groups were formed: deteriorated by one anchor category, improved by one anchor category and no change. Patients with greater anchor changes were excluded. The mean change method was used to estimate MIDs for within-group change and linear regression was used to estimate MIDs for between-group differences in change over time. For a given QLQ-C30 scale, MID estimates from multiple anchors were triangulated to a single value via a correlation-based weighted average. Results MIDs varied by QLQ-C30 scale, direction (improvement versus deterioration) and anchor. MIDs for within-group change ranged from 5 to 14 points (improvement) and –14 to –4 points (deterioration), and MIDs for between-group change over time ranged from 4 to 11 points and from –18 to –4 points. Correlation-weighted MIDs for most QLQ-C30 scales ranged from 4 to 10 points in absolute values. Conclusions Our findings aid interpretation of changes in EORTC QLQ-C30 scores over time, both within and between groups, and for performing more accurate sample size calculations for clinical trials in advanced breast cancer

A joint model for repeated events of different types and multiple longitudinal outcomes with application to a follow-up study of patients after kidney transplant

This paper presents an extension of the joint modeling strategy for the case of multiple longitudinal outcomes and repeated infections of different types over time, motivated by postkidney transplantation data. Our model comprises two parts linked by shared latent terms. On the one hand is a multivariate mixed linear model with random effects, where a low-rank thin-plate spline function is incorporated to collect the nonlinear behavior of the different profiles over time. On the other hand is an infection-specific Cox model, where the dependence between different types of infections and the related times of infection is through a random effect associated with each infection type to catch the within dependence and a shared frailty parameter to capture the dependence between infection types. We implemented the parameterization used in joint models which uses the fitted longitudinal measurements as time-dependent covariates in a relative risk model. Our proposed model was implemented in OpenBUGS using the MCMC approac

Dynamic prediction of mortality among patients in intensive care using the sequential organ failure assessment (SOFA) score: a joint competing risk survival and longitudinal modeling approach

In intensive care units (ICUs), besides routinely collected admission data, a daily monitoring of organ dysfunction using scoring systems such as the sequential organ failure assessment (SOFA) score has become practice. Such updated information is valuable in making accurate predictions of patients' survival. Few prediction models that incorporate this updated information have been reported. We used follow-up data of ICU patients who either died or were discharged at the end of hospital stay, without censored cases. We propose a joint model comprising a linear mixed effects submodel for the development of longitudinal SOFA scores and a proportional subdistribution hazards submodel for death as end point with discharge as competing risk. The two parts are linked by shared latent terms. Because there was no censoring, it was straightforward to fit our joint model using available software. We compared predictive values, based on the Brier score and the area under the receiver operating characteristic curve, from our model with those obtained from an earlier modeling approach by Toma etal. [Journal of Biomedical Informatics 40, 649, (2007)] that relied on patterns discovered in the SOFA scores over a given period of tim

Prediction of COPD-specific health-related quality of life in primary care COPD patients: a prospective cohort study

Background: Health-related quality of life (HRQL) is an important patient-reported outcome for chronic obstructive pulmonary disease (COPD). Aim: We developed models predicting chronic respiratory questionnaire (CRQ) dyspnoea, fatigue, emotional function, mastery and overall HRQL at 6 and 24 months using predictors easily available in primary care. Methods: We used the “least absolute shrinkage and selection operator” (lasso) method to build the models and assessed their predictive performance. Results were displayed using nomograms. Results: For each domain-specific CRQ outcome, the corresponding score at baseline was the best predictor. Depending on the domain, these predictions could be improved by adding one to six other predictors, such as the other domain-specific CRQ scores, health status and depression score. To predict overall HRQL, fatigue and dyspnoea scores were the best predictors. Predicted and observed values were on average the same, indicating good calibration. Explained variance ranged from 0.23 to 0.58, indicating good discrimination. Conclusions: To predict COPD-specific HRQL in primary care COPD patients, previous HRQL was the best predictor in our models. Asking patients explicitly about dyspnoea, fatigue, depression and how they cope with COPD provides additional important information about future HRQL whereas FEV1 or other commonly used predictors add little to the prediction of HRQL

Long-term prediction of HAV antibody dynamics obtained with complete and plasma cell driven kinetics (PCDK) models (95% confidence intervals determined using bootstrap percentile intervals).

<p>Long-term prediction of HAV antibody dynamics obtained with complete and plasma cell driven kinetics (PCDK) models (95% confidence intervals determined using bootstrap percentile intervals).</p