Large-scale external validation and comparison of prognostic models: an application to chronic obstructive pulmonary disease

Background: External validations and comparisons of prognostic models or scores are a prerequisite for their use in routine clinical care but are lacking in most medical fields including chronic obstructive pulmonary disease (COPD). Our aim was to externally validate and concurrently compare prognostic scores for 3-year all-cause mortality in mostly multimorbid patients with COPD. Methods: We relied on 24 cohort studies of the COPD Cohorts Collaborative International Assessment consortium, corresponding to primary, secondary, and tertiary care in Europe, the Americas, and Japan. These studies include globally 15,762 patients with COPD (1871 deaths and 42,203 person years of follow-up). We used network meta-analysis adapted to multiple score comparison (MSC), following a frequentist two-stage approach; thus, we were able to compare all scores in a single analytical framework accounting for correlations among scores within cohorts. We assessed transitivity, heterogeneity, and inconsistency and provided a performance ranking of the prognostic scores. Results: Depending on data availability, between two and nine prognostic scores could be calculated for each cohort. The BODE score (body mass index, airflow obstruction, dyspnea, and exercise capacity) had a median area under the curve (AUC) of 0.679 [1st quartile–3rd quartile = 0.655–0.733] across cohorts. The ADO score (age, dyspnea, and airflow obstruction) showed the best performance for predicting mortality (difference AUCADO – AUCBODE = 0.015 [95% confidence interval (CI) = −0.002 to 0.032]; p = 0.08) followed by the updated BODE (AUCBODE updated – AUCBODE = 0.008 [95% CI = −0.005 to +0.022]; p = 0.23). The assumption of transitivity was not violated. Heterogeneity across direct comparisons was small, and we did not identify any local or global inconsistency.La lista completa de autores puede verse en el archivo asociado.Facultad de Ciencias Médica

Large-scale external validation and comparison of prognostic models: an application to chronic obstructive pulmonary disease

Background: External validations and comparisons of prognostic models or scores are a prerequisite for their use in routine clinical care but are lacking in most medical fields including chronic obstructive pulmonary disease (COPD). Our aim was to externally validate and concurrently compare prognostic scores for 3-year all-cause mortality in mostly multimorbid patients with COPD. Methods: We relied on 24 cohort studies of the COPD Cohorts Collaborative International Assessment consortium, corresponding to primary, secondary, and tertiary care in Europe, the Americas, and Japan. These studies include globally 15,762 patients with COPD (1871 deaths and 42,203 person years of follow-up). We used network meta-analysis adapted to multiple score comparison (MSC), following a frequentist two-stage approach; thus, we were able to compare all scores in a single analytical framework accounting for correlations among scores within cohorts. We assessed transitivity, heterogeneity, and inconsistency and provided a performance ranking of the prognostic scores. Results: Depending on data availability, between two and nine prognostic scores could be calculated for each cohort. The BODE score (body mass index, airflow obstruction, dyspnea, and exercise capacity) had a median area under the curve (AUC) of 0.679 [1st quartile-3rd quartile = 0.655-0.733] across cohorts. The ADO score (age, dyspnea, and airflow obstruction) showed the best performance for predicting mortality (difference AUC(ADO) - AUC(BODE) = 0.015 [95% confidence interval (CI) = - 0.002 to 0.032]; p = 0.08) followed by the updated BODE (AUCBODE updated - AUCBODE = 0.008 [95% CI = -0.005 to +0.022]; p = 0.23). The assumption of transitivity was not violated. Heterogeneity across direct comparisons was small, and we did not identify any local or global inconsistency. Conclusions: Our analyses showed best discriminatory performance for the ADO and updated BODE scores in patients with COPD. A limitation to be addressed in future studies is the extension of MSC network meta-analysis to measures of calibration. MSC network meta-analysis can be applied to prognostic scores in any medical field to identify the best scores, possibly paving the way for stratified medicine, public health, and research

Large-scale external validation and comparison of prognostic models: an application to chronic obstructive pulmonary disease

Background: External validations and comparisons of prognostic models or scores are a prerequisite for their use in routine clinical care but are lacking in most medical fields including chronic obstructive pulmonary disease (COPD). Our aim was to externally validate and concurrently compare prognostic scores for 3-year all-cause mortality in mostly multimorbid patients with COPD. Methods: We relied on 24 cohort studies of the COPD Cohorts Collaborative International Assessment consortium, corresponding to primary, secondary, and tertiary care in Europe, the Americas, and Japan. These studies include globally 15,762 patients with COPD (1871 deaths and 42,203 person years of follow-up). We used network meta-analysis adapted to multiple score comparison (MSC), following a frequentist two-stage approach; thus, we were able to compare all scores in a single analytical framework accounting for correlations among scores within cohorts. We assessed transitivity, heterogeneity, and inconsistency and provided a performance ranking of the prognostic scores. Results: Depending on data availability, between two and nine prognostic scores could be calculated for each cohort. The BODE score (body mass index, airflow obstruction, dyspnea, and exercise capacity) had a median area under the curve (AUC) of 0.679 [1st quartile–3rd quartile = 0.655–0.733] across cohorts. The ADO score (age, dyspnea, and airflow obstruction) showed the best performance for predicting mortality (difference AUCADO – AUCBODE = 0.015 [95% confidence interval (CI) = −0.002 to 0.032]; p = 0.08) followed by the updated BODE (AUCBODE updated – AUCBODE = 0.008 [95% CI = −0.005 to +0.022]; p = 0.23). The assumption of transitivity was not violated. Heterogeneity across direct comparisons was small, and we did not identify any local or global inconsistency.La lista completa de autores puede verse en el archivo asociado.Facultad de Ciencias Médica

External Validation and Recalculation of the CODEX Index in COPD Patients. A 3CIAplus Cohort Study

The CODEX index was developed and validated in patients hospitalized for COPD exacerbation to predict the risk of death and readmission within one year after discharge. Our study aimed to validate the CODEX index in a large external population of COPD patients with variable durations of follow-up. Additionally, we aimed to recalculate the thresholds of the CODEX index using the cutoffs of variables previously suggested in the 3CIA study (mCODEX). Individual data on 2,755 patients included in the COPD Cohorts Collaborative International Assessment Plus (3CIA+) were explored. A further two cohorts (ESMI AND EGARPOC-2) were added. To validate the CODEX index, the relationship between mortality and the CODEX index was assessed using cumulative/dynamic ROC curves at different follow-up periods, ranging from 3 months up to 10 years. Calibration was performed using univariate and multivariate Cox proportional hazard models and Hosmer-Lemeshow test. A total of 3,321 (87.8% males) patients were included with a mean ± SD age of 66.9 ± 10.5 years, and a median follow-up of 1,064 days (IQR 25-75% 426-1643), totaling 11,190 person-years. The CODEX index was statistically associated with mortality in the short- (≤3 months), medium- (≤1 year) and long-term (10 years), with an area under the curve of 0.72, 0.70 and 0.76, respectively. The mCODEX index performed better in the medium-term (<1 year) than the original CODEX, and similarly in the long-term. In conclusion, CODEX and mCODEX index are good predictors of mortality in patients with COPD, regardless of disease severity or duration of follow-up

Large-scale external validation and comparison of prognostic models: an application to chronic obstructive pulmonary disease

Background: External validations and comparisons of prognostic models or scores are a prerequisite for their use in routine clinical care but are lacking in most medical fields including chronic obstructive pulmonary disease (COPD). Our aim was to externally validate and concurrently compare prognostic scores for 3-year all-cause mortality in mostly multimorbid patients with COPD. Methods: We relied on 24 cohort studies of the COPD Cohorts Collaborative International Assessment consortium, corresponding to primary, secondary, and tertiary care in Europe, the Americas, and Japan. These studies include globally 15,762 patients with COPD (1871 deaths and 42,203 person years of follow-up). We used network meta-analysis adapted to multiple score comparison (MSC), following a frequentist two-stage approach; thus, we were able to compare all scores in a single analytical framework accounting for correlations among scores within cohorts. We assessed transitivity, heterogeneity, and inconsistency and provided a performance ranking of the prognostic scores. Results: Depending on data availability, between two and nine prognostic scores could be calculated for each cohort. The BODE score (body mass index, airflow obstruction, dyspnea, and exercise capacity) had a median area under the curve (AUC) of 0.679 [1st quartile–3rd quartile = 0.655–0.733] across cohorts. The ADO score (age, dyspnea, and airflow obstruction) showed the best performance for predicting mortality (difference AUCADO – AUCBODE = 0.015 [95% confidence interval (CI) = −0.002 to 0.032]; p = 0.08) followed by the updated BODE (AUCBODE updated – AUCBODE = 0.008 [95% CI = −0.005 to +0.022]; p = 0.23). The assumption of transitivity was not violated. Heterogeneity across direct comparisons was small, and we did not identify any local or global inconsistency. Conclusions: Our analyses showed best discriminatory performance for the ADO and updated BODE scores in patients with COPD. A limitation to be addressed in future studies is the extension of MSC network meta-analysis to measures of calibration. MSC network meta-analysis can be applied to prognostic scores in any medical field to identify the best scores, possibly paving the way for stratified medicine, public health, and research.Other UBCNon UBCReviewedFacult

Large-scale external validation and comparison of prognostic models: an application to chronic obstructive pulmonary disease

Background: External validations and comparisons of prognostic models or scores are a prerequisite for their use in routine clinical care but are lacking in most medical fields including chronic obstructive pulmonary disease (COPD). Our aim was to externally validate and concurrently compare prognostic scores for 3-year all-cause mortality in mostly multimorbid patients with COPD. Methods: We relied on 24 cohort studies of the COPD Cohorts Collaborative International Assessment consortium, corresponding to primary, secondary, and tertiary care in Europe, the Americas, and Japan. These studies include globally 15,762 patients with COPD (1871 deaths and 42,203 person years of follow-up). We used network meta-analysis adapted to multiple score comparison (MSC), following a frequentist two-stage approach; thus, we were able to compare all scores in a single analytical framework accounting for correlations among scores within cohorts. We assessed transitivity, heterogeneity, and inconsistency and provided a performance ranking of the prognostic scores. Results: Depending on data availability, between two and nine prognostic scores could be calculated for each cohort. The BODE score (body mass index, airflow obstruction, dyspnea, and exercise capacity) had a median area under the curve (AUC) of 0.679 [1st quartile–3rd quartile = 0.655–0.733] across cohorts. The ADO score (age, dyspnea, and airflow obstruction) showed the best performance for predicting mortality (difference AUCADO – AUCBODE = 0.015 [95% confidence interval (CI) = −0.002 to 0.032]; p = 0.08) followed by the updated BODE (AUCBODE updated – AUCBODE = 0.008 [95% CI = −0.005 to +0.022]; p = 0.23). The assumption of transitivity was not violated. Heterogeneity across direct comparisons was small, and we did not identify any local or global inconsistency. Conclusions: Our analyses showed best discriminatory performance for the ADO and updated BODE scores in patients with COPD. A limitation to be addressed in future studies is the extension of MSC network meta-analysis to measures of calibration. MSC network meta-analysis can be applied to prognostic scores in any medical field to identify the best scores, possibly paving the way for stratified medicine, public health, and research