Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers

<p>Abstract</p> <p>Background</p> <p>Decision curve analysis is a novel method for evaluating diagnostic tests, prediction models and molecular markers. It combines the mathematical simplicity of accuracy measures, such as sensitivity and specificity, with the clinical applicability of decision analytic approaches. Most critically, decision curve analysis can be applied directly to a data set, and does not require the sort of external data on costs, benefits and preferences typically required by traditional decision analytic techniques.</p> <p>Methods</p> <p>In this paper we present several extensions to decision curve analysis including correction for overfit, confidence intervals, application to censored data (including competing risk) and calculation of decision curves directly from predicted probabilities. All of these extensions are based on straightforward methods that have previously been described in the literature for application to analogous statistical techniques.</p> <p>Results</p> <p>Simulation studies showed that repeated 10-fold crossvalidation provided the best method for correcting a decision curve for overfit. The method for applying decision curves to censored data had little bias and coverage was excellent; for competing risk, decision curves were appropriately affected by the incidence of the competing risk and the association between the competing risk and the predictor of interest. Calculation of decision curves directly from predicted probabilities led to a smoothing of the decision curve.</p> <p>Conclusion</p> <p>Decision curve analysis can be easily extended to many of the applications common to performance measures for prediction models. Software to implement decision curve analysis is provided.</p

The potential for using risk models in future lung cancer screening trials

Computed tomography screening for early diagnosis of lung cancer is one of the more potentially useful strategies, aside from smoking cessation programmes, for reducing mortality and improving the current poor survival from this disease. The long-term success of lung cancer screening will be dependent upon identifying populations at sufficient risk in order to maximise the benefit-to-harm ratio of the intervention. Risk prediction models could potentially play a major role in the selection of high-risk individuals who would benefit most from screening intervention programmes for the early detection of lung cancer. Improvements of developed lung cancer risk prediction models (through incorporation of objective clinical factors and genetic and molecular biomarkers for precise and accurate estimation of risks), demonstration of their clinical usefulness in decision making, and their use in future screening programmes are the focus of current research

Usefulness of the 2MACE Score to Predicts Adverse Cardiovascular Events in Patients With Atrial Fibrillation

[Abstract] We investigated the incidence of nonembolic adverse events in 2 cohorts of patients with atrial fibrillation (AF) and validated the 2MACE score ([metabolic syndrome, age ≥75] [doubled]; [myocardial infarction or revascularization, congestive heart failure {HF}, and stroke, transient ischemic attack or thromboembolism]) as predictor of major adverse cardiovascular events (MACEs). We recruited 2,630 patients with AF from 2 different cohorts (Murcia AF and FANTASIIA). The 2MACE score was calculated, and during a median of 7.2 years (Murcia AF cohort) and 1.01 years (FANTASIIA) of follow-up, we recorded all nonembolic adverse events and MACEs (composite of nonfatal myocardial infarction or revascularization and cardiovascular death). Receiver operating characteristic curves comparison, reclassification and discriminatory analyses, and decision curve analyses were performed to compare predictive ability and clinical usefulness of the 2MACE score against CHA2DS2-VASc. During follow-up, there were 65 MACEs in the Murcia cohort and 60 in the FANTASIIA cohort. Events rates were higher in the high-risk category (score ≥3) (1.94%/year vs 0.81%/year in the Murcia cohort; 6.01%/year vs 1.71%/year, in FANTASIIA, both p <0.001). The predictive performance of 2MACE according to the receiver operating characteristic curve was significantly higher than that of CHA2DS2-VASc (0.662 vs 0.618, p = 0.008 in the Murcia cohort; 0.656 vs 0.565, p = 0.003 in FANTASIIA). Decision curve analyses demonstrated improved clinical usefulness of the 2MACE compared with the CHA2DS2-VASc score. In conclusion, in “real-world” patients with AF, the 2MACE score is a good predictor of MACEs. A score ≥3 should be used to categorize patients at “high risk,” in identifying patients at risk of MACE.Instituto de Salud Carlos III; PI13/00513Instituto de Salud Carlos III; P14/00253Fundación Séneca; 19245/PI/14Instituto Murciano de Investigación Biosanitaria; IMIB16/AP/01/0

Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests

Many decisions in medicine involve trade-offs, such as between diagnosing patients with disease versus unnecessary additional testing for those who are healthy. Net benefit is an increasingly reported decision analytic measure that puts benefits and harms on the same scale. This is achieved by specifying an exchange rate, a clinical judgment of the relative value of benefits (such as detecting a cancer) and harms (such as unnecessary biopsy) associated with models, markers, and tests. The exchange rate can be derived by asking simple questions, such as the maximum number of patients a doctor would recommend for biopsy to find one cancer. As the answers to these sorts of questions are subjective, it is possible to plot net benefit for a range of reasonable exchange rates in a "decision curve." For clinical prediction models, the exchange rate is related to the probability threshold to determine whether a patient is classified as being positive or negative for a disease. Net benefit is useful for determining whether basing clinical decisions on a model, marker, or test would do more good than harm. This is in contrast to traditional measures such as sensitivity, specificity, or area under the curve, which are statistical abstractions not directly informative about clinical value. Recent years have seen an increase in practical applications of net benefit analysis to research data. This is a welcome development, since decision analytic techniques are of particular value when the purpose of a model, marker, or test is to help doctors make better clinical decisions

Prediction of liver disease in patients whose liver function tests have been checked in primary care : model development and validation using population-based observational cohorts

Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.Peer reviewedPublisher PD

Predicting outcomes after acute reperfusion therapy for basilar artery occlusion

Background and purpose Basilar artery occlusion (BAO) leads to high rates of morbidity and mortality, despite successful recanalization. The discordance between flow restoration and long-term functional status clouds clinical decision-making regarding further aggressive care. We sought to develop and validate a practical, prognostic tool for the prediction of 3-month favorable outcome after acute reperfusion therapy for BAO. Methods This retrospective, multicenter, observational study was conducted at four high-volume stroke centers in the USA and Europe. Multivariate regression analysis was performed to identify predictors of favorable outcome (90-day modified Rankin scale scores 0-2) and derive a clinically applicable prognostic model (the Pittsburgh Outcomes after Stroke Thrombectomy-Vertebrobasilar (POST-VB) score). The POST-VB score was evaluated and internally validated with regard to calibration and discriminatory ability. External validity was assessed in patient cohorts at three separate centers. Results In the derivation cohort of 59 patients, independent predictors of favorable outcome included smaller brainstem infarct volume on post-procedure magnetic resonance imaging (P = 125. Conclusions The POST-VB score effectively predicts 3-month functional outcome following acute reperfusion therapy for BAO and may aid in guiding post-procedural care.Peer reviewe

IL-1R2-based biomarker models predict melioidosis mortality independent of clinical data

Introduction: Melioidosis is an often-fatal tropical infectious disease caused by the Gram-negative bacillus Burkholderia pseudomallei, but few studies have identified promising biomarker candidates to predict outcome. Methods: In 78 prospectively enrolled patients hospitalized with melioidosis, six candidate protein biomarkers, identified from the literature, were measured in plasma at enrollment. A multi-biomarker model was developed using least absolute shrinkage and selection operator (LASSO) regression, and mortality discrimination was compared to a clinical variable model by receiver operating characteristic curve analysis. Mortality prediction was confirmed in an external validation set of 191 prospectively enrolled patients hospitalized with melioidosis. Results: LASSO regression selected IL-1R2 and soluble triggering receptor on myeloid cells 1 (sTREM-1) for inclusion in the candidate biomarker model. The areas under the receiver operating characteristic curve (AUC) for mortality discrimination for the IL-1R2 + sTREM-1 model (AUC 0.81, 95% CI 0.72–0.91) as well as for an IL-1R2-only model (AUC 0.78, 95% CI 0.68–0.88) were higher than for a model based on a modified Sequential Organ Failure Assessment (SOFA) score (AUC 0.69, 95% CI 0.56–0.81, p p = 0.03, respectively). In the external validation set, the IL-1R2 + sTREM-1 model (AUC 0.86, 95% CI 0.81–0.92) had superior 28-day mortality discrimination compared to a modified SOFA model (AUC 0.80, 95% CI 0.74–0.86, p p = 0.33). Conclusion: Biomarker models containing IL-1R2 had improved 28-day mortality prediction compared to clinical variable models in melioidosis and may be targets for future, rapid test development