Kansrijk beslissen

Stel: u, of één van uw naasten, wordt getroffen door een hartinfarct. In dat geval sterft een deel van het hart doordat er te weinig bloed naartoe stroomt. De oorzaak is vaak een stolsel in één van de kransslagaders. U wordt met spoed naar het ziekenhuis gebracht. Laten we aannemen dat dit het Erasmus MC is. De Eerste Hulp bevindt zich - heel toepasselijk- in het hart van dit complex. U wilt uiteraard zo goed mogelijk behandeld worden. Wat betekent ‘zo goed mogelijk’? U wilt díe testen en behandelingen ondergaan die uiteindelijk leiden tot de beste prognose. Prognose kunnen we uitdrukken in de kans om het hartinfarct in relatief goede gezondheid te overleven. Omgekeerd geformuleerd willen we het risico op overlijden door het infarct zo klein mogelijk maken

Towards better clinical prediction models

Clinical prediction models provide risk estimates for the presence of disease (diagnosis) or an event in the future course of disease (prognosis) for individual patients. Although publications that present and evaluate such models are becoming more frequent, the methodology is often suboptimal. We propose that seven steps should be considered in developing prediction models: (i) consideration of the research question and initial data inspection; (ii) coding of predictors; (iii) model specification; (iv) model estimation; (v) evaluation of model performance; (vi) internal validation; and (vii) model presentation. The validity of a prediction model is ideally assessed in fully independent data, where we propose four key measures to evaluate model performance: calibration-in-the-large, or the model intercept (A); calibration slope (B); discrimination, with a concordance statistic (C); and clinical usefulness, with decision-curve analysis (D). As an application, we develop and validate prediction models for 30-day mortality in patients with an acute myocardial infarction. This illustrates the usefulness of the proposed framework to strengthen the methodological rigour and quality for prediction models in cardiovascular research

Monitoring prognosis in severe traumatic brain injury

The choice of disease-specific versus generic scales is common to many fields of medicine. In the area of traumatic brain injury, evidence is coming forward that disease-specific prognostic models and disease-specific scoring systems are preferable in the intensive care setting. In monitoring prognosis, the use of a calibration belt in validation studies potentially provides accurate and intuitively attractive insight into performance. This approach deserves further empirical evaluation of its added value as well as its limitations

Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers

Predicting the probability of the occurrence of a binary outcome or condition is important in biomedical research. While assessing discrimination is an essential issue in developing and validating binary prediction models, less attention has been paid to methods for assessing model calibration. Calibration refers to the degree of agreement between observed and predicted probabilities and is often assessed by testing for lack-of-fit. The objective of our study was to examine the ability of graphical methods to assess the calibration of logistic regression models. We examined lack of internal calibration, which was related to misspecification of the logistic regression model, and external calibration, which was related to an overfit model or to shrinkage of the linear predictor. We conducted an extensive set of Monte Carlo simulations with a locally weighted least squares regression smoother (i.e., the loess algorithm) to examine the ability of graphical methods to assess model calibration. We found that loess-based methods were able to provide evidence of moderate departures from linearity and indicate omission of a moderately strong interaction. Misspecification of the link function was harder to detect. Visual patterns were clearer with higher sample sizes, higher incidence of the outcome, or higher discrimination. Loess-based methods were also able to identify the lack of calibration in external validation samples when an overfit regression model had been used. In conclusion, loess-based smoothing methods are adequate tools to graphically assess calibration and merit wider application

The number of subjects per variable required in linear regression analyses

Objectives To determine the number of independent variables that can be included in a linear regression model. Study Design and Setting We used a series of Monte Carlo simulations to examine the impact of the number of subjects per variable (SPV) on the accuracy of estimated regression coefficients and standard errors, on the empirical coverage of estimated confidence intervals, and on the accuracy of the estimated R2 of the fitted model. Results A minimum of approximately two SPV tended to result in estimation of regression coefficients with relative bias of less than 10%. Furthermore, with this minimum number of SPV, the standard errors of the regression coefficients were accurately estimated and estimated confidence intervals had approximately the advertised coverage rates. A much higher number of SPV were necessary to minimize bias in estimating the model R2, although adjusted R2 estimates behaved well. The bias in estimating the model R2 statistic was inversely proportional to the magnitude of the proportion of variation explained by the population regression model. Conclusion Linear regression models require only two SPV for adequate estimation of regression coefficients, standard errors, and confidence intervals

Validation in prediction research: the waste by data splitting

Accurate prediction of medical outcomes is important for diagnosis and prognosis. The standard requirement in major medical journals is nowadays that validity outside the development sample needs to be shown. Is such data splitting an example of a waste of resources? In large samples, interest should shift to assessment of heterogeneity in model performance across settings. In small samples, cross-validation and bootstrapping are more efficient approaches. In conclusion, random data splitting should be abolished for validation of prediction models

Prognostic Modeling for Clinical Decision Making: theory and applications

Clinical decision making is concerned with choices made for individual patients, with a focus on diagnosis, therapy and prognosis. These choices can be supported by cll1pirical research. Diagnostic research includes the assessment of test characteristics, such as sensitivity and specificity, while therapeutic research is preferably performed in randomized clinical trials. Both diagnostic and therapeutic decisions aim to improve the prognosis for the patient. Prognosis is therefore at the heart of clinical decision making

Feature selection and validated predictive performance in the domain of Legionella pneumophila: a comparative study

textabstractBackground: Genetic comparisons of clinical and environmental Legionella strains form an essential part of outbreak investigations. DNA microarrays often comprise many DNA markers (features). Feature selection and the development of prediction models are particularly challenging in this domain with many variables and comparatively few subjects or data points. We aimed to compare modeling strategies to develop prediction models for classifying infections as clinical or environmental. Methods: We applied a bootstrap strategy for preselecting important features to a database containing 222 Legionella pneumophila strains with 448 continuous markers and a dichotomous outcome (clinical or environmental). Feature selection was done with 50 bootstrap samples resulting in a top 10 of most important features for each of four modeling techniques: classification and regression trees (CART), random forests (RF), support vector machines (SVM) and least absolute shrinkage and selection operator (LASSO). Validation was done in a second bootstrap resampling loop (200x) for evaluation of discriminatory model performance according to the AUC. Results: The top 5 of selected features differed considerably between the various modeling techniques, with only one common feature ("LePn.007B8"). The mean validated AUC-values of the SVM model and the CART model were 0.859 and 0.873 respectively. The LASSO and the RF model showed higher validated AUC-values (0.925 and 0.975 respectively). Conclusions: In the domain of Legionella pneumophila, which comprises many potential features for classifying of infections as clinical or environmental, the RF and LASSO techniques provide good prediction models. The identification of potentially biologically relevant features is highly dependent on the technique used, and should hence be interpreted with caution