A Mathematical Model for Interpretable Clinical Decision Support with Applications in Gynecology

Over time, methods for the development of clinical decision support (CDS) systems have evolved from interpretable and easy-to-use scoring systems to very complex and non-interpretable mathematical models. In order to accomplish effective decision support, CDS systems should provide information on how the model arrives at a certain decision. To address the issue of incompatibility between performance, interpretability and applicability of CDS systems, this paper proposes an innovative model structure, automatically leading to interpretable and easily applicable models. The resulting models can be used to guide clinicians when deciding upon the appropriate treatment, estimating patient-specific risks and to improve communication with patients.We propose the interval coded scoring (ICS) system, which imposes that the effect of each variable on the estimated risk is constant within consecutive intervals. The number and position of the intervals are automatically obtained by solving an optimization problem, which additionally performs variable selection. The resulting model can be visualised by means of appealing scoring tables and color bars. ICS models can be used within software packages, in smartphone applications, or on paper, which is particularly useful for bedside medicine and home-monitoring. The ICS approach is illustrated on two gynecological problems: diagnosis of malignancy of ovarian tumors using a dataset containing 3,511 patients, and prediction of first trimester viability of pregnancies using a dataset of 1,435 women. Comparison of the performance of the ICS approach with a range of prediction models proposed in the literature illustrates the ability of ICS to combine optimal performance with the interpretability of simple scoring systems.The ICS approach can improve patient-clinician communication and will provide additional insights in the importance and influence of available variables. Future challenges include extensions of the proposed methodology towards automated detection of interaction effects, multi-class decision support systems, prognosis and high-dimensional data

Illustration of the effect of iteratively reweighted

<p><b> regularization.</b> The unweighted model results in the black solid functional form. After iteratively reweighted regularization, the estimated functional form becomes much sparser (see gray dashed line). Small and clinically irrelevant intervals are removed from the functional form.</p

Application of the ICS approach to the prediction of non-viable pregnancies.

<p>(a) Picture-based representation by means of color bars, representing the intervals in which the variable effect is estimated to be constant. For each of the represented bars, the points corresponding to the value of the patient's covariates are obtained. The total score is obtained by summing all points. The color bar at the bottom represents the predicted risk associated with the final score. (b) Estimated link function, linking the score with the risk of a non-viable pregnancy at the end of the first trimester. (c) Calibration of the ICS model on the test set. For each possible value of the predicted risk (some values were taken together in order to obtain at least 10% of the patients in each group), the observed percentage of non-viable pregnancies is calculated (dots). A 95% confidence interval on the percentage of the observed non-viable pregnancies is illustrated by means of the vertical lines. Fhr: fetal heart rate.</p

Link between the scores obtained from Table 3 and the estimate of the risk.

<p>Link between the scores obtained from <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0034312#pone-0034312-t003" target="_blank">Table 3</a> and the estimate of the risk.</p

Summary of the performance of previously built models and the ICS model for the prediction of malignancy of adnexal masses.

<p>The measures are calculated on the test set of 997 patients from external centers. Except for the AUC, all measures were calculated using the cut-off mentioned in the original paper.</p

Advantages and disadvantages of different classification methods in clinical decision making.

1<p>Logistic regression.</p>2<p>(Least-squares) Support Vector Machine.</p>3<p>Artificial Neural Network.</p>4<p>Post-processing in order to obtain interpretable and easily applicable models.</p

Summary of the test set performance of the ICS-based score system and two classical score systems (M1 and M2) for the prediction of non-viability of pregnancies.

<p>The classical score systems are based on a logistic regression model using the variables selected with ICS (LR). In a second step, the variables are manually divided into intervals. M1 uses a high number of intervals for continuous variables, M2 uses fewer intervals. The ICS approach is able to obtain good performance using a small number of intervals. The classical score systems are able to obtain good performance provided that a large number of intervals is considered.</p

Summary of the test set performance of the ICS-based score system and two classical score systems (M1 and M2) for the prediction of malignancy of adnexal masses.

<p>The classical score systems are based on a logistic regression model using the variables selected with ICS (LR). In a second step, the variables are manually divided into intervals. M1 uses a high number of intervals for continuous variables, M2 uses fewer intervals. The ICS approach is able to obtain good performance using a small number of intervals. The classical score systems are able to obtain good performance provided that a large number of intervals is considered.</p

Application of the ICS approach to the diagnosis of the malignancy of adnexal masses.

<p>(a) Picture-based representation by means of bar charts (without color indications) representing the intervals in which the variable effect is estimated to be constant. The bottom bar represents the predicted risk associated with the final score, obtained by summing all contributions of all variables. A software implementation is provided as <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0034312#pone.0034312.s003" target="_blank">Movie S2</a> and <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0034312#pone.0034312.s004" target="_blank">Movie S3</a>. (b) Estimated link function, linking the score with the risk of a malignant tumor. (c) Calibration of the ICS model on the test set. For each possible value of the predicted risk (some values were taken together in order to obtain at least 10% of the patients in each group), the observed percentage of malignancies is calculated (dots). A 95% confidence interval on the percentage of the observed malignancies is illustrated by means of the vertical lines.</p