How do surgeons' probability estimates of operative mortality compare with a decision analytic model?

Abstract

The aim of this study is to compare surgeons' estimates of operative mortality of patients with an abdominal aneurysm ( = dilation of the aorta) with the operative mortality derived from a decision analytic model and to determine how surgeons use clinical information. Four experienced surgeons are asked to estimate, among other things, the operative mortality of 137 patients. Results concerning the accuracy of surgeons' estimates show that surgeons' average operative mortality estimates are quite accurate as compared to the calculated mortalities. The standard deviations of surgeons' estimates are lower than the standard deviation of the model, however, indicating that the surgeons are not as good in distinguishing the high and low risk patients. Furthermore, surgeons show substantial inconsistencies in the weighing of the clinical information, and also differ from the model in how clinical information is weighed. Finally, when comparing the operative mortalities of the patients who died and those who did not, the model shows a modest, but higher discrimination than the surgeons. Physicians' performance seems to be influenced by the difficulty of the task (i.e. the unpredictability of the event and the multidimensionality of the task). In order to improve physicians' probability estimates, the calculations of the decision model can be used as learning tool