A Process for Evaluating Explanations for Transparent and Trustworthy AI Prediction Models

This study proposes a process to generate and validate algorithmic explanations for the reasoning of an AI prediction model, implemented using a Bayesian network (BN). The intention of the generated explanations is to increase the transparency and trustworthiness of a decision-support system that uses a BN prediction model. To achieve this, explanations should be presented in an easy-to-understand, clear, and concise natural language narrative. We have developed an algorithm for explaining the reasoning of a prediction made using a BN. For the narrative part of the explanation, we use a template which presents the 'content' part of the explanation; this content is a word-less information structure that applies to all BNs. The template, on the other hand, needs to be designed specifically for each BN model. In this paper, we use a BN for the risk of trauma-induced coagulopathy, a critical bleeding problem. We outline a process for using experts' explanations as the basis for designing the explanation template. We do not believe that an algorithmic explanation needs to be indistinguishable from expert explanations; instead we aim to imitate the narrative structure of explanations given by experts, although we find that there is considerable variation in these. We then consider how the generated explanations can be evaluated, since a direct comparison (in the style of a Turing test) would likely fail. We describe a study using questionnaires and interviews to evaluate the effect of an algorithmic explanation on the transparency and also on the trustworthiness of the predictions made by the system. The preliminary results of our study suggest that the presence of an explanation makes the AI model more transparent but not necessarily more trustworthy

Updating and recalibrating causal probabilistic models on a new target population.

OBJECTIVE: Very often the performance of a Bayesian Network (BN) is affected when applied to a new target population. This is mainly because of differences in population characteristics. External validation of the model performance on different populations is a standard approach to test model's generalisability. However, a good predictive performance is not enough to show that the model represents the unique population characteristics and can be adopted in the new environment. METHODS: In this paper, we present a methodology for updating and recalibrating developed BN models - both their structure and parameters - to better account for the characteristics of the target population. Attention has been given on incorporating expert knowledge and recalibrating latent variables, which are usually omitted from data-driven models. RESULTS: The method is successfully applied to a clinical case study about the prediction of trauma-induced coagulopathy, where a BN has already been developed for civilian trauma patients and now it is recalibrated on combat casualties. CONCLUSION: The methodology proposed in this study is important for developing credible models that can demonstrate a good predictive performance when applied to a target population. Another advantage of the proposed methodology is that it is not limited to data-driven techniques and shows how expert knowledge can also be used when updating and recalibrating the model

Diagnostic accuracy of clinical examination to identify life- and limb-threatening injuries in trauma patients

Abstract Background Timely and accurate identification of life- and limb-threatening injuries (LLTIs) is a fundamental objective of trauma care that directly informs triage and treatment decisions. However, the diagnostic accuracy of clinical examination to detect LLTIs is largely unknown, due to the risk of contamination from in-hospital diagnostics in existing studies. Our aim was to assess the diagnostic accuracy of initial clinical examination for detecting life- and limb-threatening injuries (LLTIs). Secondary aims were to identify factors associated with missed injury and overdiagnosis, and determine the impact of clinician uncertainty on diagnostic accuracy. Methods Retrospective diagnostic accuracy study of consecutive adult (≥ 16 years) patients examined at the scene of injury by experienced trauma clinicians, and admitted to a Major Trauma Center between 01/01/2019 and 31/12/2020. Diagnoses of LLTIs made on contemporaneous clinical records were compared to hospital coded diagnoses. Diagnostic performance measures were calculated overall, and based on clinician uncertainty. Multivariate logistic regression analyses identified factors affecting missed injury and overdiagnosis. Results Among 947 trauma patients, 821 were male (86.7%), median age was 31 years (range 16–89), 569 suffered blunt mechanisms (60.1%), and 522 (55.1%) sustained LLTIs. Overall, clinical examination had a moderate ability to detect LLTIs, which varied by body region: head (sensitivity 69.7%, positive predictive value (PPV) 59.1%), chest (sensitivity 58.7%, PPV 53.3%), abdomen (sensitivity 51.9%, PPV 30.7%), pelvis (sensitivity 23.5%, PPV 50.0%), and long bone fracture (sensitivity 69.9%, PPV 74.3%). Clinical examination poorly detected life-threatening thoracic (sensitivity 48.1%, PPV 13.0%) and abdominal (sensitivity 43.6%, PPV 20.0%) bleeding. Missed injury was more common in patients with polytrauma (OR 1.83, 95% CI 1.62–2.07) or shock (systolic blood pressure OR 0.993, 95% CI 0.988–0.998). Overdiagnosis was more common in shock (OR 0.991, 95% CI 0.986–0.995) or when clinicians were uncertain (OR 6.42, 95% CI 4.63–8.99). Uncertainty improved sensitivity but reduced PPV, impeding diagnostic precision. Conclusions Clinical examination performed by experienced trauma clinicians has only a moderate ability to detect LLTIs. Clinicians must appreciate the limitations of clinical examination, and the impact of uncertainty, when making clinical decisions in trauma. This study provides impetus for diagnostic adjuncts and decision support systems in trauma