Towards Explainable and Trustworthy AI for Decision Support in Medicine: An Overview of Methods and Good Practices

Artificial Intelligence (AI) is defined as intelligence exhibited by machines, such as electronic computers. It can involve reasoning, problem solving, learning and knowledge representation, which are mostly in focus in the medical domain. Other forms of intelligence, including autonomous behavior, are also parts of AI. Data driven methods for decision support have been employed in the medical domain for some time. Machine learning (ML) is used for a wide range of complex tasks across many sectors of the industry. However, a broader spectrum of AI, including deep learning (DL) as well as autonomous agents, have been recently gaining more focus and have risen expectation for solving numerous problems in the medical domain. A barrier towards AI adoption, or rather a concern, is trust in AI, which is often hindered by issues like lack of understanding of a black-box model function, or lack of credibility related to reporting of results. Explainability and interpretability are prerequisites for the development of AI-based systems that are lawful, ethical and robust. In this respect, this paper presents an overview of concepts, best practices, and success stories, and opens the discussion for multidisciplinary work towards establishing trustworthy AI

A neural network for prediction of risk of nosocomial infection at intensive care units: a didactic preliminary model

OBJECTIVE: To propose a preliminary artificial intelligence model, based on artificial neural networks, for predicting the risk of nosocomial infection at intensive care units. METHODS: An artificial neural network is designed that employs supervised learning. The generation of the datasets was based on data derived from the Japanese Nosocomial Infection Surveillance system. It is studied how the Java Neural Network Simulator learns to categorize these patients to predict their risk of nosocomial infection. The simulations are performed with several backpropagation learning algorithms and with several groups of parameters, comparing their results through the sum of the squared errors and mean errors per pattern. RESULTS: The backpropagation with momentum algorithm showed better performance than the backpropagation algorithm. The performance improved with the xor. README file parameter values compared to the default parameters. There were no failures in the categorization of the patients into their risk of nosocomial infection. CONCLUSION: While this model is still based on a synthetic dataset, the excellent performance observed with a small number of patterns suggests that using higher numbers of variables and network layers to analyze larger volumes of data can create powerful artificial neural networks, potentially capable of precisely anticipating nosocomial infection at intensive care units. Using a real database during the simulations has the potential to realize the predictive ability of this model

A Multidatabase ExTRaction PipEline (METRE) for Facile Cross Validation in Critical Care Research

Transforming raw EHR data into machine learning model-ready inputs requires considerable effort. One widely used EHR database is Medical Information Mart for Intensive Care (MIMIC). Prior work on MIMIC-III cannot query the updated and improved MIMIC-IV version. Besides, the need to use multicenter datasets further highlights the challenge of EHR data extraction. Therefore, we developed an extraction pipeline that works on both MIMIC-IV and eICU Collaborative Research Database and allows for model cross validation using these 2 databases. Under the default choices, the pipeline extracted 38766 and 126448 ICU records for MIMIC-IV and eICU, respectively. Using the extracted time-dependent variables, we compared the Area Under the Curve (AUC) performance with prior works on clinically relevant tasks such as in-hospital mortality prediction. METRE achieved comparable performance with AUC 0.723- 0.888 across all tasks. Additionally, when we evaluated the model directly on MIMIC-IV data using a model trained on eICU, we observed that the AUC change can be as small as +0.019 or -0.015. Our open-source pipeline transforms MIMIC-IV and eICU into structured data frames and allows researchers to perform model training and testing using data collected from different institutions, which is of critical importance for model deployment under clinical contexts

Predicting Cardiac Arrest and Respiratory Failure Using Feasible Artificial Intelligence with Simple Trajectories of Patient Data

We introduce a Feasible Artificial Intelligence with Simple Trajectories for Predicting Adverse Catastrophic Events (FAST-PACE) solution for preparing immediate intervention in emergency situations. FAST-PACE utilizes a concise set of collected features to construct an artificial intelligence model that predicts the onset of cardiac arrest or acute respiratory failure from 1 h to 6 h prior to its occurrence. Data from the trajectory of 29,181 patients in intensive care units of two hospitals includes periodic vital signs, a history of treatment, current health status, and recent surgery. It excludes the results of laboratory data to construct a feasible application in wards, out-hospital emergency care, emergency transport, or other clinical situations where instant medical decisions are required with restricted patient data. These results are superior to previous warning scores including the Modified Early Warning Score (MEWS) and the National Early Warning Score (NEWS). The primary outcome was the feasibility of an artificial intelligence (AI) model predicting adverse events 1 h to 6 h prior to occurrence without lab data; the area under the receiver operating characteristic curve of this model was 0.886 for cardiac arrest and 0.869 for respiratory failure 6 h before occurrence. The secondary outcome was the superior prediction performance to MEWS (net reclassification improvement of 0.507 for predicting cardiac arrest and 0.341 for predicting respiratory failure) and NEWS (net reclassification improvement of 0.412 for predicting cardiac arrest and 0.215 for predicting respiratory failure) 6 h before occurrence. This study suggests that AI consisting of simple vital signs and a brief interview could predict a cardiac arrest or acute respiratory failure 6 h earlier.ope

A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?

Artificial intelligence (AI) models are increasingly finding applications in the field of medicine. Concerns have been raised about the explainability of the decisions that are made by these AI models. In this article, we give a systematic analysis of explainable artificial intelligence (XAI), with a primary focus on models that are currently being used in the field of healthcare. The literature search is conducted following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) standards for relevant work published from 1 January 2012 to 02 February 2022. The review analyzes the prevailing trends in XAI and lays out the major directions in which research is headed. We investigate the why, how, and when of the uses of these XAI models and their implications. We present a comprehensive examination of XAI methodologies as well as an explanation of how a trustworthy AI can be derived from describing AI models for healthcare fields. The discussion of this work will contribute to the formalization of the XAI field.Comment: 15 pages, 3 figures, accepted for publication in the IEEE Transactions on Artificial Intelligenc