An incremental explanation of inference in Bayesian networks for increasing model trustworthiness and supporting clinical decision making

Various AI models are increasingly being considered as part of clinical decision-support tools. However, the trustworthiness of such models is rarely considered. Clinicians are more likely to use a model if they can understand and trust its predictions. Key to this is if its underlying reasoning can be explained. A Bayesian network (BN) model has the advantage that it is not a black-box and its reasoning can be explained. In this paper, we propose an incremental explanation of inference that can be applied to ‘hybrid’ BNs, i.e. those that contain both discrete and continuous nodes. The key questions that we answer are: (1) which important evidence supports or contradicts the prediction, and (2) through which intermediate variables does the information flow. The explanation is illustrated using a real clinical case study. A small evaluation study is also conducted

Bayesian Networks for Clinical Decision Making : Support, Assurance, Trust

PhD thesisBayesian networks have been widely proposed to assist clinical decision making. Their popularity is due to their ability to combine different sources of information and reason under uncertainty, using sound probabilistic laws. Despite their benefit, there is still a gap between developing a Bayesian network that has a good predictive accuracy and having a model that makes a significant difference to clinical decision making. This thesis tries to bridge that gap and proposes three novel contributions. The first contribution is a modelling approach that captures the progress of an acute condition and the dynamic way that clinicians gather information and take decisions in irregular stages of care. The proposed method shows how to design a model to generate predictions with the potential to support decision making in successive stages of care. The second contribution is to show how counterfactual reasoning with a Bayesian network can be used as a healthcare governance tool to estimate the effect of treatment decisions other than those occurred. In addition, we extend counterfactual reasoning in situations where the targeted decision and its effect belong to different stages of the patient’s care. The third contribution is an explanation of the Bayesian network’s reasoning. No model is going to be used if it is unclear how it reasons. Presenting an explanation, alongside a prediction, has the potential to increase the acceptability of the network. The proposed technique indicates which important evidence supports or contradicts the prediction and through which intermediate variables the information flows. The above contributions are explored using two clinical case studies. A clinical case study on combat trauma care is used to investigate the first two contributions. The third contribution is explored using a Bayesian network developed by others to provide decision support in treating acute traumatic coagulopathy in the emergency department. Both case studies are done in collaboration with the Royal London Hospital and the Royal Centre for Defence Medicine

Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1A2C1011198) , (Institute for Information & communications Technology Planning & Evaluation) (IITP) grant funded by the Korea government (MSIT) under the ICT Creative Consilience Program (IITP-2021-2020-0-01821) , and AI Platform to Fully Adapt and Reflect Privacy-Policy Changes (No. 2022-0-00688).Artificial intelligence (AI) is currently being utilized in a wide range of sophisticated applications, but the outcomes of many AI models are challenging to comprehend and trust due to their black-box nature. Usually, it is essential to understand the reasoning behind an AI mode ľs decision-making. Thus, the need for eXplainable AI (XAI) methods for improving trust in AI models has arisen. XAI has become a popular research subject within the AI field in recent years. Existing survey papers have tackled the concepts of XAI, its general terms, and post-hoc explainability methods but there have not been any reviews that have looked at the assessment methods, available tools, XAI datasets, and other related aspects. Therefore, in this comprehensive study, we provide readers with an overview of the current research and trends in this rapidly emerging area with a case study example. The study starts by explaining the background of XAI, common definitions, and summarizing recently proposed techniques in XAI for supervised machine learning. The review divides XAI techniques into four axes using a hierarchical categorization system: (i) data explainability, (ii) model explainability, (iii) post-hoc explainability, and (iv) assessment of explanations. We also introduce available evaluation metrics as well as open-source packages and datasets with future research directions. Then, the significance of explainability in terms of legal demands, user viewpoints, and application orientation is outlined, termed as XAI concerns. This paper advocates for tailoring explanation content to specific user types. An examination of XAI techniques and evaluation was conducted by looking at 410 critical articles, published between January 2016 and October 2022, in reputed journals and using a wide range of research databases as a source of information. The article is aimed at XAI researchers who are interested in making their AI models more trustworthy, as well as towards researchers from other disciplines who are looking for effective XAI methods to complete tasks with confidence while communicating meaning from data.National Research Foundation of Korea Ministry of Science, ICT & Future Planning, Republic of Korea Ministry of Science & ICT (MSIT), Republic of Korea 2021R1A2C1011198Institute for Information amp; communications Technology Planning amp; Evaluation) (IITP) - Korea government (MSIT) under the ICT Creative Consilience Program IITP-2021-2020-0-01821AI Platform to Fully Adapt and Reflect Privacy-Policy Changes2022-0-0068

Mapping of machine learning approaches for description, prediction, and causal inference in the social and health sciences

Machine learning (ML) methodology used in the social and health sciences needs to fit the intended research purposes of description, prediction, or causal inference. This paper provides a comprehensive, systematic meta-mapping of research questions in the social and health sciences to appropriate ML approaches by incorporating the necessary requirements to statistical analysis in these disciplines. We map the established classification into description, prediction, counterfactual prediction, and causal structural learning to common research goals, such as estimating prevalence of adverse social or health outcomes, predicting the risk of an event, and identifying risk factors or causes of adverse outcomes, and explain common ML performance metrics. Such mapping may help to fully exploit the benefits of ML while considering domain-specific aspects relevant to the social and health sciences and hopefully contribute to the acceleration of the uptake of ML applications to advance both basic and applied social and health sciences research

On Experimentation in Software-Intensive Systems

Context: Delivering software that has value to customers is a primary concern of every software company. Prevalent in web-facing companies, controlled experiments are used to validate and deliver value in incremental deployments. At the same that web-facing companies are aiming to automate and reduce the cost of each experiment iteration, embedded systems companies are starting to adopt experimentation practices and leverage their activities on the automation developments made in the online domain. Objective: This thesis has two main objectives. The first objective is to analyze how software companies can run and optimize their systems through automated experiments. This objective is investigated from the perspectives of the software architecture, the algorithms for the experiment execution and the experimentation process. The second objective is to analyze how non web-facing companies can adopt experimentation as part of their development process to validate and deliver value to their customers continuously. This objective is investigated from the perspectives of the software development process and focuses on the experimentation aspects that are distinct from web-facing companies. Method: To achieve these objectives, we conducted research in close collaboration with industry and used a combination of different empirical research methods: case studies, literature reviews, simulations, and empirical evaluations. Results: This thesis provides six main results. First, it proposes an architecture framework for automated experimentation that can be used with different types of experimental designs in both embedded systems and web-facing systems. Second, it proposes a new experimentation process to capture the details of a trustworthy experimentation process that can be used as the basis for an automated experimentation process. Third, it identifies the restrictions and pitfalls of different multi-armed bandit algorithms for automating experiments in industry. This thesis also proposes a set of guidelines to help practitioners select a technique that minimizes the occurrence of these pitfalls. Fourth, it proposes statistical models to analyze optimization algorithms that can be used in automated experimentation. Fifth, it identifies the key challenges faced by embedded systems companies when adopting controlled experimentation, and we propose a set of strategies to address these challenges. Sixth, it identifies experimentation techniques and proposes a new continuous experimentation model for mission-critical and business-to-business. Conclusion: The results presented in this thesis indicate that the trustworthiness in the experimentation process and the selection of algorithms still need to be addressed before automated experimentation can be used at scale in industry. The embedded systems industry faces challenges in adopting experimentation as part of its development process. In part, this is due to the low number of users and devices that can be used in experiments and the diversity of the required experimental designs for each new situation. This limitation increases both the complexity of the experimentation process and the number of techniques used to address this constraint

Decision-Support for Rheumatoid Arthritis Using Bayesian Networks: Diagnosis, Management, and Personalised Care.

PhD Theses.Bayesian networks (BNs) have been widely proposed for medical decision support. One advantage of a BN is reasoning under uncertainty, which is pervasive in medicine. Another advantage is that a BN can be built from both data and knowledge and so can be applied in circumstances where a complete dataset is not available. In this thesis, we examine how BNs can be used for the decision support challenges of chronic diseases. As a case study, we study Rheumatoid Arthritis (RA), which is a chronic inflammatory disease causing swollen and painful joints. The work has been done as part of a collaborative project including clinicians from Barts and the London NHS Trust involved in the treatment of RA. The work covers three stages of decision support, with progressively less available data. The first decision support stage is diagnosis. Various criteria have been proposed by clinicians for early diagnosis but these criteria are deterministic and so do not capture diagnostic uncertainty, which is a concern for patients with mild symptoms in the early stages of the disease. We address this problem by building a BN model for diagnosing RA. The diagnostic BN model is built using both a dataset of 360 patients provided by the clinicians and their knowledge as experts in this domain. The choice of factors to include in the diagnostic model is informed by knowledge, including a model of the care pathway which shows what information is available for diagnosis. Knowledge is used to classify the factors as risk factors, relevant comorbidities, evidence of pathogenesis mechanism, signs, symptoms, and serology results, so that the structure of BN model matches the clinical understanding of RA. Since most of the factors are present in the dataset, we are able to train the parameters of the diagnostic BN from the data. This diagnostic BN model obtains promising results in differentiating RA cases from other inflammatory arthritis cases. Aware that eliciting knowledge is time-consuming and could limit the uptake of these techniques, we consider two alternative approaches. First, we compare its diagnostic performance with an alternative BN model entirely learnt from data; we argue that having a clinically meaningful structure allows us to explain clinical scenarios in a way that cannot be done with the model learnt purely from data. We also examine whether useful knowledge can be retrieved from existing vi medical ontologies, such as SNOMED CT and UMLS. Preliminary results show that it could be feasible to use such sources to partially automate knowledge collection. After patients have been diagnosed with RA, they are monitored regularly by a clinical team until the activity of their disease becomes low. The typical care arrangement has two challenges: first, regular meetings with clinicians occur infrequently at fixed intervals (e.g., every six months), during which time the activity of the disease can increase (or ‘flare’) and decrease several times. Secondly, the best medications or combinations of medications must be found for each patient, but changes can only be made when the patient visits the clinic. We therefore develop this stage of decision support in two parts: the first and simplest part looks at how the frequency of clinic appointments could be varied; the second part builds on this to support decisions to adjust medication dosage. We describe this as the ‘self-management’ decision support model. Disease activity is commonly measured with Disease Activity Score 28 (DAS28). Since the joint count parts of this can be assessed by the patient, the possibility of collecting regular (e.g., weekly) DAS28 data has been proposed. It is not yet in wide use, perhaps because of the overheads to the clinical team of reviewing data regularly. The dataset available to us for this work came from a feasibility study conducted by the clinical collaborators of one system for collecting data from patients, although the frequency is only quarterly. The aim of the ‘self-management’ decision support system is therefore to sit between patient-entered data and the clinical team, saving the work of clinically assessing all the data. Specifically, in the first part we wish to predict disease activity so that an appointment should be made sooner, distinguishing this from patients whose disease is well-managed so that the interval between appointments can be increased. To achieve this, we build a dynamic BN (DBN) model to monitor disease activity and to indicate to patients and their clinicians whether a clinical review is needed. We use the data and a set of dummy patient scenarios designed by the experts to evaluate the performance of the DBN. The second part of the ‘self-management’ decision support stage extends the DBN to give advice on adjustments to the medication dosage. This is of particular clinical interest since one class of medications used (biological disease-modifying antirheumatic drugs) are very expensive and, although effective at reducing disease activity, can have severe adverse reactions. For both these reasons, decision support that allowed a patient to ‘taper’ the dosage of medications without frequent clinic visits would be very useful. This extension does not meet all the decision support needs, which ideally would also cover decision-making about the choice of medications. However, we have found that as yet there is neither sufficient data nor knowledge for this. vii The third and final stage of decision support is targeted at patients who live with RA. RA can have profound impacts on the quality of life (QoL) of those who live with it, affecting work, financial status, friendships, and relationships. Information from patient organisations such as the leaflets prepared by the National Rheumatoid Arthritis Society (NRAS) contains advice on managing QoL, but the advice is generic, leaving it up to each patient to select the advice most relevant to their specific circumstances. Our aim is therefore to build a BN-based decision support system to personalise the recommendations for enhancing the QoL of RA patients. We have built a BN to infer three components of QoL (independence, participation, and empowerment) and shown how this can be used to target advice. Since there is no data, the BN is developed from expert knowledge and literature. To evaluate the resulting system, including the BN, we use a set of patient interviews conducted and coded by our collaborators. The recommendations of the system were compared with those of experts in a set of test scenarios created from the interviews; the comparison shows promising results

On the Design, Implementation and Application of Novel Multi-disciplinary Techniques for explaining Artificial Intelligence Models

284 p.Artificial Intelligence is a non-stopping field of research that has experienced some incredible growth lastdecades. Some of the reasons for this apparently exponential growth are the improvements incomputational power, sensing capabilities and data storage which results in a huge increment on dataavailability. However, this growth has been mostly led by a performance-based mindset that has pushedmodels towards a black-box nature. The performance prowess of these methods along with the risingdemand for their implementation has triggered the birth of a new research field. Explainable ArtificialIntelligence. As any new field, XAI falls short in cohesiveness. Added the consequences of dealing withconcepts that are not from natural sciences (explanations) the tumultuous scene is palpable. This thesiscontributes to the field from two different perspectives. A theoretical one and a practical one. The formeris based on a profound literature review that resulted in two main contributions: 1) the proposition of anew definition for Explainable Artificial Intelligence and 2) the creation of a new taxonomy for the field.The latter is composed of two XAI frameworks that accommodate in some of the raging gaps found field,namely: 1) XAI framework for Echo State Networks and 2) XAI framework for the generation ofcounterfactual. The first accounts for the gap concerning Randomized neural networks since they havenever been considered within the field of XAI. Unfortunately, choosing the right parameters to initializethese reservoirs falls a bit on the side of luck and past experience of the scientist and less on that of soundreasoning. The current approach for assessing whether a reservoir is suited for a particular task is toobserve if it yields accurate results, either by handcrafting the values of the reservoir parameters or byautomating their configuration via an external optimizer. All in all, this poses tough questions to addresswhen developing an ESN for a certain application, since knowing whether the created structure is optimalfor the problem at hand is not possible without actually training it. However, some of the main concernsfor not pursuing their application is related to the mistrust generated by their black-box" nature. Thesecond presents a new paradigm to treat counterfactual generation. Among the alternatives to reach auniversal understanding of model explanations, counterfactual examples is arguably the one that bestconforms to human understanding principles when faced with unknown phenomena. Indeed, discerningwhat would happen should the initial conditions differ in a plausible fashion is a mechanism oftenadopted by human when attempting at understanding any unknown. The search for counterfactualsproposed in this thesis is governed by three different objectives. Opposed to the classical approach inwhich counterfactuals are just generated following a minimum distance approach of some type, thisframework allows for an in-depth analysis of a target model by means of counterfactuals responding to:Adversarial Power, Plausibility and Change Intensity