Evaluation of Predictive Reliability to Foster Trust in Artificial Intelligence. A case study in Multiple Sclerosis

Applying Artificial Intelligence (AI) and Machine Learning (ML) in critical contexts, such as medicine, requires the implementation of safety measures to reduce risks of harm in case of prediction errors. Spotting ML failures is of paramount importance when ML predictions are used to drive clinical decisions. ML predictive reliability measures the degree of trust of a ML prediction on a new instance, thus allowing decision-makers to accept or reject it based on its reliability. To assess reliability, we propose a method that implements two principles. First, our approach evaluates whether an instance to be classified is coming from the same distribution of the training set. To do this, we leverage Autoencoders (AEs) ability to reconstruct the training set with low error. An instance is considered Out-of-Distribution (OOD) if the AE reconstructs it with a high error. Second, it is evaluated whether the ML classifier has good performances on samples similar to the newly classified instance by using a proxy model. We show that this approach is able to assess reliability both in a simulated scenario and on a model trained to predict disease progression of Multiple Sclerosis patients. We also developed a Python package, named relAI, to embed reliability measures into ML pipelines. We propose a simple approach that can be used in the deployment phase of any ML model to suggest whether to trust predictions or not. Our method holds the promise to provide effective support to clinicians by spotting potential ML failures during deployment.Comment: 20 pages, 7 figure

Enhanced LDL oxidation in uremic patients: An additional mechanism for accelerated atherosclerosis?

Enhanced LDL oxidation in uremic patients: An additional mechanism for accelerated atherosclerosis? Since oxidized low-density lipoprotein (LDL) is more atherogenic than native LDL, LDL oxidation was investigated in uremic patients who often develop accelerated atherogenesis. Three groups of uremic patients were studied (10 on predialysis conservative therapy, 11 on repetitive hemodialysis, 13 on peritoneal dialysis) and compared with seventy matched controls. LDL oxidation was evaluated in all patients as: (i) the susceptibility to in vitro oxidation (by measuring the resistence to Cu++-induced formation of conjugated dienes), (ii) vitamin E concentration in LDL, and (iii) presence of plasma anti-oxidized LDL antibodies, expressed as the ratio anti-oxLDL/anti-nativeLDL antibodies. The lipid profile was studied in all patients. Vitamin E concentration did not differ between the various groups, although LDL from uremic patients appeared more susceptible to in vitro and in vivo oxidation (as demonstrated by an earlier generation of conjugated dienes and by the presence of an higher antibody ratio) compared to control subjects. Subclass analysis of the different patients revealed that peritoneal dialysis treatment ameliorated the oxidation markers. However, a prolonged dialytic treatment caused a decrease in vitamin E concentration in LDL and increased their susceptibility to oxidation

Different molecular mechanisms causing 9p21 deletions in acute lymphoblastic leukemia of childhood

Deletion of chromosome 9p21 is a crucial event for the development of several cancers including acute lymphoblastic leukemia (ALL). Double strand breaks (DSBs) triggering 9p21 deletions in ALL have been reported to occur at a few defined sites by illegitimate action of the V(D)J recombination activating protein complex. We have cloned 23 breakpoint junctions for a total of 46 breakpoints in 17 childhood ALL (9 B- and 8 T-lineages) showing different size deletions at one or both homologous chromosomes 9 to investigate which particular sequences make the region susceptible to interstitial deletion. We found that half of 9p21 deletion breakpoints were mediated by ectopic V(D)J recombination mechanisms whereas the remaining half were associated to repeated sequences, including some with potential for non-B DNA structure formation. Other mechanisms, such as microhomology-mediated repair, that are common in other cancers, play only a very minor role in ALL. Nucleotide insertions at breakpoint junctions and microinversions flanking the breakpoints have been detected at 20/23 and 2/23 breakpoint junctions, respectively, both in the presence of recombination signal sequence (RSS)-like sequences and of other unspecific sequences. The majority of breakpoints were unique except for two cases, both T-ALL, showing identical deletions. Four of the 46 breakpoints coincide with those reported in other cases, thus confirming the presence of recurrent deletion hotspots. Among the six cases with heterozygous 9p deletions, we found that the remaining CDKN2A and CDKN2B alleles were hypermethylated at CpG islands

Building a normative decision support system for clinical and operational risk management in hemodialysis.

This paper describes the design and implementation of a decision support system for risk management in hemodialysis (HD) departments. The proposed system exploits a domain ontology to formalize the problem as a Bayesian network. It also relies on a software tool, able to automatically collect HD data, to learn the network conditional probabilities. By merging prior knowledge and the available data, the system allows to estimate risk profiles both for patients and HD departments. The risk management process is completed by an influence diagram that enables scenario analysis to choose the optimal decisions that mitigate a patient's risk. The methods and design of the decision support tool are described in detail, and the derived decision model is presented. Examples and case studies are also shown. The tool is one of the few examples of normative system explicitly conceived to manage operational and clinical risks in health care environments

Assessing the quality of care for end stage renal failure patients by means of artificial intelligence methodologies

End Stage Renal Disease is a severe chronic condition that corresponds to the final stage of kidney failure. Hemodialysis (HD) is the most widely used treatment method for ESRD. In order to assess the performance of HD centers, we are developing an auditing system, which resorts to (i) temporal data mining techniques, to discover relationships between the time patterns of the data automatically collected during HD sessions and the performance outcomes, and to (ii) case based reasoning (CBR) to retrieve similar time series within the HD data, in order to evaluate the frequency of critical patterns. The overall approach has demonstrated to be suitable for knowledge discovery and critical patterns similarity assessment on real patients' data, and its use in the context of an auditing system for dialysis management is helping clinicians to improve their understanding of the patients behaviour

Case-Based Retrieval to Support the Treatment of End Stage Renal Failure Patients

Objective: In the present paper, we describe an application of case-based retrieval to the domain of end stage renal failure patients, treated with hemodialysis. Materials and methods: Defining a dialysis session as a case, retrieval of past similar cases has to operate both on static and on dynamic features, since most of the monitoring variables of a dialysis session are time series. Retrieval is then articulated as a two-step procedure: (1) classification, based on static features and (2) intra-class retrieval, in which dynamic features are considered. As regards step (2), we concentrate on a classical dimensionality reduction technique for time series allowing for efficient indexing, namely discrete Fourier transform (DFT). Thanks to specific index structures (i.e. k —d trees), range queries (on local feature similarity) can be efficiently performed on our case base, allowing the physician to examine the most similar stored dialysis sessions with respect to the current one. Results: The retrieval tool has been positively tested on real patients’ data, coming from the nephrology and dialysis unit of the Vigevano hospital, in Italy. Conclusions: The overall system can be seen as a means for supporting quality assessment of the hemodialysis service, providing a useful input from the knowledge management perspective