When and How to Fool Explainable Models (and Humans) with Adversarial Examples

Reliable deployment of machine learning models such as neural networks continues to be challenging due to several limitations. Some of the main shortcomings are the lack of interpretability and the lack of robustness against adversarial examples or out-of-distribution inputs. In this paper, we explore the possibilities and limits of adversarial attacks for explainable machine learning models. First, we extend the notion of adversarial examples to fit in explainable machine learning scenarios, in which the inputs, the output classifications and the explanations of the model's decisions are assessed by humans. Next, we propose a comprehensive framework to study whether (and how) adversarial examples can be generated for explainable models under human assessment, introducing novel attack paradigms. In particular, our framework considers a wide range of relevant (yet often ignored) factors such as the type of problem, the user expertise or the objective of the explanations in order to identify the attack strategies that should be adopted in each scenario to successfully deceive the model (and the human). These contributions intend to serve as a basis for a more rigorous and realistic study of adversarial examples in the field of explainable machine learning.Comment: 12 pages, 1 figur

Promoting Learning Through Explainable Artificial Intelligence: An Experimental Study in Radiology

The deployment of machine learning (ML)-based decision support systems (DSSs) in high-risk environments such as radiology is increasing. Despite having achieved high decision accuracy, they are prone to errors. Thus, they are primarily used to assist radiologists in their decision making. However, collaborative decision making poses risks to the decision maker, e.g. automation bias and long-term performance degradation. To address these issues, we propose combining findings of the research streams of explainable artificial intelligence and education to promote human learning through interaction with ML-based DSSs. We provided radiologists with explainable vs non-explainable decision support that was high- vs low-performing in a between-subject experimental study to support manual segmentation of 690 brain tumor scans. Our results show that explainable ML-based DSSs improved human learning outcomes and prevented false learning triggered by incorrect decision support. In fact, radiologists were able to learn from errors made by the low-performing explainable ML-based DSS

BETTER MODELS FOR HIGH-STAKES TASKS

The intersection of machine learning and healthcare has the potential to transform medical diagnosis, treatment, and research. Machine learning models can analyze vast amounts of medical data and identify patterns that may be too complex for human analysis. However, one of the major challenges in this field is building trust between users and the model. Due to things like high false alarm rate and the black box nature of machine learning models, patients and medical professionals need to understand how the model arrives at its recommendations. In this work, we present several methods that aim to improve machine learning models in high-stakes environments like healthcare. Our work unifies two sub-fields of machine learning, explainable AI, and uncertainty quantification. First we develop a model-agnostic approach to deliver instance-level explanations using influence functions. Next, we show that these influence functions function are fairly robust across domains. Then, we develop an efficient method that reduces model uncertainty while modeling data uncertainty via Bayesian Neural Networks. Finally, we show that when combined our methods deliver significant utility beyond traditional methods while retaining a high level of performance via a real world deployment. Overall, the integration of uncertainty quantification and explainable AI can help overcome some of the major challenges of machine learning in healthcare. Together, they can provide healthcare professionals with powerful tools for improving patient outcomes and advancing medical research

The Blind Oracle, eXplainable Artififical Intelligence (XAI) and human agency

An explainable machine learning model is a requirement for trust. Without it the human operator cannot form a correct mental model and will distrust and reject the machine learning model. Nobody will ever trust a system which exhibit an apparent erratic behaviour. The development of eXplainable AI (XAI) techniques try to uncover how a model works internally and the reasons why they make some predictions and not others. But the ultimate objective is to use these techniques to guide the training and deployment of fair automated decision systems that support human agency and are beneficial to humanity. In addition, automated decision systems based on Machine Learning models are being used for an increasingly number of purposes. However, the use of black-box models and massive quantities of data to train them make the deployed models inscrutable. Consequently, predictions made by systems integrating these models might provoke rejection by their users when they made seemingly arbitrary predictions. Moreover, the risk is compounded by the use of models in high-risk environments or in situations when the predictions might have serious consequences.Departamento de Informática (Arquitectura y Tecnología de Computadores, Ciencias de la Computación e Inteligencia Artificial, Lenguajes y Sistemas Informáticos)Máster en Ingeniería Informátic

Usage of Network Simulators in Machine-Learning-Assisted 5G/6G Networks

Without any doubt, Machine Learning (ML) will be an important driver of future communications due to its foreseen performance when applied to complex problems. However, the application of ML to networking systems raises concerns among network operators and other stakeholders, especially regarding trustworthiness and reliability. In this paper, we devise the role of network simulators for bridging the gap between ML and communications systems. In particular, we present an architectural integration of simulators in ML-aware networks for training, testing, and validating ML models before being applied to the operative network. Moreover, we provide insights on the main challenges resulting from this integration, and then give hints discussing how they can be overcome. Finally, we illustrate the integration of network simulators into ML-assisted communications through a proof-of-concept testbed implementation of a residential Wi-Fi network