19 research outputs found
bLIMEy:Surrogate Prediction Explanations Beyond LIME
Surrogate explainers of black-box machine learning predictions are of
paramount importance in the field of eXplainable Artificial Intelligence since
they can be applied to any type of data (images, text and tabular), are
model-agnostic and are post-hoc (i.e., can be retrofitted). The Local
Interpretable Model-agnostic Explanations (LIME) algorithm is often mistakenly
unified with a more general framework of surrogate explainers, which may lead
to a belief that it is the solution to surrogate explainability. In this paper
we empower the community to "build LIME yourself" (bLIMEy) by proposing a
principled algorithmic framework for building custom local surrogate explainers
of black-box model predictions, including LIME itself. To this end, we
demonstrate how to decompose the surrogate explainers family into
algorithmically independent and interoperable modules and discuss the influence
of these component choices on the functional capabilities of the resulting
explainer, using the example of LIME.Comment: 2019 Workshop on Human-Centric Machine Learning (HCML 2019); 33rd
Conference on Neural Information Processing Systems (NeurIPS 2019),
Vancouver, Canad
LimeOut: An Ensemble Approach To Improve Process Fairness
Artificial Intelligence and Machine Learning are becoming increasingly
present in several aspects of human life, especially, those dealing with
decision making. Many of these algorithmic decisions are taken without human
supervision and through decision making processes that are not transparent.
This raises concerns regarding the potential bias of these processes towards
certain groups of society, which may entail unfair results and, possibly,
violations of human rights. Dealing with such biased models is one of the major
concerns to maintain the public trust.
In this paper, we address the question of process or procedural fairness.
More precisely, we consider the problem of making classifiers fairer by
reducing their dependence on sensitive features while increasing (or, at least,
maintaining) their accuracy. To achieve both, we draw inspiration from
"dropout" techniques in neural based approaches, and propose a framework that
relies on "feature drop-out" to tackle process fairness. We make use of "LIME
Explanations" to assess a classifier's fairness and to determine the sensitive
features to remove. This produces a pool of classifiers (through feature
dropout) whose ensemble is shown empirically to be less dependent on sensitive
features, and with improved or no impact on accuracy.Comment: 11 pages, 3 figures, 3 table
Uncertainty in Additive Feature Attribution methods
In this work, we explore various topics that fall under the umbrella of
Uncertainty in post-hoc Explainable AI (XAI) methods. We in particular focus on
the class of additive feature attribution explanation methods. We first
describe our specifications of uncertainty and compare various statistical and
recent methods to quantify the same. Next, for a particular instance, we study
the relationship between a feature's attribution and its uncertainty and
observe little correlation. As a result, we propose a modification in the
distribution from which perturbations are sampled in LIME-based algorithms such
that the important features have minimal uncertainty without an increase in
computational cost. Next, while studying how the uncertainty in explanations
varies across the feature space of a classifier, we observe that a fraction of
instances show near-zero uncertainty. We coin the term "stable instances" for
such instances and diagnose factors that make an instance stable. Next, we
study how an XAI algorithm's uncertainty varies with the size and complexity of
the underlying model. We observe that the more complex the model, the more
inherent uncertainty is exhibited by it. As a result, we propose a measure to
quantify the relative complexity of a blackbox classifier. This could be
incorporated, for example, in LIME-based algorithms' sampling densities, to
help different explanation algorithms achieve tighter confidence levels.
Together, the above measures would have a strong impact on making XAI models
relatively trustworthy for the end-user as well as aiding scientific discovery.Comment: 1
Sum-of-Parts Models: Faithful Attributions for Groups of Features
An explanation of a machine learning model is considered "faithful" if it
accurately reflects the model's decision-making process. However, explanations
such as feature attributions for deep learning are not guaranteed to be
faithful, and can produce potentially misleading interpretations. In this work,
we develop Sum-of-Parts (SOP), a class of models whose predictions come with
grouped feature attributions that are faithful-by-construction. This model
decomposes a prediction into an interpretable sum of scores, each of which is
directly attributable to a sparse group of features. We evaluate SOP on
benchmarks with standard interpretability metrics, and in a case study, we use
the faithful explanations from SOP to help astrophysicists discover new
knowledge about galaxy formation
Robust Local Explanations for Healthcare Predictive Analytics: An Application to Fragility Fracture Risk Modeling
With recent advancements in data analytics, healthcare predictive analytics (HPA) is garnering growing interest among practitioners and researchers. However, it is risky to blindly accept the results and users will not accept the HPA model if transparency is not guaranteed. To address this challenge, we propose the RObust Local EXplanations (ROLEX) method, which provides robust, instance-level explanations for any HPA model. The applicability of the ROLEX method is demonstrated using the fragility fracture prediction problem. Analysis with a large real-world dataset demonstrates that our method outperforms state-of-the-art methods in terms of local fidelity. The ROLEX method is applicable to various types of HPA problems beyond the fragility fracture problem. It is applicable to any type of supervised learning model and provides fine-grained explanations that can improve understanding of the phenomenon of interest. Finally, we discuss theoretical implications of our study in light of healthcare IS, big data, and design science
From local counterfactuals to global feature importance: efficient, robust, and model-agnostic explanations for brain connectivity networks
Background: Explainable artificial intelligence (XAI) is a technology that can enhance trust in mental state classifications by providing explanations for the reasoning behind artificial intelligence (AI) models outputs, especially for high-dimensional and highly-correlated brain signals. Feature importance and counterfactual explanations are two common approaches to generate these explanations, but both have drawbacks. While feature importance methods, such as shapley additive explanations (SHAP), can be computationally expensive and sensitive to feature correlation, counterfactual explanations only explain a single outcome instead of the entire model. Methods: To overcome these limitations, we propose a new procedure for computing global feature importance that involves aggregating local counterfactual explanations. This approach is specifically tailored to fMRI signals and is based on the hypothesis that instances close to the decision boundary and their counterfactuals mainly differ in the features identified as most important for the downstream classification task. We refer to this proposed feature importance measure as Boundary Crossing Solo Ratio (BoCSoR), since it quantifies the frequency with which a change in each feature in isolation leads to a change in classification outcome, i.e., the crossing of the model's decision boundary. Results and conclusions: Experimental results on synthetic data and real publicly available fMRI data from the Human Connect project show that the proposed BoCSoR measure is more robust to feature correlation and less computationally expensive than state-of-the-art methods. Additionally, it is equally effective in providing an explanation for the behavior of any AI model for brain signals. These properties are crucial for medical decision support systems, where many different features are often extracted from the same physiological measures and a gold standard is absent. Consequently, computing feature importance may become computationally expensive, and there may be a high probability of mutual correlation among features, leading to unreliable results from state-of-the-art XAI methods