19 research outputs found
From Robustness to Explainability and Back Again
In contrast with ad-hoc methods for eXplainable Artificial Intelligence
(XAI), formal explainability offers important guarantees of rigor. However,
formal explainability is hindered by poor scalability for some families of
classifiers, the most significant being neural networks. As a result, there are
concerns as to whether formal explainability might serve to complement other
approaches in delivering trustworthy AI. This paper addresses the limitation of
scalability of formal explainability, and proposes novel algorithms for
computing formal explanations. The novel algorithm computes explanations by
answering instead a number of robustness queries, and such that the number of
such queries is at most linear on the number of features. Consequently, the
proposed algorithm establishes a direct relationship between the practical
complexity of formal explainability and that of robustness. More importantly,
the paper generalizes the definition of formal explanation, thereby allowing
the use of robustness tools that are based on different distance norms, and
also by reasoning in terms of some target degree of robustness. The experiments
validate the practical efficiency of the proposed approach
A Refutation of Shapley Values for Explainability
Recent work demonstrated the existence of Boolean functions for which Shapley
values provide misleading information about the relative importance of features
in rule-based explanations. Such misleading information was broadly categorized
into a number of possible issues. Each of those issues relates with features
being relevant or irrelevant for a prediction, and all are significant
regarding the inadequacy of Shapley values for rule-based explainability. This
earlier work devised a brute-force approach to identify Boolean functions,
defined on small numbers of features, and also associated instances, which
displayed such inadequacy-revealing issues, and so served as evidence to the
inadequacy of Shapley values for rule-based explainability. However, an
outstanding question is how frequently such inadequacy-revealing issues can
occur for Boolean functions with arbitrary large numbers of features. It is
plain that a brute-force approach would be unlikely to provide insights on how
to tackle this question. This paper answers the above question by proving that,
for any number of features, there exist Boolean functions that exhibit one or
more inadequacy-revealing issues, thereby contributing decisive arguments
against the use of Shapley values as the theoretical underpinning of
feature-attribution methods in explainability
The Inadequacy of Shapley Values for Explainability
This paper develops a rigorous argument for why the use of Shapley values in
explainable AI (XAI) will necessarily yield provably misleading information
about the relative importance of features for predictions. Concretely, this
paper demonstrates that there exist classifiers, and associated predictions,
for which the relative importance of features determined by the Shapley values
will incorrectly assign more importance to features that are provably
irrelevant for the prediction, and less importance to features that are
provably relevant for the prediction. The paper also argues that, given recent
complexity results, the existence of efficient algorithms for the computation
of rigorous feature attribution values in the case of some restricted classes
of classifiers should be deemed unlikely at best
On Computing Probabilistic Abductive Explanations
The most widely studied explainable AI (XAI) approaches are unsound. This is
the case with well-known model-agnostic explanation approaches, and it is also
the case with approaches based on saliency maps. One solution is to consider
intrinsic interpretability, which does not exhibit the drawback of unsoundness.
Unfortunately, intrinsic interpretability can display unwieldy explanation
redundancy. Formal explainability represents the alternative to these
non-rigorous approaches, with one example being PI-explanations. Unfortunately,
PI-explanations also exhibit important drawbacks, the most visible of which is
arguably their size. Recently, it has been observed that the (absolute) rigor
of PI-explanations can be traded off for a smaller explanation size, by
computing the so-called relevant sets. Given some positive {\delta}, a set S of
features is {\delta}-relevant if, when the features in S are fixed, the
probability of getting the target class exceeds {\delta}. However, even for
very simple classifiers, the complexity of computing relevant sets of features
is prohibitive, with the decision problem being NPPP-complete for circuit-based
classifiers. In contrast with earlier negative results, this paper investigates
practical approaches for computing relevant sets for a number of widely used
classifiers that include Decision Trees (DTs), Naive Bayes Classifiers (NBCs),
and several families of classifiers obtained from propositional languages.
Moreover, the paper shows that, in practice, and for these families of
classifiers, relevant sets are easy to compute. Furthermore, the experiments
confirm that succinct sets of relevant features can be obtained for the
families of classifiers considered.Comment: arXiv admin note: text overlap with arXiv:2207.04748,
arXiv:2205.0956
On Deciding Feature Membership in Explanations of SDD & Related Classifiers
When reasoning about explanations of Machine Learning (ML) classifiers, a pertinent query is to decide whether some sensitive features can serve for explaining a given prediction. Recent work showed that the feature membership problem (FMP) is hard for for a broad class of classifiers. In contrast, this paper shows that for a number of families of classifiers, FMP is in NP. Concretely, the paper proves that any classifier for which an explanation can be computed in polynomial time, then deciding feature membership in an explanation can be decided with one NP oracle call. The paper then proposes propositional encodings for classifiers represented with Sentential Decision Diagrams (SDDs) and for other related propositional languages. The experimental results confirm the practical efficiency of the proposed approach
From Decision Trees to Explained Decision Sets
International audienceRecent work demonstrated that path explanation redundancy is ubiquitous in decision trees, i.e. most often paths in decision trees include literals that are redundant for explaining a prediction. The implication of this result is that decision trees must be explained. Nevertheless, there are applications of DTs where running an explanation algorithm is impractical. For example, in settings that are time or power constrained, running software algorithms for explaining predictions would be undesirable. Although the explanations for paths in DTs do not generally represent themselves a decision tree, this paper shows that one can construct a decision set from some of the decision tree explanations, such that the decision set is not only explained, but it also exhibits a number of properties that are critical for replacing the original decision tree
From Robustness to Explainability and Back Again
In contrast with ad-hoc methods for eXplainable Artificial Intelligence (XAI), formal explainability offers important guarantees of rigor. However, formal explainability is hindered by poor scalability for some families of classifiers, the most significant being neural networks. As a result, there are concerns as to whether formal explainability might serve to complement other approaches in delivering trustworthy AI. This paper addresses the limitation of scalability of formal explainability, and proposes novel algorithms for computing formal explanations. The novel algorithm computes explanations by answering instead a number of robustness queries, and such that the number of such queries is at most linear on the number of features. Consequently, the proposed algorithm establishes a direct relationship between the practical complexity of formal explainability and that of robustness. More importantly, the paper generalizes the definition of formal explanation, thereby allowing the use of robustness tools that are based on different distance norms, and also by reasoning in terms of some target degree of robustness. The experiments validate the practical efficiency of the proposed approach
A Refutation of Shapley Values for Explainability
Recent work demonstrated the existence of Boolean functions for which Shapley values provide misleading information about the relative importance of features in rule-based explanations. Such misleading information was broadly categorized into a number of possible issues. Each of those issues relates with features being relevant or irrelevant for a prediction, and all are significant regarding the inadequacy of Shapley values for rule-based explainability. This earlier work devised a brute-force approach to identify Boolean functions, defined on small numbers of features, and also associated instances, which displayed such inadequacy-revealing issues, and so served as evidence to the inadequacy of Shapley values for rule-based explainability. However, an outstanding question is how frequently such inadequacy-revealing issues can occur for Boolean functions with arbitrary large numbers of features. It is plain that a brute-force approach would be unlikely to provide insights on how to tackle this question. This paper answers the above question by proving that, for any number of features, there exist Boolean functions that exhibit one or more inadequacy-revealing issues, thereby contributing decisive arguments against the use of Shapley values as the theoretical underpinning of feature-attribution methods in explainability
Explainability is NOT a Game
Publication prévue lors d'une conférence de l'Association for Computing Machinery (ACM)Explainable artificial intelligence (XAI) aims to help human decision-makers in understanding complex machine learning (ML) models. One of the hallmarks of XAI are measures of relative feature importance, which are theoretically justified through the use of Shapley values. This paper builds on recent work and offers a simple argument for why Shapley values can provide misleading measures of relative feature importance, by assigning more importance to features that are irrelevant for a prediction, and assigning less importance to features that are relevant for a prediction. The significance of these results is that they effectively challenge the many proposed uses of measures of relative feature importance in a fast-growing range of high-stakes application domains. CCS Concepts • Computing methodologies → Artificial intelligence; Machine learning algorithms; Machine learning; • Theory of computation → Automated reasoning