14 research outputs found
Stable and actionable explanations of black-box models through factual and counterfactual rules
Recent years have witnessed the rise of accurate but obscure classification models that hide the logic of their internal decision processes. Explaining the decision taken by a black-box classifier on a specific input instance is therefore of striking interest. We propose a local rule-based model-agnostic explanation method providing stable and actionable explanations. An explanation consists of a factual logic rule, stating the reasons for the black-box decision, and a set of actionable counterfactual logic rules, proactively suggesting the changes in the instance that lead to a different outcome. Explanations are computed from a decision tree that mimics the behavior of the black-box locally to the instance to explain. The decision tree is obtained through a bagging-like approach that favors stability and fidelity: first, an ensemble of decision trees is learned from neighborhoods of the instance under investigation; then, the ensemble is merged into a single decision tree. Neighbor instances are synthetically generated through a genetic algorithm whose fitness function is driven by the black-box behavior. Experiments show that the proposed method advances the state-of-the-art towards a comprehensive approach that successfully covers stability and actionability of factual and counterfactual explanations
Decomposing Counterfactual Explanations for Consequential Decision Making
The goal of algorithmic recourse is to reverse unfavorable decisions (e.g.,
from loan denial to approval) under automated decision making by suggesting
actionable feature changes (e.g., reduce the number of credit cards). To
generate low-cost recourse the majority of methods work under the assumption
that the features are independently manipulable (IMF). To address the feature
dependency issue the recourse problem is usually studied through the causal
recourse paradigm. However, it is well known that strong assumptions, as
encoded in causal models and structural equations, hinder the applicability of
these methods in complex domains where causal dependency structures are
ambiguous. In this work, we develop \texttt{DEAR} (DisEntangling Algorithmic
Recourse), a novel and practical recourse framework that bridges the gap
between the IMF and the strong causal assumptions. \texttt{DEAR} generates
recourses by disentangling the latent representation of co-varying features
from a subset of promising recourse features to capture the main practical
recourse desiderata. Our experiments on real-world data corroborate our
theoretically motivated recourse model and highlight our framework's ability to
provide reliable, low-cost recourse in the presence of feature dependencies
GLOBE-CE: A Translation-Based Approach for Global Counterfactual Explanations
Counterfactual explanations have been widely studied in explainability, with
a range of application dependent methods prominent in fairness, recourse and
model understanding. The major shortcoming associated with these methods,
however, is their inability to provide explanations beyond the local or
instance-level. While many works touch upon the notion of a global explanation,
typically suggesting to aggregate masses of local explanations in the hope of
ascertaining global properties, few provide frameworks that are both reliable
and computationally tractable. Meanwhile, practitioners are requesting more
efficient and interactive explainability tools. We take this opportunity to
propose Global & Efficient Counterfactual Explanations (GLOBE-CE), a flexible
framework that tackles the reliability and scalability issues associated with
current state-of-the-art, particularly on higher dimensional datasets and in
the presence of continuous features. Furthermore, we provide a unique
mathematical analysis of categorical feature translations, utilising it in our
method. Experimental evaluation with publicly available datasets and user
studies demonstrate that GLOBE-CE performs significantly better than the
current state-of-the-art across multiple metrics (e.g., speed, reliability).Comment: Published as a conference paper at ICML 2023 (9 page main text, 3
page references, 16 page appendix
Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse
As machine learning models are increasingly being employed to make
consequential decisions in real-world settings, it becomes critical to ensure
that individuals who are adversely impacted (e.g., loan denied) by the
predictions of these models are provided with a means for recourse. While
several approaches have been proposed to construct recourses for affected
individuals, the recourses output by these methods either achieve low costs
(i.e., ease-of-implementation) or robustness to small perturbations (i.e.,
noisy implementations of recourses), but not both due to the inherent
trade-offs between the recourse costs and robustness. Furthermore, prior
approaches do not provide end users with any agency over navigating the
aforementioned trade-offs. In this work, we address the above challenges by
proposing the first algorithmic framework which enables users to effectively
manage the recourse cost vs. robustness trade-offs. More specifically, our
framework Probabilistically ROBust rEcourse (\texttt{PROBE}) lets users choose
the probability with which a recourse could get invalidated (recourse
invalidation rate) if small changes are made to the recourse i.e., the recourse
is implemented somewhat noisily. To this end, we propose a novel objective
function which simultaneously minimizes the gap between the achieved
(resulting) and desired recourse invalidation rates, minimizes recourse costs,
and also ensures that the resulting recourse achieves a positive model
prediction. We develop novel theoretical results to characterize the recourse
invalidation rates corresponding to any given instance w.r.t. different classes
of underlying models (e.g., linear models, tree based models etc.), and
leverage these results to efficiently optimize the proposed objective.
Experimental evaluation with multiple real world datasets demonstrate the
efficacy of the proposed framework
On the Generation of Realistic and Robust Counterfactual Explanations for Algorithmic Recourse
This recent widespread deployment of machine learning algorithms presents many new challenges. Machine learning algorithms are usually opaque and can be particularly difficult to interpret. When humans are involved, algorithmic and automated decisions can negatively impact people’s lives. Therefore, end users would like to be insured against potential harm. One popular way to achieve this is to provide end users access to algorithmic recourse, which gives end users negatively affected by algorithmic decisions the opportunity to reverse unfavorable decisions, e.g., from a loan denial to a loan acceptance. In this thesis, we design recourse algorithms to meet various end user needs. First, we propose methods for the generation of realistic recourses. We use generative models to suggest recourses likely to occur under the data distribution. To this end, we shift the recourse action from the input space to the generative model’s latent space, allowing to generate counterfactuals that lie in regions with data support. Second, we observe that small changes applied to the recourses prescribed to end users likely invalidate the suggested recourse after being nosily implemented in practice. Motivated by this observation, we design methods for the generation of robust recourses and for assessing the robustness of recourse algorithms to data deletion requests. Third, the lack of a commonly used code-base for counterfactual explanation and algorithmic recourse algorithms and the vast array of evaluation measures in literature make it difficult to compare the per formance of different algorithms. To solve this problem, we provide an open source benchmarking library that streamlines the evaluation process and can be used for benchmarking, rapidly developing new methods, and setting up new
experiments. In summary, our work contributes to a more reliable interaction of end users and machine learned models by covering fundamental aspects of the recourse process and suggests new solutions towards generating realistic and robust counterfactual explanations for algorithmic recourse
How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
Context: Machine Learning (ML) has been at the heart of many innovations over
the past years. However, including it in so-called 'safety-critical' systems
such as automotive or aeronautic has proven to be very challenging, since the
shift in paradigm that ML brings completely changes traditional certification
approaches.
Objective: This paper aims to elucidate challenges related to the
certification of ML-based safety-critical systems, as well as the solutions
that are proposed in the literature to tackle them, answering the question 'How
to Certify Machine Learning Based Safety-critical Systems?'.
Method: We conduct a Systematic Literature Review (SLR) of research papers
published between 2015 to 2020, covering topics related to the certification of
ML systems. In total, we identified 217 papers covering topics considered to be
the main pillars of ML certification: Robustness, Uncertainty, Explainability,
Verification, Safe Reinforcement Learning, and Direct Certification. We
analyzed the main trends and problems of each sub-field and provided summaries
of the papers extracted.
Results: The SLR results highlighted the enthusiasm of the community for this
subject, as well as the lack of diversity in terms of datasets and type of
models. It also emphasized the need to further develop connections between
academia and industries to deepen the domain study. Finally, it also
illustrated the necessity to build connections between the above mention main
pillars that are for now mainly studied separately.
Conclusion: We highlighted current efforts deployed to enable the
certification of ML based software systems, and discuss some future research
directions.Comment: 60 pages (92 pages with references and complements), submitted to a
journal (Automated Software Engineering). Changes: Emphasizing difference
traditional software engineering / ML approach. Adding Related Works, Threats
to Validity and Complementary Materials. Adding a table listing papers
reference for each section/subsection
Machine Learning for Cyber Physical Systems
This open access proceedings presents new approaches to Machine Learning for Cyber Physical Systems, experiences and visions. It contains selected papers from the fifth international Conference ML4CPS – Machine Learning for Cyber Physical Systems, which was held in Berlin, March 12-13, 2020. Cyber Physical Systems are characterized by their ability to adapt and to learn: They analyze their environment and, based on observations, they learn patterns, correlations and predictive models. Typical applications are condition monitoring, predictive maintenance, image processing and diagnosis. Machine Learning is the key technology for these developments