118 research outputs found
Explainable Planning
As AI is increasingly being adopted into application solutions, the challenge
of supporting interaction with humans is becoming more apparent. Partly this is
to support integrated working styles, in which humans and intelligent systems
cooperate in problem-solving, but also it is a necessary step in the process of
building trust as humans migrate greater responsibility to such systems. The
challenge is to find effective ways to communicate the foundations of AI-driven
behaviour, when the algorithms that drive it are far from transparent to
humans. In this paper we consider the opportunities that arise in AI planning,
exploiting the model-based representations that form a familiar and common
basis for communication with users, while acknowledging the gap between
planning algorithms and human problem-solving.Comment: Presented at the IJCAI-17 workshop on Explainable AI
(http://home.earthlink.net/~dwaha/research/meetings/ijcai17-xai/). Melbourne,
August 201
CASP Solutions for Planning in Hybrid Domains
CASP is an extension of ASP that allows for numerical constraints to be added
in the rules. PDDL+ is an extension of the PDDL standard language of automated
planning for modeling mixed discrete-continuous dynamics.
In this paper, we present CASP solutions for dealing with PDDL+ problems,
i.e., encoding from PDDL+ to CASP, and extensions to the algorithm of the EZCSP
CASP solver in order to solve CASP programs arising from PDDL+ domains. An
experimental analysis, performed on well-known linear and non-linear variants
of PDDL+ domains, involving various configurations of the EZCSP solver, other
CASP solvers, and PDDL+ planners, shows the viability of our solution.Comment: Under consideration in Theory and Practice of Logic Programming
(TPLP
Bayesian Hierarchical Models for Counterfactual Estimation
Counterfactual explanations utilize feature perturbations to analyze the
outcome of an original decision and recommend an actionable recourse. We argue
that it is beneficial to provide several alternative explanations rather than a
single point solution and propose a probabilistic paradigm to estimate a
diverse set of counterfactuals. Specifically, we treat the perturbations as
random variables endowed with prior distribution functions. This allows
sampling multiple counterfactuals from the posterior density, with the added
benefit of incorporating inductive biases, preserving domain specific
constraints and quantifying uncertainty in estimates. More importantly, we
leverage Bayesian hierarchical modeling to share information across different
subgroups of a population, which can both improve robustness and measure
fairness. A gradient based sampler with superior convergence characteristics
efficiently computes the posterior samples. Experiments across several datasets
demonstrate that the counterfactuals estimated using our approach are valid,
sparse, diverse and feasible
PDDL+ Planning with Hybrid Automata: Foundations of Translating Must Behavior
Planning in hybrid domains poses a special challenge due to the involved mixed discrete-continuous dynamics. A recent solving approach for such domains is based on applying model checking techniques on a translation of PDDL+ planning problems to hybrid automata. However, the proposed translation is limited because must behavior is only overapproximated, and hence, processes and events are not re- flected exactly. In this paper, we present the theoretical foundation of an exact PDDL+ translation. We propose a schema to convert a hybrid automaton with must transitions into an equivalent hybrid automaton featuring only may transitions
SHAP@k:Efficient and Probably Approximately Correct (PAC) Identification of Top-k Features
The SHAP framework provides a principled method to explain the predictions of
a model by computing feature importance. Motivated by applications in finance,
we introduce the Top-k Identification Problem (TkIP), where the objective is to
identify the k features with the highest SHAP values. While any method to
compute SHAP values with uncertainty estimates (such as KernelSHAP and
SamplingSHAP) can be trivially adapted to solve TkIP, doing so is highly sample
inefficient. The goal of our work is to improve the sample efficiency of
existing methods in the context of solving TkIP. Our key insight is that TkIP
can be framed as an Explore-m problem--a well-studied problem related to
multi-armed bandits (MAB). This connection enables us to improve sample
efficiency by leveraging two techniques from the MAB literature: (1) a better
stopping-condition (to stop sampling) that identifies when PAC (Probably
Approximately Correct) guarantees have been met and (2) a greedy sampling
scheme that judiciously allocates samples between different features. By
adopting these methods we develop KernelSHAP@k and SamplingSHAP@k to
efficiently solve TkIP, offering an average improvement of in
sample-efficiency and runtime across most common credit related datasets
Comparing Apples to Oranges: Learning Similarity Functions for Data Produced by Different Distributions
Similarity functions measure how comparable pairs of elements are, and play a
key role in a wide variety of applications, e.g., notions of Individual
Fairness abiding by the seminal paradigm of Dwork et al., as well as Clustering
problems. However, access to an accurate similarity function should not always
be considered guaranteed, and this point was even raised by Dwork et al. For
instance, it is reasonable to assume that when the elements to be compared are
produced by different distributions, or in other words belong to different
``demographic'' groups, knowledge of their true similarity might be very
difficult to obtain. In this work, we present an efficient sampling framework
that learns these across-groups similarity functions, using only a limited
amount of experts' feedback. We show analytical results with rigorous
theoretical bounds, and empirically validate our algorithms via a large suite
of experiments.Comment: Accepted at NeurIPS 202
Towards learning to explain with concept bottleneck models: mitigating information leakage
Concept bottleneck models perform classification by first predicting which of
a list of human provided concepts are true about a datapoint. Then a downstream
model uses these predicted concept labels to predict the target label. The
predicted concepts act as a rationale for the target prediction. Model trust
issues emerge in this paradigm when soft concept labels are used: it has
previously been observed that extra information about the data distribution
leaks into the concept predictions. In this work we show how Monte-Carlo
Dropout can be used to attain soft concept predictions that do not contain
leaked information
Robust Counterfactual Explanations for Neural Networks With Probabilistic Guarantees
There is an emerging interest in generating robust counterfactual
explanations that would remain valid if the model is updated or changed even
slightly. Towards finding robust counterfactuals, existing literature often
assumes that the original model and the new model are bounded in the
parameter space, i.e., .
However, models can often change significantly in the parameter space with
little to no change in their predictions or accuracy on the given dataset. In
this work, we introduce a mathematical abstraction termed
\emph{naturally-occurring} model change, which allows for arbitrary changes in
the parameter space such that the change in predictions on points that lie on
the data manifold is limited. Next, we propose a measure -- that we call
\emph{Stability} -- to quantify the robustness of counterfactuals to potential
model changes for differentiable models, e.g., neural networks. Our main
contribution is to show that counterfactuals with sufficiently high value of
\emph{Stability} as defined by our measure will remain valid after potential
``naturally-occurring'' model changes with high probability (leveraging
concentration bounds for Lipschitz function of independent Gaussians). Since
our quantification depends on the local Lipschitz constant around a data point
which is not always available, we also examine practical relaxations of our
proposed measure and demonstrate experimentally how they can be incorporated to
find robust counterfactuals for neural networks that are close, realistic, and
remain valid after potential model changes. This work also has interesting
connections with model multiplicity, also known as, the Rashomon effect.Comment: International Conference on Machine Learning (ICML), 202
Robust Counterfactual Explanations for Tree-Based Ensembles
Counterfactual explanations inform ways to achieve a desired outcome from a
machine learning model. However, such explanations are not robust to certain
real-world changes in the underlying model (e.g., retraining the model,
changing hyperparameters, etc.), questioning their reliability in several
applications, e.g., credit lending. In this work, we propose a novel strategy
-- that we call RobX -- to generate robust counterfactuals for tree-based
ensembles, e.g., XGBoost. Tree-based ensembles pose additional challenges in
robust counterfactual generation, e.g., they have a non-smooth and
non-differentiable objective function, and they can change a lot in the
parameter space under retraining on very similar data. We first introduce a
novel metric -- that we call Counterfactual Stability -- that attempts to
quantify how robust a counterfactual is going to be to model changes under
retraining, and comes with desirable theoretical properties. Our proposed
strategy RobX works with any counterfactual generation method (base method) and
searches for robust counterfactuals by iteratively refining the counterfactual
generated by the base method using our metric Counterfactual Stability. We
compare the performance of RobX with popular counterfactual generation methods
(for tree-based ensembles) across benchmark datasets. The results demonstrate
that our strategy generates counterfactuals that are significantly more robust
(nearly 100% validity after actual model changes) and also realistic (in terms
of local outlier factor) over existing state-of-the-art methods.Comment: Accepted at ICML 202
- âŠ