Many problems in computer vision have recently been tackled using models
whose predictions cannot be easily interpreted, most commonly deep neural
networks. Surrogate explainers are a popular post-hoc interpretability method
to further understand how a model arrives at a particular prediction. By
training a simple, more interpretable model to locally approximate the decision
boundary of a non-interpretable system, we can estimate the relative importance
of the input features on the prediction. Focusing on images, surrogate
explainers, e.g., LIME, generate a local neighbourhood around a query image by
sampling in an interpretable domain. However, these interpretable domains have
traditionally been derived exclusively from the intrinsic features of the query
image, not taking into consideration the manifold of the data the
non-interpretable model has been exposed to in training (or more generally, the
manifold of real images). This leads to suboptimal surrogates trained on
potentially low probability images. We address this limitation by aligning the
local neighbourhood on which the surrogate is trained with the original
training data distribution, even when this distribution is not accessible. We
propose two approaches to do so, namely (1) altering the method for sampling
the local neighbourhood and (2) using perceptual metrics to convey some of the
properties of the distribution of natural images.Comment: 12 pages, 7 figure
