Artificial intelligence (AI) systems power the world we live in. Deep neural
networks (DNNs) are able to solve tasks in an ever-expanding landscape of
scenarios, but our eagerness to apply these powerful models leads us to focus
on their performance and deprioritises our ability to understand them. Current
research in the field of explainable AI tries to bridge this gap by developing
various perturbation or gradient-based explanation techniques. For images,
these techniques fail to fully capture and convey the semantic information
needed to elucidate why the model makes the predictions it does. In this work,
we develop a new form of explanation that is radically different in nature from
current explanation methods, such as Grad-CAM. Perception visualization
provides a visual representation of what the DNN perceives in the input image
by depicting what visual patterns the latent representation corresponds to.
Visualizations are obtained through a reconstruction model that inverts the
encoded features, such that the parameters and predictions of the original
models are not modified. Results of our user study demonstrate that humans can
better understand and predict the system's decisions when perception
visualizations are available, thus easing the debugging and deployment of deep
models as trusted systems.Comment: Accepted paper at BMVC 2021 (Proceedings not available yet