552 research outputs found
Towards Visually Explaining Variational Autoencoders
Recent advances in Convolutional Neural Network (CNN) model interpretability
have led to impressive progress in visualizing and understanding model
predictions. In particular, gradient-based visual attention methods have driven
much recent effort in using visual attention maps as a means for visual
explanations. A key problem, however, is these methods are designed for
classification and categorization tasks, and their extension to explaining
generative models, e.g. variational autoencoders (VAE) is not trivial. In this
work, we take a step towards bridging this crucial gap, proposing the first
technique to visually explain VAEs by means of gradient-based attention. We
present methods to generate visual attention from the learned latent space, and
also demonstrate such attention explanations serve more than just explaining
VAE predictions. We show how these attention maps can be used to localize
anomalies in images, demonstrating state-of-the-art performance on the MVTec-AD
dataset. We also show how they can be infused into model training, helping
bootstrap the VAE into learning improved latent space disentanglement,
demonstrated on the Dsprites dataset
pVAE: a physics-integrated generative model. Application to the semantic segmentation of optical remote sensing images
The combination of machine learning models with physical models is a recent
research path to learn robust data representations. In this paper, we introduce
pVAE, a generative model that integrates a perfect physical model which
partially explains the true underlying factors of variation in the data. To
fully leverage our hybrid design, we propose a semi-supervised optimization
procedure and an inference scheme that comes along meaningful uncertainty
estimates. We apply pVAE to the semantic segmentation of high-resolution
hyperspectral remote sensing images. Our experiments on a simulated data set
demonstrated the benefits of our hybrid model against conventional machine
learning models in terms of extrapolation capabilities and interpretability. In
particular, we show that pVAE naturally has high disentanglement
capabilities. Our code and data have been made publicly available at
https://github.com/Romain3Ch216/p3VAE.Comment: 21 pages, 11 figures, submitted to the International Journal of
Computer Visio
- …