5 research outputs found
Neural Poisson Surface Reconstruction: Resolution-Agnostic Shape Reconstruction from Point Clouds
We introduce Neural Poisson Surface Reconstruction (nPSR), an architecture
for shape reconstruction that addresses the challenge of recovering 3D shapes
from points. Traditional deep neural networks face challenges with common 3D
shape discretization techniques due to their computational complexity at higher
resolutions. To overcome this, we leverage Fourier Neural Operators to solve
the Poisson equation and reconstruct a mesh from oriented point cloud
measurements. nPSR exhibits two main advantages: First, it enables efficient
training on low-resolution data while achieving comparable performance at
high-resolution evaluation, thanks to the resolution-agnostic nature of FNOs.
This feature allows for one-shot super-resolution. Second, our method surpasses
existing approaches in reconstruction quality while being differentiable and
robust with respect to point sampling rates. Overall, the neural Poisson
surface reconstruction not only improves upon the limitations of classical deep
neural networks in shape reconstruction but also achieves superior results in
terms of reconstruction quality, running time, and resolution agnosticism
Explaining Image Classifiers with Multiscale Directional Image Representation
Image classifiers are known to be difficult to interpret and therefore
require explanation methods to understand their decisions. We present
ShearletX, a novel mask explanation method for image classifiers based on the
shearlet transform -- a multiscale directional image representation. Current
mask explanation methods are regularized by smoothness constraints that protect
against undesirable fine-grained explanation artifacts. However, the smoothness
of a mask limits its ability to separate fine-detail patterns, that are
relevant for the classifier, from nearby nuisance patterns, that do not affect
the classifier. ShearletX solves this problem by avoiding smoothness
regularization all together, replacing it by shearlet sparsity constraints. The
resulting explanations consist of a few edges, textures, and smooth parts of
the original image, that are the most relevant for the decision of the
classifier. To support our method, we propose a mathematical definition for
explanation artifacts and an information theoretic score to evaluate the
quality of mask explanations. We demonstrate the superiority of ShearletX over
previous mask based explanation methods using these new metrics, and present
exemplary situations where separating fine-detail patterns allows explaining
phenomena that were not explainable before
Learning Interpretable Queries for Explainable Image Classification with Information Pursuit
Information Pursuit (IP) is an explainable prediction algorithm that greedily
selects a sequence of interpretable queries about the data in order of
information gain, updating its posterior at each step based on observed
query-answer pairs. The standard paradigm uses hand-crafted dictionaries of
potential data queries curated by a domain expert or a large language model
after a human prompt. However, in practice, hand-crafted dictionaries are
limited by the expertise of the curator and the heuristics of prompt
engineering. This paper introduces a novel approach: learning a dictionary of
interpretable queries directly from the dataset. Our query dictionary learning
problem is formulated as an optimization problem by augmenting IP's variational
formulation with learnable dictionary parameters. To formulate learnable and
interpretable queries, we leverage the latent space of large vision and
language models like CLIP. To solve the optimization problem, we propose a new
query dictionary learning algorithm inspired by classical sparse dictionary
learning. Our experiments demonstrate that learned dictionaries significantly
outperform hand-crafted dictionaries generated with large language models
Deep microlocal reconstruction for limited-angle tomography
We present a deep-learning-based algorithm to jointly solve a reconstruction problem and a wavefront set extraction problem in tomographic imaging. The algorithm is based on a recently developed digital wavefront set extractor as well as the well-known microlocal canonical relation for the Radon transform. We use the wavefront set information about x-ray data to improve the reconstruction by requiring that the underlying neural networks simultaneously extract the correct ground truth wavefront set and ground truth image. As a necessary theoretical step, we identify the digital microlocal canonical relations for deep convolutional residual neural networks. We find strong numerical evidence for the effectiveness of this approach