1,201 research outputs found
Recommended from our members
Data-scalable Hessian preconditioning for distributed parameter PDE-constrained inverse problems
Hessian preconditioners are the key to efficient numerical solution of large-scale distributed parameter PDE-constrained inverse problems with highly informative data. Such inverse problems arise in many applications, yet solving them remains computationally costly. With existing methods, the computational cost depends on spectral properties of the Hessian which worsen as more informative data are used to reconstruct the unknown parameter field. The best case scenario from a scientific standpoint (lots of high-quality data) is therefore the worst case scenario from a computational standpoint (large computational cost).
In this dissertation, we argue that the best way to overcome this predicament is to build data-scalable Hessian/KKT preconditioners---preconditioners that perform well even if the data are highly informative about the parameter. We present a novel data-scalable KKT preconditioner for a diffusion inverse problem, a novel data-scalable Hessian preconditioner for an advection inverse problem, and a novel data-scalable domain decomposition preconditioner for an auxiliary operator that arises in connection with KKT preconditioning for a wave inverse problem. Our novel preconditioners outperform existing preconditioners in all three cases: they are robust to large numbers of observations in the diffusion inverse problem, large Peclet numbers in the advection inverse problem, and high wave frequencies in the wave inverse problem.Computational Science, Engineering, and Mathematic
Point spread function approximation of high rank Hessians with locally supported non-negative integral kernels
We present an efficient matrix-free point spread function (PSF) method for
approximating operators that have locally supported non-negative integral
kernels. The method computes impulse responses of the operator at scattered
points, and interpolates these impulse responses to approximate integral kernel
entries. Impulse responses are computed by applying the operator to Dirac comb
batches of point sources, which are chosen by solving an ellipsoid packing
problem. Evaluation of kernel entries allows us to construct a hierarchical
matrix (H-matrix) approximation of the operator. Further matrix computations
are performed with H-matrix methods. We use the method to build preconditioners
for the Hessian operator in two inverse problems governed by partial
differential equations (PDEs): inversion for the basal friction coefficient in
an ice sheet flow problem and for the initial condition in an
advective-diffusive transport problem. While for many ill-posed inverse
problems the Hessian of the data misfit term exhibits a low rank structure, and
hence a low rank approximation is suitable, for many problems of practical
interest the numerical rank of the Hessian is still large. But Hessian impulse
responses typically become more local as the numerical rank increases, which
benefits the PSF method. Numerical results reveal that the PSF preconditioner
clusters the spectrum of the preconditioned Hessian near one, yielding roughly
5x-10x reductions in the required number of PDE solves, as compared to
regularization preconditioning and no preconditioning. We also present a
numerical study for the influence of various parameters (that control the shape
of the impulse responses) on the effectiveness of the advection-diffusion
Hessian approximation. The results show that the PSF-based preconditioners are
able to form good approximations of high-rank Hessians using a small number of
operator applications
Geometric deep learning: going beyond Euclidean data
Many scientific fields study data with an underlying structure that is a
non-Euclidean space. Some examples include social networks in computational
social sciences, sensor networks in communications, functional networks in
brain imaging, regulatory networks in genetics, and meshed surfaces in computer
graphics. In many applications, such geometric data are large and complex (in
the case of social networks, on the scale of billions), and are natural targets
for machine learning techniques. In particular, we would like to use deep
neural networks, which have recently proven to be powerful tools for a broad
range of problems from computer vision, natural language processing, and audio
analysis. However, these tools have been most successful on data with an
underlying Euclidean or grid-like structure, and in cases where the invariances
of these structures are built into networks used to model them. Geometric deep
learning is an umbrella term for emerging techniques attempting to generalize
(structured) deep neural models to non-Euclidean domains such as graphs and
manifolds. The purpose of this paper is to overview different examples of
geometric deep learning problems and present available solutions, key
difficulties, applications, and future research directions in this nascent
field
Fast space-variant elliptical filtering using box splines
The efficient realization of linear space-variant (non-convolution) filters
is a challenging computational problem in image processing. In this paper, we
demonstrate that it is possible to filter an image with a Gaussian-like
elliptic window of varying size, elongation and orientation using a fixed
number of computations per pixel. The associated algorithm, which is based on a
family of smooth compactly supported piecewise polynomials, the
radially-uniform box splines, is realized using pre-integration and local
finite-differences. The radially-uniform box splines are constructed through
the repeated convolution of a fixed number of box distributions, which have
been suitably scaled and distributed radially in an uniform fashion. The
attractive features of these box splines are their asymptotic behavior, their
simple covariance structure, and their quasi-separability. They converge to
Gaussians with the increase of their order, and are used to approximate
anisotropic Gaussians of varying covariance simply by controlling the scales of
the constituent box distributions. Based on the second feature, we develop a
technique for continuously controlling the size, elongation and orientation of
these Gaussian-like functions. Finally, the quasi-separable structure, along
with a certain scaling property of box distributions, is used to efficiently
realize the associated space-variant elliptical filtering, which requires O(1)
computations per pixel irrespective of the shape and size of the filter.Comment: 12 figures; IEEE Transactions on Image Processing, vol. 19, 201
Continuous-variable quantum neural networks
We introduce a general method for building neural networks on quantum
computers. The quantum neural network is a variational quantum circuit built in
the continuous-variable (CV) architecture, which encodes quantum information in
continuous degrees of freedom such as the amplitudes of the electromagnetic
field. This circuit contains a layered structure of continuously parameterized
gates which is universal for CV quantum computation. Affine transformations and
nonlinear activation functions, two key elements in neural networks, are
enacted in the quantum network using Gaussian and non-Gaussian gates,
respectively. The non-Gaussian gates provide both the nonlinearity and the
universality of the model. Due to the structure of the CV model, the CV quantum
neural network can encode highly nonlinear transformations while remaining
completely unitary. We show how a classical network can be embedded into the
quantum formalism and propose quantum versions of various specialized model
such as convolutional, recurrent, and residual networks. Finally, we present
numerous modeling experiments built with the Strawberry Fields software
library. These experiments, including a classifier for fraud detection, a
network which generates Tetris images, and a hybrid classical-quantum
autoencoder, demonstrate the capability and adaptability of CV quantum neural
networks
A Panorama on Multiscale Geometric Representations, Intertwining Spatial, Directional and Frequency Selectivity
The richness of natural images makes the quest for optimal representations in
image processing and computer vision challenging. The latter observation has
not prevented the design of image representations, which trade off between
efficiency and complexity, while achieving accurate rendering of smooth regions
as well as reproducing faithful contours and textures. The most recent ones,
proposed in the past decade, share an hybrid heritage highlighting the
multiscale and oriented nature of edges and patterns in images. This paper
presents a panorama of the aforementioned literature on decompositions in
multiscale, multi-orientation bases or dictionaries. They typically exhibit
redundancy to improve sparsity in the transformed domain and sometimes its
invariance with respect to simple geometric deformations (translation,
rotation). Oriented multiscale dictionaries extend traditional wavelet
processing and may offer rotation invariance. Highly redundant dictionaries
require specific algorithms to simplify the search for an efficient (sparse)
representation. We also discuss the extension of multiscale geometric
decompositions to non-Euclidean domains such as the sphere or arbitrary meshed
surfaces. The etymology of panorama suggests an overview, based on a choice of
partially overlapping "pictures". We hope that this paper will contribute to
the appreciation and apprehension of a stream of current research directions in
image understanding.Comment: 65 pages, 33 figures, 303 reference
Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere
Fourier Neural Operators (FNOs) have proven to be an efficient and effective
method for resolution-independent operator learning in a broad variety of
application areas across scientific machine learning. A key reason for their
success is their ability to accurately model long-range dependencies in
spatio-temporal data by learning global convolutions in a computationally
efficient manner. To this end, FNOs rely on the discrete Fourier transform
(DFT), however, DFTs cause visual and spectral artifacts as well as pronounced
dissipation when learning operators in spherical coordinates since they
incorrectly assume a flat geometry. To overcome this limitation, we generalize
FNOs on the sphere, introducing Spherical FNOs (SFNOs) for learning operators
on spherical geometries. We apply SFNOs to forecasting atmospheric dynamics,
and demonstrate stable auto\-regressive rollouts for a year of simulated time
(1,460 steps), while retaining physically plausible dynamics. The SFNO has
important implications for machine learning-based simulation of climate
dynamics that could eventually help accelerate our response to climate change
- …