5,164 research outputs found
Deep Learning Methods for Parallel Magnetic Resonance Image Reconstruction
Following the success of deep learning in a wide range of applications,
neural network-based machine learning techniques have received interest as a
means of accelerating magnetic resonance imaging (MRI). A number of ideas
inspired by deep learning techniques from computer vision and image processing
have been successfully applied to non-linear image reconstruction in the spirit
of compressed sensing for both low dose computed tomography and accelerated
MRI. The additional integration of multi-coil information to recover missing
k-space lines in the MRI reconstruction process, is still studied less
frequently, even though it is the de-facto standard for currently used
accelerated MR acquisitions. This manuscript provides an overview of the recent
machine learning approaches that have been proposed specifically for improving
parallel imaging. A general background introduction to parallel MRI is given
that is structured around the classical view of image space and k-space based
methods. Both linear and non-linear methods are covered, followed by a
discussion of recent efforts to further improve parallel imaging using machine
learning, and specifically using artificial neural networks. Image-domain based
techniques that introduce improved regularizers are covered as well as k-space
based methods, where the focus is on better interpolation strategies using
neural networks. Issues and open problems are discussed as well as recent
efforts for producing open datasets and benchmarks for the community.Comment: 14 pages, 7 figure
Understanding Machine-learned Density Functionals
Kernel ridge regression is used to approximate the kinetic energy of
non-interacting fermions in a one-dimensional box as a functional of their
density. The properties of different kernels and methods of cross-validation
are explored, and highly accurate energies are achieved. Accurate {\em
constrained optimal densities} are found via a modified Euler-Lagrange
constrained minimization of the total energy. A projected gradient descent
algorithm is derived using local principal component analysis. Additionally, a
sparse grid representation of the density can be used without degrading the
performance of the methods. The implications for machine-learned density
functional approximations are discussed
Sampling of Planar Curves: Theory and Fast Algorithms
We introduce a continuous domain framework for the recovery of a planar curve
from a few samples. We model the curve as the zero level set of a trigonometric
polynomial. We show that the exponential feature maps of the points on the
curve lie on a low-dimensional subspace. We show that the null-space vector of
the feature matrix can be used to uniquely identify the curve, given a
sufficient number of samples. The worst-case theoretical guarantees show that
the number of samples required for unique recovery depends on the bandwidth of
the underlying trigonometric polynomial, which is a measure of the complexity
of the curve. We introduce an iterative algorithm that relies on the low-rank
property of the feature maps to recover the curves when the samples are noisy
or when the true bandwidth of the curve is unknown. We also demonstrate the
preliminary utility of the proposed curve representation in the context of
image segmentation
Image Reconstruction with Predictive Filter Flow
We propose a simple, interpretable framework for solving a wide range of
image reconstruction problems such as denoising and deconvolution. Given a
corrupted input image, the model synthesizes a spatially varying linear filter
which, when applied to the input image, reconstructs the desired output. The
model parameters are learned using supervised or self-supervised training. We
test this model on three tasks: non-uniform motion blur removal,
lossy-compression artifact reduction and single image super resolution. We
demonstrate that our model substantially outperforms state-of-the-art methods
on all these tasks and is significantly faster than optimization-based
approaches to deconvolution. Unlike models that directly predict output pixel
values, the predicted filter flow is controllable and interpretable, which we
demonstrate by visualizing the space of predicted filters for different tasks.Comment: https://www.ics.uci.edu/~skong2/pff.htm
Dual optimization for convex constrained objectives without the gradient-Lipschitz assumption
The minimization of convex objectives coming from linear supervised learning
problems, such as penalized generalized linear models, can be formulated as
finite sums of convex functions. For such problems, a large set of stochastic
first-order solvers based on the idea of variance reduction are available and
combine both computational efficiency and sound theoretical guarantees (linear
convergence rates). Such rates are obtained under both gradient-Lipschitz and
strong convexity assumptions. Motivated by learning problems that do not meet
the gradient-Lipschitz assumption, such as linear Poisson regression, we work
under another smoothness assumption, and obtain a linear convergence rate for a
shifted version of Stochastic Dual Coordinate Ascent (SDCA) that improves the
current state-of-the-art. Our motivation for considering a solver working on
the Fenchel-dual problem comes from the fact that such objectives include many
linear constraints, that are easier to deal with in the dual. Our approach and
theoretical findings are validated on several datasets, for Poisson regression
and another objective coming from the negative log-likelihood of the Hawkes
process, which is a family of models which proves extremely useful for the
modeling of information propagation in social networks and causality inference
Estimation of low-rank tensors via convex optimization
In this paper, we propose three approaches for the estimation of the Tucker
decomposition of multi-way arrays (tensors) from partial observations. All
approaches are formulated as convex minimization problems. Therefore, the
minimum is guaranteed to be unique. The proposed approaches can automatically
estimate the number of factors (rank) through the optimization. Thus, there is
no need to specify the rank beforehand. The key technique we employ is the
trace norm regularization, which is a popular approach for the estimation of
low-rank matrices. In addition, we propose a simple heuristic to improve the
interpretability of the obtained factorization. The advantages and
disadvantages of three proposed approaches are demonstrated through numerical
experiments on both synthetic and real world datasets. We show that the
proposed convex optimization based approaches are more accurate in predictive
performance, faster, and more reliable in recovering a known multilinear
structure than conventional approaches.Comment: 19 pages, 7 figure
Generalized system identification with stable spline kernels
Regularized least-squares approaches have been successfully applied to linear
system identification. Recent approaches use quadratic penalty terms on the
unknown impulse response defined by stable spline kernels, which control model
space complexity by leveraging regularity and bounded-input bounded-output
stability. This paper extends linear system identification to a wide class of
nonsmooth stable spline estimators, where regularization functionals and data
misfits can be selected from a rich set of piecewise linear-quadratic (PLQ)
penalties. This class includes the 1-norm, Huber, and Vapnik, in addition to
the least-squares penalty.
By representing penalties through their conjugates, the modeler can specify
any piecewise linear-quadratic penalty for misfit and regularizer, as well as
inequality constraints on the response. The interior-point solver we implement
(IPsolve) is locally quadratically convergent, with
arithmetic operations per iteration, where the number of unknown impulse
response coefficients and the number of observed output measurements.
IPsolve is competitive with available alternatives for system identification.
This is shown by a comparison with TFOCS, libSVM, and the FISTA algorithm. The
code is open source (https://github.com/saravkin/IPsolve).
The impact of the approach for system identification is illustrated with
numerical experiments featuring robust formulations for contaminated data,
relaxation systems, nonnegativity and unimodality constraints on the impulse
response, and sparsity promoting regularization. Incorporating constraints
yields particularly significant improvements.Comment: 23 pages, 6 figure
The Random Feature Model for Input-Output Maps between Banach Spaces
Well known to the machine learning community, the random feature model, originally introduced by Rahimi and Recht in 2008, is a parametric approximation to kernel interpolation or regression methods. It is typically used to approximate functions mapping a finite-dimensional input space to the real line. In this paper, we instead propose a methodology for use of the random feature model as a data-driven surrogate for operators that map an input Banach space to an output Banach space. Although the methodology is quite general, we consider operators defined by partial differential equations (PDEs); here, the inputs and outputs are themselves functions, with the input parameters being functions required to specify the problem, such as initial data or coefficients, and the outputs being solutions of the problem. Upon discretization, the model inherits several desirable attributes from this infinite-dimensional, function space viewpoint, including mesh-invariant approximation error with respect to the true PDE solution map and the capability to be trained at one mesh resolution and then deployed at different mesh resolutions. We view the random feature model as a non-intrusive data-driven emulator, provide a mathematical framework for its interpretation, and demonstrate its ability to efficiently and accurately approximate the nonlinear parameter-to-solution maps of two prototypical PDEs arising in physical science and engineering applications: viscous Burgers' equation and a variable coefficient elliptic equation
Linearly constrained Gaussian processes
We consider a modification of the covariance function in Gaussian processes
to correctly account for known linear constraints. By modelling the target
function as a transformation of an underlying function, the constraints are
explicitly incorporated in the model such that they are guaranteed to be
fulfilled by any sample drawn or prediction made. We also propose a
constructive procedure for designing the transformation operator and illustrate
the result on both simulated and real-data examples.Comment: A few fixes and added citation inforomatio
Data-driven Seismic Waveform Inversion: A Study on the Robustness and Generalization
Acoustic- and elastic-waveform inversion is an important and widely used
method to reconstruct subsurface velocity image. Waveform inversion is a
typical non-linear and ill-posed inverse problem. Existing physics-driven
computational methods for solving waveform inversion suffer from the cycle
skipping and local minima issues, and not to mention solving waveform inversion
is computationally expensive. In recent years, data-driven methods become a
promising way to solve the waveform inversion problem. However, most deep
learning frameworks suffer from generalization and over-fitting issue. In this
paper, we developed a real-time data-driven technique and we call it
VelocityGAN, to accurately reconstruct subsurface velocities. Our VelocityGAN
is built on a generative adversarial network (GAN) and trained end-to-end to
learn a mapping function from the raw seismic waveform data to the velocity
image. Different from other encoder-decoder based data-driven seismic waveform
inversion approaches, our VelocityGAN learns regularization from data and
further impose the regularization to the generator so that inversion accuracy
is improved. We further develop a transfer learning strategy based on
VelocityGAN to alleviate the generalization issue. A series of experiments are
conducted on the synthetic seismic reflection data to evaluate the
effectiveness, efficiency, and generalization of VelocityGAN. We not only
compare it with existing physics-driven approaches and data-driven frameworks
but also conduct several transfer learning experiments. The experiment results
show that VelocityGAN achieves state-of-the-art performance among the baselines
and can improve the generalization results to some extent
- …