805 research outputs found
Low-Rank Eigenvector Compression of Posterior Covariance Matrices for Linear Gaussian Inverse Problems
We consider the problem of estimating the uncertainty in statistical inverse
problems using Bayesian inference. When the probability density of the noise
and the prior are Gaussian, the solution of such a statistical inverse problem
is also Gaussian. Therefore, the underlying solution is characterized by the
mean and covariance matrix of the posterior probability density. However, the
covariance matrix of the posterior probability density is full and large.
Hence, the computation of such a matrix is impossible for large dimensional
parameter spaces. It is shown that for many ill-posed problems, the Hessian
matrix of the data misfit part has low numerical rank and it is therefore
possible to perform a low-rank approach to approximate the posterior covariance
matrix. For such a low-rank approximation, one needs to solve a forward partial
differential equation (PDE) and the adjoint PDE in both space and time. This in
turn gives complexity for both, computation and storage,
where is the dimension of the spatial domain and is the dimension
of the time domain. Such computations and storage demand are infeasible for
large problems. To overcome this obstacle, we develop a new approach that
utilizes a recently developed low-rank in time algorithm together with the
low-rank Hessian method. We reduce both the computational complexity and
storage requirement from to . We
use numerical experiments to illustrate the advantages of our approach
Randomized Dimension Reduction on Massive Data
Scalability of statistical estimators is of increasing importance in modern
applications and dimension reduction is often used to extract relevant
information from data. A variety of popular dimension reduction approaches can
be framed as symmetric generalized eigendecomposition problems. In this paper
we outline how taking into account the low rank structure assumption implicit
in these dimension reduction approaches provides both computational and
statistical advantages. We adapt recent randomized low-rank approximation
algorithms to provide efficient solutions to three dimension reduction methods:
Principal Component Analysis (PCA), Sliced Inverse Regression (SIR), and
Localized Sliced Inverse Regression (LSIR). A key observation in this paper is
that randomization serves a dual role, improving both computational and
statistical performance. This point is highlighted in our experiments on real
and simulated data.Comment: 31 pages, 6 figures, Key Words:dimension reduction, generalized
eigendecompositon, low-rank, supervised, inverse regression, random
projections, randomized algorithms, Krylov subspace method
Time-limited Balanced Truncation for Data Assimilation Problems
Balanced truncation is a well-established model order reduction method which
has been applied to a variety of problems. Recently, a connection between
linear Gaussian Bayesian inference problems and the system-theoretic concept of
balanced truncation has been drawn. Although this connection is new, the
application of balanced truncation to data assimilation is not a novel idea: it
has already been used in four-dimensional variational data assimilation
(4D-Var). This paper discusses the application of balanced truncation to linear
Gaussian Bayesian inference, and, in particular, the 4D-Var method, thereby
strengthening the link between systems theory and data assimilation further.
Similarities between both types of data assimilation problems enable a
generalisation of the state-of-the-art approach to the use of arbitrary prior
covariances as reachability Gramians. Furthermore, we propose an enhanced
approach using time-limited balanced truncation that allows to balance Bayesian
inference for unstable systems and in addition improves the numerical results
for short observation periods.Comment: 24 pages, 5 figure
How to estimate the 3D power spectrum of the Lyman- forest
We derive and numerically implement an algorithm for estimating the 3D power
spectrum of the Lyman- (Ly-) forest flux fluctuations. The
algorithm exploits the unique geometry of Ly- forest data to
efficiently measure the cross-spectrum between lines of sight as a function of
parallel wavenumber, transverse separation and redshift. The key to fast
evaluation is to approximate the global covariance matrix as block-diagonal,
where only pixels from the same spectrum are correlated. We then compute the
eigenvectors of the derivative of the signal covariance with respect to
cross-spectrum parameters, and project the inverse-covariance-weighted spectra
onto them. This acts much like a radial Fourier transform over redshift
windows. The resulting cross-spectrum inference is then converted into our
final product, an approximation of the likelihood for the 3D power spectrum
expressed as second order Taylor expansion around a fiducial model. We
demonstrate the accuracy and scalability of the algorithm and comment on
possible extensions. Our algorithm will allow efficient analysis of the
upcoming Dark Energy Spectroscopic Instrument dataset.Comment: 29 pages, many figures. Minor changes in v2, accepted in JCA
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives
Part 2 of this monograph builds on the introduction to tensor networks and
their operations presented in Part 1. It focuses on tensor network models for
super-compressed higher-order representation of data/parameters and related
cost functions, while providing an outline of their applications in machine
learning and data analytics. A particular emphasis is on the tensor train (TT)
and Hierarchical Tucker (HT) decompositions, and their physically meaningful
interpretations which reflect the scalability of the tensor network approach.
Through a graphical approach, we also elucidate how, by virtue of the
underlying low-rank tensor approximations and sophisticated contractions of
core tensors, tensor networks have the ability to perform distributed
computations on otherwise prohibitively large volumes of data/parameters,
thereby alleviating or even eliminating the curse of dimensionality. The
usefulness of this concept is illustrated over a number of applied areas,
including generalized regression and classification (support tensor machines,
canonical correlation analysis, higher order partial least squares),
generalized eigenvalue decomposition, Riemannian optimization, and in the
optimization of deep neural networks. Part 1 and Part 2 of this work can be
used either as stand-alone separate texts, or indeed as a conjoint
comprehensive review of the exciting field of low-rank tensor networks and
tensor decompositions.Comment: 232 page
- …