101 research outputs found
Scalable iterative methods for sampling from massive Gaussian random vectors
Sampling from Gaussian Markov random fields (GMRFs), that is multivariate
Gaussian ran- dom vectors that are parameterised by the inverse of their
covariance matrix, is a fundamental problem in computational statistics. In
this paper, we show how we can exploit arbitrarily accu- rate approximations to
a GMRF to speed up Krylov subspace sampling methods. We also show that these
methods can be used when computing the normalising constant of a large
multivariate Gaussian distribution, which is needed for both any
likelihood-based inference method. The method we derive is also applicable to
other structured Gaussian random vectors and, in particu- lar, we show that
when the precision matrix is a perturbation of a (block) circulant matrix, it
is still possible to derive O(n log n) sampling schemes.Comment: 17 Pages, 4 Figure
Fast and Robust Parametric Estimation of Jointly Sparse Channels
We consider the joint estimation of multipath channels obtained with a set of
receiving antennas and uniformly probed in the frequency domain. This scenario
fits most of the modern outdoor communication protocols for mobile access or
digital broadcasting among others.
Such channels verify a Sparse Common Support property (SCS) which was used in
a previous paper to propose a Finite Rate of Innovation (FRI) based sampling
and estimation algorithm. In this contribution we improve the robustness and
computational complexity aspects of this algorithm. The method is based on
projection in Krylov subspaces to improve complexity and a new criterion called
the Partial Effective Rank (PER) to estimate the level of sparsity to gain
robustness.
If P antennas measure a K-multipath channel with N uniformly sampled
measurements per channel, the algorithm possesses an O(KPNlogN) complexity and
an O(KPN) memory footprint instead of O(PN^3) and O(PN^2) for the direct
implementation, making it suitable for K << N. The sparsity is estimated online
based on the PER, and the algorithm therefore has a sense of introspection
being able to relinquish sparsity if it is lacking. The estimation performances
are tested on field measurements with synthetic AWGN, and the proposed
algorithm outperforms non-sparse reconstruction in the medium to low SNR range
(< 0dB), increasing the rate of successful symbol decodings by 1/10th in
average, and 1/3rd in the best case. The experiments also show that the
algorithm does not perform worse than a non-sparse estimation algorithm in
non-sparse operating conditions, since it may fall-back to it if the PER
criterion does not detect a sufficient level of sparsity.
The algorithm is also tested against a method assuming a "discrete" sparsity
model as in Compressed Sensing (CS). The conducted test indicates a trade-off
between speed and accuracy.Comment: 11 pages, 9 figures, submitted to IEEE JETCAS special issue on
Compressed Sensing, Sep. 201
CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebra
Many areas of machine learning and science involve large linear algebra
problems, such as eigendecompositions, solving linear systems, computing matrix
exponentials, and trace estimation. The matrices involved often have Kronecker,
convolutional, block diagonal, sum, or product structure. In this paper, we
propose a simple but general framework for large-scale linear algebra problems
in machine learning, named CoLA (Compositional Linear Algebra). By combining a
linear operator abstraction with compositional dispatch rules, CoLA
automatically constructs memory and runtime efficient numerical algorithms.
Moreover, CoLA provides memory efficient automatic differentiation, low
precision computation, and GPU acceleration in both JAX and PyTorch, while also
accommodating new objects, operations, and rules in downstream packages via
multiple dispatch. CoLA can accelerate many algebraic operations, while making
it easy to prototype matrix structures and algorithms, providing an appealing
drop-in tool for virtually any computational effort that requires linear
algebra. We showcase its efficacy across a broad range of applications,
including partial differential equations, Gaussian processes, equivariant model
construction, and unsupervised learning.Comment: Code available at https://github.com/wilson-labs/col
Structured Eigenvalue Problems
Most eigenvalue problems arising in practice are known to be structured. Structure is often introduced by discretization and linearization techniques but may also be a consequence of properties induced by the original problem. Preserving this structure can help preserve physically relevant symmetries in the eigenvalues of the matrix and may improve the accuracy and efficiency of an eigenvalue computation. The purpose of this brief survey is to highlight these facts for some common matrix structures. This includes a treatment of rather general concepts such as structured condition numbers and backward errors as well as an overview of algorithms and applications for several matrix classes including symmetric, skew-symmetric, persymmetric, block cyclic, Hamiltonian, symplectic and orthogonal matrices
The structure of iterative methods for symmetric linear discrete ill-posed problems
The iterative solution of large linear discrete ill-posed problems with an error contaminated data vector requires the use of specially designed methods in order to avoid severe error propagation. Range restricted minimal residual methods have been found to be well suited for the solution of many such problems. This paper discusses the structure of matrices that arise in a range restricted minimal residual method for the solution of large linear discrete ill-posed problems with a symmetric matrix. The exploitation of the structure results in a method that is competitive with respect to computer storage, number of iterations, and accuracy.Acknowledgments We would like to thank the referees for comments. The work of F. M. was supported
by DirecciĂłn General de InvestigaciĂłn CientĂfica y TĂ©cnica, Ministerio de EconomĂa y Competitividad of
Spain under grant MTM2012-36732-C03-01. Work of L. R. was supported by Universidad Carlos III de
Madrid in the Department of Mathematics during the academic year 2010-2011 within the framework of
the Chair of Excellence Program and by NSF grant DMS-1115385
Eigenvector sensitivity under general and structured perturbations of tridiagonal Toeplitz-type matrices
The sensitivity of eigenvalues of structured matrices under general or
structured perturbations of the matrix entries has been thoroughly studied in
the literature. Error bounds are available and the pseudospectrum can be
computed to gain insight. Few investigations have focused on analyzing the
sensitivity of eigenvectors under general or structured perturbations. The
present paper discusses this sensitivity for tridiagonal Toeplitz and
Toeplitz-type matrices.Comment: 21 pages, 4 figure
Preconditioned fast solvers for large linear systems with specific sparse and/or Toeplitz-like structures and applications
In this thesis, the design of the preconditioners we propose starts from applications instead of treating the problem in a completely general way. The reason is that not all types of linear systems can be addressed with the same tools. In this sense, the techniques for designing efficient iterative solvers depends mostly on properties inherited from the continuous problem, that has originated the discretized sequence of matrices. Classical examples are locality, isotropy in the PDE context, whose discrete counterparts are sparsity and matrices constant along the diagonals, respectively.
Therefore, it is often important to take into account the properties of the originating continuous model for obtaining better performances and for providing an accurate convergence analysis. We consider linear systems that arise in the solution of both linear and nonlinear partial differential equation of both integer and fractional type. For the latter case, an introduction to both the theory and the numerical treatment is given.
All the algorithms and the strategies presented in this thesis are developed having in mind their parallel implementation. In particular, we consider the processor-co-processor framework, in which the main part of the computation is performed on a Graphics Processing Unit (GPU) accelerator.
In Part I we introduce our proposal for sparse approximate inverse preconditioners for either the solution of time-dependent Partial Differential Equations (PDEs), Chapter 3, and Fractional Differential Equations (FDEs), containing both classical and fractional terms, Chapter 5. More precisely, we propose a new technique for updating preconditioners for dealing with sequences of linear systems for PDEs and FDEs, that can be used also to compute matrix functions of large matrices via quadrature formula in Chapter 4 and for optimal control of FDEs in Chapter 6. At last, in Part II, we consider structured preconditioners for quasi-Toeplitz systems. The focus is towards the numerical treatment of discretized convection-diffusion equations in Chapter 7 and on the solution of FDEs with linear multistep formula in boundary value form in Chapter 8
Preconditioners for Krylov subspace methods: An overview
When simulating a mechanism from science or engineering, or an industrial process, one is frequently required to construct a mathematical model, and then resolve this model numerically. If accurate numerical solutions are necessary or desirable, this can involve solving large-scale systems of equations. One major class of solution methods is that of preconditioned iterative methods, involving preconditioners which are computationally cheap to apply while also capturing information contained in the linear system. In this article, we give a short survey of the field of preconditioning. We introduce a range of preconditioners for partial differential equations, followed by optimization problems, before discussing preconditioners constructed with less standard objectives in mind
Preconditioned fast solvers for large linear systems with specific sparse and/or Toeplitz-like structures and applications
In this thesis, the design of the preconditioners we propose starts from applications instead of treating the problem in a completely general way. The reason is that not all types of linear systems can be addressed with the same tools. In this sense, the techniques for designing efficient iterative solvers depends mostly on properties inherited from the continuous problem, that has originated the discretized sequence of matrices. Classical examples are locality, isotropy in the PDE context, whose discrete counterparts are sparsity and matrices constant along the diagonals, respectively.
Therefore, it is often important to take into account the properties of the originating continuous model for obtaining better performances and for providing an accurate convergence analysis. We consider linear systems that arise in the solution of both linear and nonlinear partial differential equation of both integer and fractional type. For the latter case, an introduction to both the theory and the numerical treatment is given.
All the algorithms and the strategies presented in this thesis are developed having in mind their parallel implementation. In particular, we consider the processor-co-processor framework, in which the main part of the computation is performed on a Graphics Processing Unit (GPU) accelerator.
In Part I we introduce our proposal for sparse approximate inverse preconditioners for either the solution of time-dependent Partial Differential Equations (PDEs), Chapter 3, and Fractional Differential Equations (FDEs), containing both classical and fractional terms, Chapter 5. More precisely, we propose a new technique for updating preconditioners for dealing with sequences of linear systems for PDEs and FDEs, that can be used also to compute matrix functions of large matrices via quadrature formula in Chapter 4 and for optimal control of FDEs in Chapter 6. At last, in Part II, we consider structured preconditioners for quasi-Toeplitz systems. The focus is towards the numerical treatment of discretized convection-diffusion equations in Chapter 7 and on the solution of FDEs with linear multistep formula in boundary value form in Chapter 8
- âŠ