2,926 research outputs found
Preconditioning Kernel Matrices
The computational and storage complexity of kernel machines presents the
primary barrier to their scaling to large, modern, datasets. A common way to
tackle the scalability issue is to use the conjugate gradient algorithm, which
relieves the constraints on both storage (the kernel matrix need not be stored)
and computation (both stochastic gradients and parallelization can be used).
Even so, conjugate gradient is not without its own issues: the conditioning of
kernel matrices is often such that conjugate gradients will have poor
convergence in practice. Preconditioning is a common approach to alleviating
this issue. Here we propose preconditioned conjugate gradients for kernel
machines, and develop a broad range of preconditioners particularly useful for
kernel matrices. We describe a scalable approach to both solving kernel
machines and learning their hyperparameters. We show this approach is exact in
the limit of iterations and outperforms state-of-the-art approximations for a
given computational budget
Composing Scalable Nonlinear Algebraic Solvers
Most efficient linear solvers use composable algorithmic components, with the
most common model being the combination of a Krylov accelerator and one or more
preconditioners. A similar set of concepts may be used for nonlinear algebraic
systems, where nonlinear composition of different nonlinear solvers may
significantly improve the time to solution. We describe the basic concepts of
nonlinear composition and preconditioning and present a number of solvers
applicable to nonlinear partial differential equations. We have developed a
software framework in order to easily explore the possible combinations of
solvers. We show that the performance gains from using composed solvers can be
substantial compared with gains from standard Newton-Krylov methods.Comment: 29 pages, 14 figures, 13 table
Robust Dropping Criteria for F-norm Minimization Based Sparse Approximate Inverse Preconditioning
Dropping tolerance criteria play a central role in Sparse Approximate Inverse
preconditioning. Such criteria have received, however, little attention and
have been treated heuristically in the following manner: If the size of an
entry is below some empirically small positive quantity, then it is set to
zero. The meaning of "small" is vague and has not been considered rigorously.
It has not been clear how dropping tolerances affect the quality and
effectiveness of a preconditioner . In this paper, we focus on the adaptive
Power Sparse Approximate Inverse algorithm and establish a mathematical theory
on robust selection criteria for dropping tolerances. Using the theory, we
derive an adaptive dropping criterion that is used to drop entries of small
magnitude dynamically during the setup process of . The proposed criterion
enables us to make both as sparse as possible as well as to be of
comparable quality to the potentially denser matrix which is obtained without
dropping. As a byproduct, the theory applies to static F-norm minimization
based preconditioning procedures, and a similar dropping criterion is given
that can be used to sparsify a matrix after it has been computed by a static
sparse approximate inverse procedure. In contrast to the adaptive procedure,
dropping in the static procedure does not reduce the setup time of the matrix
but makes the application of the sparser for Krylov iterations cheaper.
Numerical experiments reported confirm the theory and illustrate the robustness
and effectiveness of the dropping criteria.Comment: 27 pages, 2 figure
Preconditioned Locally Harmonic Residual Method for Computing Interior Eigenpairs of Certain Classes of Hermitian Matrices
We propose a Preconditioned Locally Harmonic Residual (PLHR) method for
computing several interior eigenpairs of a generalized Hermitian eigenvalue
problem, without traditional spectral transformations, matrix factorizations,
or inversions. PLHR is based on a short-term recurrence, easily extended to a
block form, computing eigenpairs simultaneously. PLHR can take advantage of
Hermitian positive definite preconditioning, e.g., based on an approximate
inverse of an absolute value of a shifted matrix, introduced in [SISC, 35
(2013), pp. A696-A718]. Our numerical experiments demonstrate that PLHR is
efficient and robust for certain classes of large-scale interior eigenvalue
problems, involving Laplacian and Hamiltonian operators, especially if memory
requirements are tight
Scalable iterative methods for sampling from massive Gaussian random vectors
Sampling from Gaussian Markov random fields (GMRFs), that is multivariate
Gaussian ran- dom vectors that are parameterised by the inverse of their
covariance matrix, is a fundamental problem in computational statistics. In
this paper, we show how we can exploit arbitrarily accu- rate approximations to
a GMRF to speed up Krylov subspace sampling methods. We also show that these
methods can be used when computing the normalising constant of a large
multivariate Gaussian distribution, which is needed for both any
likelihood-based inference method. The method we derive is also applicable to
other structured Gaussian random vectors and, in particu- lar, we show that
when the precision matrix is a perturbation of a (block) circulant matrix, it
is still possible to derive O(n log n) sampling schemes.Comment: 17 Pages, 4 Figure
On the Singular Neumann Problem in Linear Elasticity
The Neumann problem of linear elasticity is singular with a kernel formed by
the rigid motions of the body. There are several tricks that are commonly used
to obtain a non-singular linear system. However, they often cause reduced
accuracy or lead to poor convergence of the iterative solvers. In this paper,
different well-posed formulations of the problem are studied through
discretization by the finite element method, and preconditioning strategies
based on operator preconditioning are discussed. For each formulation we derive
preconditioners that are independent of the discretization parameter.
Preconditioners that are robust with respect to the first Lam\'e constant are
constructed for the pure displacement formulations, while a preconditioner that
is robust in both Lam\'e constants is constructed for the mixed formulation. It
is shown that, for convergence in the first Sobolev norm, it is crucial to
respect the orthogonality constraint derived from the continuous problem. Based
on this observation a modification to the conjugate gradient method is proposed
that achieves optimal error convergence of the computed solution
- …