142 research outputs found
Communication-Avoiding Algorithms for a High-Performance Hyperbolic PDE Engine
The study of waves has always been an important subject of research. Earthquakes, for example,
have a direct impact on the daily lives of millions of people while gravitational waves reveal
insight into the composition and history of the Universe. These physical phenomena, despite
being tackled traditionally by different fields of physics, have in common that they are modelled
the same way mathematically: as a system of hyperbolic partial differential equations (PDEs).
The ExaHyPE project (“An Exascale Hyperbolic PDE Engine") translates this similarity into
a software engine that can be quickly adapted to simulate a wide range of hyperbolic partial
differential equations. ExaHyPE’s key idea is that the user only specifies the physics while the
engine takes care of the parallelisation and the interplay of the underlying numerical methods.
Consequently, a first simulation code for a new hyperbolic PDE can often be realised within a
few hours. This is a task that traditionally can take weeks, months, even years for researchers
starting from scratch.
My main contribution to ExaHyPE is the development of the core infrastructure. This
comprises the development and implementation of ExaHyPE’s solvers and adaptive mesh
refinement procedures, it’s MPI+X parallelisation as well as high-level aspects of ExaHyPE’s
application-tailored code generation, which allows to adapt ExaHyPE to model many different
hyperbolic PDE systems. Like any high-performance computing code, ExaHyPE has to tackle the
challenges of the coming exascale computing era, notably network communication latencies and
the growing memory wall. In this thesis, I propose memory-efficient realisations of ExaHyPE’s
solvers that avoid data movement together with a novel task-based MPI+X parallelisation
concept that allows to hide network communication behind computation in dynamically adaptive
simulations
Analysis and Design of Communication Avoiding Algorithms for Out of Memory(OOM) SVD
Many applications — including big data analytics, information retrieval, gene expression analysis, and numerical weather prediction – require the solution of large, dense singular value decomposition (SVD). The size of matrices used in many of these applications is becoming too large to fit into into a computer’s main memory at one time, and the traditional SVD algorithms that require all the matrix components to be loaded into memory before computation starts cannot be used directly. Moving data (communication) between levels of memory hierarchy and the disk exposes extra challenges to design SVD for such big matrices because of the exponential growth in the gap between floating-point arithmetic rate and bandwidth for many different storage devices on modern high performance computers. In this dissertation, we have analyzed communication overhead on hierarchical memory systems and disks for SVD algorithms and designed communication-avoiding (CA) Out of Memory (OOM) SVD algorithms. By Out of Memory we mean that the matrix is too big to fit in the main memory and therefore must reside in external or internal storage. We have studied communication overhead for classical one-stage blocked SVD and two-stage tiled SVD algorithms and proposed our OOM SVD algorithm, which reduces the communication cost. We have presented theoretical analysis and strategies to design CA OOM SVD algorithms, developed optimized implementation of CA OOM SVD for multicore architecture, and presented its performance results.
When matrices are tall, performance of OOM SVD can be improved significantly by carrying out QR decomposition on the original matrix in the first place. The upper triangular matrix generated by QR decomposition may fit in the main memory, and in-core SVD can be used efficiently. Even if the upper triangular matrix does not fit in the main memory, OOM SVD will work on a smaller matrix. That is why we have analyzed communication reduction for OOM QR algorithm, implemented optimized OOM tiled QR for multicore systems and showed performance improvement of OOM SVD algorithms for tall matrices
Computing the R of the QR factorization of tall and skinny matrices using MPI_Reduce
A QR factorization of a tall and skinny matrix with n columns can be
represented as a reduction. The operation used along the reduction tree has in
input two n-by-n upper triangular matrices and in output an n-by-n upper
triangular matrix which is defined as the R factor of the two input matrices
stacked the one on top of the other. This operation is binary, associative, and
commutative. We can therefore leverage the MPI library capabilities by using
user-defined MPI operations and MPI_Reduce to perform this reduction. The
resulting code is compact and portable. In this context, the user relies on the
MPI library to select a reduction tree appropriate for the underlying
architecture
Efficient Algorithms for Solving Structured Eigenvalue Problems Arising in the Description of Electronic Excitations
Matrices arising in linear-response time-dependent density functional theory and many-body perturbation theory, in particular in the Bethe-Salpeter approach, show a 2 × 2 block structure. The motivation to devise new algorithms, instead of using general purpose eigenvalue solvers, comes from the need to solve large problems on high performance computers. This requires parallelizable and communication-avoiding algorithms and implementations. We point out various novel directions for diagonalizing structured matrices. These include the solution of skew-symmetric eigenvalue problems in ELPA, as well as structure preserving spectral divide-and-conquer schemes employing generalized polar decompostions
Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation
Across a variety of scientific disciplines, sparse inverse covariance
estimation is a popular tool for capturing the underlying dependency
relationships in multivariate data. Unfortunately, most estimators are not
scalable enough to handle the sizes of modern high-dimensional data sets (often
on the order of terabytes), and assume Gaussian samples. To address these
deficiencies, we introduce HP-CONCORD, a highly scalable optimization method
for estimating a sparse inverse covariance matrix based on a regularized
pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal
gradient method uses a novel communication-avoiding linear algebra algorithm
and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving
parallel scalability on problems with up to ~819 billion parameters (1.28
million dimensions); even on a single node, HP-CONCORD demonstrates
scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to
estimate the underlying dependency structure of the brain from fMRI data, and
use the result to identify functional regions automatically. The results show
good agreement with a clustering from the neuroscience literature.Comment: Main paper: 15 pages, appendix: 24 page
OverSketch: Approximate Matrix Multiplication for the Cloud
We propose OverSketch, an approximate algorithm for distributed matrix
multiplication in serverless computing. OverSketch leverages ideas from matrix
sketching and high-performance computing to enable cost-efficient
multiplication that is resilient to faults and straggling nodes pervasive in
low-cost serverless architectures. We establish statistical guarantees on the
accuracy of OverSketch and empirically validate our results by solving a
large-scale linear program using interior-point methods and demonstrate a 34%
reduction in compute time on AWS Lambda.Comment: Published in Proc. IEEE Big Data 2018. Updated version provides
details of distributed sketching and highlights other advantages of
OverSketc
- …