12,898 research outputs found
Distributed Solution of Large-Scale Linear Systems via Accelerated Projection-Based Consensus
Solving a large-scale system of linear equations is a key step at the heart
of many algorithms in machine learning, scientific computing, and beyond. When
the problem dimension is large, computational and/or memory constraints make it
desirable, or even necessary, to perform the task in a distributed fashion. In
this paper, we consider a common scenario in which a taskmaster intends to
solve a large-scale system of linear equations by distributing subsets of the
equations among a number of computing machines/cores. We propose an accelerated
distributed consensus algorithm, in which at each iteration every machine
updates its solution by adding a scaled version of the projection of an error
signal onto the nullspace of its system of equations, and where the taskmaster
conducts an averaging over the solutions with momentum. The convergence
behavior of the proposed algorithm is analyzed in detail and analytically shown
to compare favorably with the convergence rate of alternative distributed
methods, namely distributed gradient descent, distributed versions of
Nesterov's accelerated gradient descent and heavy-ball method, the block
Cimmino method, and ADMM. On randomly chosen linear systems, as well as on
real-world data sets, the proposed method offers significant speed-up relative
to all the aforementioned methods. Finally, our analysis suggests a novel
variation of the distributed heavy-ball method, which employs a particular
distributed preconditioning, and which achieves the same theoretical
convergence rate as the proposed consensus-based method
Research and Education in Computational Science and Engineering
Over the past two decades the field of computational science and engineering
(CSE) has penetrated both basic and applied research in academia, industry, and
laboratories to advance discovery, optimize systems, support decision-makers,
and educate the scientific and engineering workforce. Informed by centuries of
theory and experiment, CSE performs computational experiments to answer
questions that neither theory nor experiment alone is equipped to answer. CSE
provides scientists and engineers of all persuasions with algorithmic
inventions and software systems that transcend disciplines and scales. Carried
on a wave of digital technology, CSE brings the power of parallelism to bear on
troves of data. Mathematics-based advanced computing has become a prevalent
means of discovery and innovation in essentially all areas of science,
engineering, technology, and society; and the CSE community is at the core of
this transformation. However, a combination of disruptive
developments---including the architectural complexity of extreme-scale
computing, the data revolution that engulfs the planet, and the specialization
required to follow the applications to new frontiers---is redefining the scope
and reach of the CSE endeavor. This report describes the rapid expansion of CSE
and the challenges to sustaining its bold advances. The report also presents
strategies and directions for CSE research and education for the next decade.Comment: Major revision, to appear in SIAM Revie
- …