16,507 research outputs found
Simple, Parallel, High-Performance Virtual Machines for Extreme Computations
We introduce a high-performance virtual machine (VM) written in a numerically
fast language like Fortran or C to evaluate very large expressions. We discuss
the general concept of how to perform computations in terms of a VM and present
specifically a VM that is able to compute tree-level cross sections for any
number of external legs, given the corresponding byte code from the optimal
matrix element generator, O'Mega. Furthermore, this approach allows to
formulate the parallel computation of a single phase space point in a simple
and obvious way. We analyze hereby the scaling behaviour with multiple threads
as well as the benefits and drawbacks that are introduced with this method. Our
implementation of a VM can run faster than the corresponding native, compiled
code for certain processes and compilers, especially for very high
multiplicities, and has in general runtimes in the same order of magnitude. By
avoiding the tedious compile and link steps, which may fail for source code
files of gigabyte sizes, new processes or complex higher order corrections that
are currently out of reach could be evaluated with a VM given enough computing
power.Comment: 19 pages, 8 figure
Challenging the Computational Metaphor: Implications for How We Think
This paper explores the role of the traditional computational metaphor in our thinking as computer scientists, its influence on epistemological styles, and its implications for our understanding of cognition. It proposes to replace the conventional metaphor--a sequence of steps--with the notion of a community of interacting entities, and examines the ramifications of such a shift on these various ways in which we think
Making extreme computations possible with virtual machines
State-of-the-art algorithms generate scattering amplitudes for high-energy
physics at leading order for high-multiplicity processes as compiled code (in
Fortran, C or C++). For complicated processes the size of these libraries can
become tremendous (many GiB). We show that amplitudes can be translated to
byte-code instructions, which even reduce the size by one order of magnitude.
The byte-code is interpreted by a Virtual Machine with runtimes comparable to
compiled code and a better scaling with additional legs. We study the
properties of this algorithm, as an extension of the Optimizing Matrix Element
Generator (O'Mega). The bytecode matrix elements are available as alternative
input for the event generator WHIZARD. The bytecode interpreter can be
implemented very compactly, which will help with a future implementation on
massively parallel GPUs.Comment: 5 pages, 2 figures. arXiv admin note: substantial text overlap with
arXiv:1411.383
Asynchronous iterative computations with Web information retrieval structures: The PageRank case
There are several ideas being used today for Web information retrieval, and
specifically in Web search engines. The PageRank algorithm is one of those that
introduce a content-neutral ranking function over Web pages. This ranking is
applied to the set of pages returned by the Google search engine in response to
posting a search query. PageRank is based in part on two simple common sense
concepts: (i)A page is important if many important pages include links to it.
(ii)A page containing many links has reduced impact on the importance of the
pages it links to. In this paper we focus on asynchronous iterative schemes to
compute PageRank over large sets of Web pages. The elimination of the
synchronizing phases is expected to be advantageous on heterogeneous platforms.
The motivation for a possible move to such large scale distributed platforms
lies in the size of matrices representing Web structure. In orders of
magnitude: pages with nonzero elements and bytes
just to store a small percentage of the Web (the already crawled); distributed
memory machines are necessary for such computations. The present research is
part of our general objective, to explore the potential of asynchronous
computational models as an underlying framework for very large scale
computations over the Grid. The area of ``internet algorithmics'' appears to
offer many occasions for computations of unprecedent dimensionality that would
be good candidates for this framework.Comment: 8 pages to appear at ParCo2005 Conference Proceeding
Local SGD Converges Fast and Communicates Little
Mini-batch stochastic gradient descent (SGD) is state of the art in large
scale distributed training. The scheme can reach a linear speedup with respect
to the number of workers, but this is rarely seen in practice as the scheme
often suffers from large network delays and bandwidth limits. To overcome this
communication bottleneck recent works propose to reduce the communication
frequency. An algorithm of this type is local SGD that runs SGD independently
in parallel on different workers and averages the sequences only once in a
while.
This scheme shows promising results in practice, but eluded thorough
theoretical analysis. We prove concise convergence rates for local SGD on
convex problems and show that it converges at the same rate as mini-batch SGD
in terms of number of evaluated gradients, that is, the scheme achieves linear
speedup in the number of workers and mini-batch size. The number of
communication rounds can be reduced up to a factor of T^{1/2}---where T denotes
the number of total steps---compared to mini-batch SGD. This also holds for
asynchronous implementations. Local SGD can also be used for large scale
training of deep learning models.
The results shown here aim serving as a guideline to further explore the
theoretical and practical aspects of local SGD in these applications.Comment: to appear at ICLR 2019, 19 page
The Mode of Computing
The Turing Machine is the paradigmatic case of computing machines, but there
are others, such as Artificial Neural Networks, Table Computing,
Relational-Indeterminate Computing and diverse forms of analogical computing,
each of which based on a particular underlying intuition of the phenomenon of
computing. This variety can be captured in terms of system levels,
re-interpreting and generalizing Newell's hierarchy, which includes the
knowledge level at the top and the symbol level immediately below it. In this
re-interpretation the knowledge level consists of human knowledge and the
symbol level is generalized into a new level that here is called The Mode of
Computing. Natural computing performed by the brains of humans and non-human
animals with a developed enough neural system should be understood in terms of
a hierarchy of system levels too. By analogy from standard computing machinery
there must be a system level above the neural circuitry levels and directly
below the knowledge level that is named here The mode of Natural Computing. A
central question for Cognition is the characterization of this mode. The Mode
of Computing provides a novel perspective on the phenomena of computing,
interpreting, the representational and non-representational views of cognition,
and consciousness.Comment: 35 pages, 8 figure
- …