142,284 research outputs found
Enhancing R with Advanced Compilation Tools and Methods
I describe an approach to compiling common idioms in R code directly to
native machine code and illustrate it with several examples. Not only can this
yield significant performance gains, but it allows us to use new approaches to
computing in R. Importantly, the compilation requires no changes to R itself,
but is done entirely via R packages. This allows others to experiment with
different compilation strategies and even to define new domain-specific
languages within R. We use the Low-Level Virtual Machine (LLVM) compiler
toolkit to create the native code and perform sophisticated optimizations on
the code. By adopting this widely used software within R, we leverage its
ability to generate code for different platforms such as CPUs and GPUs, and
will continue to benefit from its ongoing development. This approach
potentially allows us to develop high-level R code that is also fast, that can
be compiled to work with different data representations and sources, and that
could even be run outside of R. The approach aims to both provide a compiler
for a limited subset of the R language and also to enable R programmers to
write other compilers. This is another approach to help us write high-level
descriptions of what we want to compute, not how.Comment: Published in at http://dx.doi.org/10.1214/13-STS462 the Statistical
Science (http://www.imstat.org/sts/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Content-access QoS in peer-to-peer networks using a fast MDS erasure code
This paper describes an enhancement of content access Quality of Service in peer to peer (P2P) networks. The main idea is to use an erasure code to distribute the information over the peers. This distribution increases the users’ choice on disseminated encoded data and therefore statistically enhances the overall throughput of the transfer. A performance evaluation based on an original model using the results of a measurement campaign of sequential and parallel downloads in a real P2P network over Internet is presented. Based on a bandwidth distribution, statistical content-access QoS are guaranteed in function of both the content replication level in the network and the file dissemination strategies. A simple application in the context of media streaming is proposed. Finally, the constraints on the erasure code related to the proposed system are analysed and a new fast MDS erasure code is proposed, implemented and evaluated
Tracing the Connections Between MPI-IO Calls and their Corresponding PVFS2 Disk Operations
Fast and efficient input/output is a very important aspect of today’s high performance computing. Often these systems are very complex so no prediction can be made what an I/O call at the program level will trigger at the system level. One kind of such complex systems are parallel programs based on MPICH2 for message passing and PVFS2 as parallel file system. This thesis is about enhancing the environment in a way that we can find the necessary information in the traces of a program run. MPI-IO calls can now be related to disk operations. The concept of our extension of the involved software packages is to generate a unique ID for every MPI-IO call and to pass it along to the I/O-servers. After they got written into the corresponding traces we can find matching events in them and visualize them in the Jumpshot trace viewer. An easy to follow example is presented and detailed hints for future work are given
Design and Evaluation of a Collective IO Model for Loosely Coupled Petascale Programming
Loosely coupled programming is a powerful paradigm for rapidly creating
higher-level applications from scientific programs on petascale systems,
typically using scripting languages. This paradigm is a form of many-task
computing (MTC) which focuses on the passing of data between programs as
ordinary files rather than messages. While it has the significant benefits of
decoupling producer and consumer and allowing existing application programs to
be executed in parallel with no recoding, its typical implementation using
shared file systems places a high performance burden on the overall system and
on the user who will analyze and consume the downstream data. Previous efforts
have achieved great speedups with loosely coupled programs, but have done so
with careful manual tuning of all shared file system access. In this work, we
evaluate a prototype collective IO model for file-based MTC. The model enables
efficient and easy distribution of input data files to computing nodes and
gathering of output results from them. It eliminates the need for such manual
tuning and makes the programming of large-scale clusters using a loosely
coupled model easier. Our approach, inspired by in-memory approaches to
collective operations for parallel programming, builds on fast local file
systems to provide high-speed local file caches for parallel scripts, uses a
broadcast approach to handle distribution of common input data, and uses
efficient scatter/gather and caching techniques for input and output. We
describe the design of the prototype model, its implementation on the Blue
Gene/P supercomputer, and present preliminary measurements of its performance
on synthetic benchmarks and on a large-scale molecular dynamics application.Comment: IEEE Many-Task Computing on Grids and Supercomputers (MTAGS08) 200
B+-tree Index Optimization by Exploiting Internal Parallelism of Flash-based Solid State Drives
Previous research addressed the potential problems of the hard-disk oriented
design of DBMSs of flashSSDs. In this paper, we focus on exploiting potential
benefits of flashSSDs. First, we examine the internal parallelism issues of
flashSSDs by conducting benchmarks to various flashSSDs. Then, we suggest
algorithm-design principles in order to best benefit from the internal
parallelism. We present a new I/O request concept, called psync I/O that can
exploit the internal parallelism of flashSSDs in a single process. Based on
these ideas, we introduce B+-tree optimization methods in order to utilize
internal parallelism. By integrating the results of these methods, we present a
B+-tree variant, PIO B-tree. We confirmed that each optimization method
substantially enhances the index performance. Consequently, PIO B-tree enhanced
B+-tree's insert performance by a factor of up to 16.3, while improving
point-search performance by a factor of 1.2. The range search of PIO B-tree was
up to 5 times faster than that of the B+-tree. Moreover, PIO B-tree
outperformed other flash-aware indexes in various synthetic workloads. We also
confirmed that PIO B-tree outperforms B+-tree in index traces collected inside
the Postgresql DBMS with TPC-C benchmark.Comment: VLDB201
- …