60,139 research outputs found
Developing High Performance Computing Resources for Teaching Cluster and Grid Computing courses
High-Performance Computing (HPC) and the ability to process large amounts of data are of
paramount importance for UK business and economy as outlined by Rt Hon David Willetts
MP at the HPC and Big Data conference in February 2014. However there is a shortage of
skills and available training in HPC to prepare and expand the workforce for the HPC and
Big Data research and development. Currently, HPC skills are acquired mainly by students
and staff taking part in HPC-related research projects, MSc courses, and at the dedicated
training centres such as Edinburgh University’s EPCC. There are few UK universities teaching
the HPC, Clusters and Grid Computing courses at the undergraduate level. To address the
issue of skills shortages in the HPC it is essential to provide teaching and training as part of
both postgraduate and undergraduate courses. The design and development of such courses is
challenging since the technologies and software in the fields of large scale distributed systems
such as Cluster, Cloud and Grid computing are undergoing continuous change. The students
completing the HPC courses should be proficient in these evolving technologies and equipped
with practical and theoretical skills for future jobs in this fast developing area.
In this paper we present our experience in developing the HPC, Cluster and Grid modules
including a review of existing HPC courses offered at the UK universities. The topics covered in
the modules are described, as well as the coursework projects based on practical laboratory work.
We conclude with an evaluation based on our experience over the last ten years in developing
and delivering the HPC modules on the undergraduate courses, with suggestions for future work
Distributed N-body Simulation on the Grid Using Dedicated Hardware
We present performance measurements of direct gravitational N -body
simulation on the grid, with and without specialized (GRAPE-6) hardware. Our
inter-continental virtual organization consists of three sites, one in Tokyo,
one in Philadelphia and one in Amsterdam. We run simulations with up to 196608
particles for a variety of topologies. In many cases, high performance
simulations over the entire planet are dominated by network bandwidth rather
than latency. With this global grid of GRAPEs our calculation time remains
dominated by communication over the entire range of N, which was limited due to
the use of three sites. Increasing the number of particles will result in a
more efficient execution. Based on these timings we construct and calibrate a
model to predict the performance of our simulation on any grid infrastructure
with or without GRAPE. We apply this model to predict the simulation
performance on the Netherlands DAS-3 wide area computer. Equipping the DAS-3
with GRAPE-6Af hardware would achieve break-even between calculation and
communication at a few million particles, resulting in a compute time of just
over ten hours for 1 N -body time unit. Key words: high-performance computing,
grid, N-body simulation, performance modellingComment: (in press) New Astronomy, 24 pages, 5 figure
Recent development and perspectives of machines for lattice QCD
I highlight recent progress in cluster computer technology and assess status
and prospects of cluster computers for lattice QCD with respect to the
development of QCDOC and apeNEXT. Taking the LatFor test case, I specify a
512-processor QCD-cluster better than 1$/Mflops.Comment: 14 pages, 17 figures, Lattice2003(plenary
New distributed offline processing scheme at Belle
The offline processing of the data collected by the Belle detector has been
recently upgraded to cope with the excellent performance of the KEKB
accelerator. The 127/fb of data (120 TB on tape) collected between autumn 2003
and summer 2004 has been processed in 2 months, thanks to the high speed and
stability of the new, distributed processing scheme. We present here this new
processing scheme and its performance.Comment: 4 pages, 8 figures, uses CHEP2004.cl
A batch scheduler with high level components
In this article we present the design choices and the evaluation of a batch
scheduler for large clusters, named OAR. This batch scheduler is based upon an
original design that emphasizes on low software complexity by using high level
tools. The global architecture is built upon the scripting language Perl and
the relational database engine Mysql. The goal of the project OAR is to prove
that it is possible today to build a complex system for ressource management
using such tools without sacrificing efficiency and scalability. Currently, our
system offers most of the important features implemented by other batch
schedulers such as priority scheduling (by queues), reservations, backfilling
and some global computing support. Despite the use of high level tools, our
experiments show that our system has performances close to other systems.
Furthermore, OAR is currently exploited for the management of 700 nodes (a
metropolitan GRID) and has shown good efficiency and robustness
- …