9,025 research outputs found
Using Java for distributed computing in the Gaia satellite data processing
In recent years Java has matured to a stable easy-to-use language with the
flexibility of an interpreter (for reflection etc.) but the performance and
type checking of a compiled language. When we started using Java for
astronomical applications around 1999 they were the first of their kind in
astronomy. Now a great deal of astronomy software is written in Java as are
many business applications.
We discuss the current environment and trends concerning the language and
present an actual example of scientific use of Java for high-performance
distributed computing: ESA's mission Gaia. The Gaia scanning satellite will
perform a galactic census of about 1000 million objects in our galaxy. The Gaia
community has chosen to write its processing software in Java. We explore the
manifold reasons for choosing Java for this large science collaboration.
Gaia processing is numerically complex but highly distributable, some parts
being embarrassingly parallel. We describe the Gaia processing architecture and
its realisation in Java. We delve into the astrometric solution which is the
most advanced and most complex part of the processing. The Gaia simulator is
also written in Java and is the most mature code in the system. This has been
successfully running since about 2005 on the supercomputer "Marenostrum" in
Barcelona. We relate experiences of using Java on a large shared machine.
Finally we discuss Java, including some of its problems, for scientific
computing.Comment: Experimental Astronomy, August 201
A Distributed Economics-based Infrastructure for Utility Computing
Existing attempts at utility computing revolve around two approaches. The
first consists of proprietary solutions involving renting time on dedicated
utility computing machines. The second requires the use of heavy, monolithic
applications that are difficult to deploy, maintain, and use.
We propose a distributed, community-oriented approach to utility computing.
Our approach provides an infrastructure built on Web Services in which modular
components are combined to create a seemingly simple, yet powerful system. The
community-oriented nature generates an economic environment which results in
fair transactions between consumers and providers of computing cycles while
simultaneously encouraging improvements in the infrastructure of the
computational grid itself.Comment: 8 pages, 1 figur
Unleashing the Power of Distributed CPU/GPU Architectures: Massive Astronomical Data Analysis and Visualization case study
Upcoming and future astronomy research facilities will systematically
generate terabyte-sized data sets moving astronomy into the Petascale data era.
While such facilities will provide astronomers with unprecedented levels of
accuracy and coverage, the increases in dataset size and dimensionality will
pose serious computational challenges for many current astronomy data analysis
and visualization tools. With such data sizes, even simple data analysis tasks
(e.g. calculating a histogram or computing data minimum/maximum) may not be
achievable without access to a supercomputing facility.
To effectively handle such dataset sizes, which exceed today's single machine
memory and processing limits, we present a framework that exploits the
distributed power of GPUs and many-core CPUs, with a goal of providing data
analysis and visualizing tasks as a service for astronomers. By mixing shared
and distributed memory architectures, our framework effectively utilizes the
underlying hardware infrastructure handling both batched and real-time data
analysis and visualization tasks. Offering such functionality as a service in a
"software as a service" manner will reduce the total cost of ownership, provide
an easy to use tool to the wider astronomical community, and enable a more
optimized utilization of the underlying hardware infrastructure.Comment: 4 Pages, 1 figures, To appear in the proceedings of ADASS XXI, ed.
P.Ballester and D.Egret, ASP Conf. Serie
Improving the scalability of parallel N-body applications with an event driven constraint based execution model
The scalability and efficiency of graph applications are significantly
constrained by conventional systems and their supporting programming models.
Technology trends like multicore, manycore, and heterogeneous system
architectures are introducing further challenges and possibilities for emerging
application domains such as graph applications. This paper explores the space
of effective parallel execution of ephemeral graphs that are dynamically
generated using the Barnes-Hut algorithm to exemplify dynamic workloads. The
workloads are expressed using the semantics of an Exascale computing execution
model called ParalleX. For comparison, results using conventional execution
model semantics are also presented. We find improved load balancing during
runtime and automatic parallelism discovery improving efficiency using the
advanced semantics for Exascale computing.Comment: 11 figure
Devito: Towards a generic Finite Difference DSL using Symbolic Python
Domain specific languages (DSL) have been used in a variety of fields to
express complex scientific problems in a concise manner and provide automated
performance optimization for a range of computational architectures. As such
DSLs provide a powerful mechanism to speed up scientific Python computation
that goes beyond traditional vectorization and pre-compilation approaches,
while allowing domain scientists to build applications within the comforts of
the Python software ecosystem. In this paper we present Devito, a new finite
difference DSL that provides optimized stencil computation from high-level
problem specifications based on symbolic Python expressions. We demonstrate
Devito's symbolic API and performance advantages over traditional Python
acceleration methods before highlighting its use in the scientific context of
seismic inversion problems.Comment: pyHPC 2016 conference submissio
Large-Scale Simulations of Clusters of Galaxies
We discuss some of the computational challenges encountered in simulating the
evolution of clusters of galaxies. Eulerian adaptive mesh refinement (AMR)
techniques can successfully address these challenges but are currently being
used by only a few groups. We describe our publicly available AMR code, FLASH,
which uses an object-oriented framework to manage its AMR library, physics
modules, and automated verification. We outline the development of the FLASH
framework to include collisionless particles, permitting it to be used for
cluster simulation.Comment: 3 pages, 3 figures, to appear in Proceedings of the VII International
Workshop on Advanced Computing and Analysis Techniques in Physics Research
(ACAT 2000), Fermilab, Oct. 16-20, 200
- …