123,953 research outputs found
The PROOF Distributed Parallel Analysis Framework based on ROOT
The development of the Parallel ROOT Facility, PROOF, enables a physicist to
analyze and understand much larger data sets on a shorter time scale. It makes
use of the inherent parallelism in event data and implements an architecture
that optimizes I/O and CPU utilization in heterogeneous clusters with
distributed storage. The system provides transparent and interactive access to
gigabytes today. Being part of the ROOT framework PROOF inherits the benefits
of a performant object storage system and a wealth of statistical and
visualization tools. This paper describes the key principles of the PROOF
architecture and the implementation of the system. We will illustrate its
features using a simple example and present measurements of the scalability of
the system. Finally we will discuss how PROOF can be interfaced and make use of
the different Grid solutions.Comment: Talk from the 2003 Computing in High Energy and Nuclear Physics
(CHEP03), La Jolla, CA, USA, March 2003, 5 pages, LaTeX, 4 eps figures. PSN
TULT00
Recommended from our members
A Distributed, Component-based Solution for Scientific Information Management
This paper describes the design and performance of a distributed, multi-tier architecture for scientific information management and data exploration. A novel aspect of this framework is its integration of Java IDL, the CORBA distributed object computing middleware with JavaBeans, the Java Component model to provide a flexible, interactive framework for distributed, high-performance scientific data visualization. CORBA server objects running in a distributed, collaborative environment provide data acquisition and perform data-intensive computations. Clients as Java Bean components use these server objects for data retrieval and provide an interactive environment for visualization. The server objects use JDBC, the Java application programming interface to databases, to retrieve data from the distributed data stores. We discuss the system framework and its components and describe an example application and its performance
Interactive Visualization of the Largest Radioastronomy Cubes
3D visualization is an important data analysis and knowledge discovery tool,
however, interactive visualization of large 3D astronomical datasets poses a
challenge for many existing data visualization packages. We present a solution
to interactively visualize larger-than-memory 3D astronomical data cubes by
utilizing a heterogeneous cluster of CPUs and GPUs. The system partitions the
data volume into smaller sub-volumes that are distributed over the rendering
workstations. A GPU-based ray casting volume rendering is performed to generate
images for each sub-volume, which are composited to generate the whole volume
output, and returned to the user. Datasets including the HI Parkes All Sky
Survey (HIPASS - 12 GB) southern sky and the Galactic All Sky Survey (GASS - 26
GB) data cubes were used to demonstrate our framework's performance. The
framework can render the GASS data cube with a maximum render time < 0.3 second
with 1024 x 1024 pixels output resolution using 3 rendering workstations and 8
GPUs. Our framework will scale to visualize larger datasets, even of Terabyte
order, if proper hardware infrastructure is available.Comment: 15 pages, 12 figures, Accepted New Astronomy July 201
Unleashing the Power of Distributed CPU/GPU Architectures: Massive Astronomical Data Analysis and Visualization case study
Upcoming and future astronomy research facilities will systematically
generate terabyte-sized data sets moving astronomy into the Petascale data era.
While such facilities will provide astronomers with unprecedented levels of
accuracy and coverage, the increases in dataset size and dimensionality will
pose serious computational challenges for many current astronomy data analysis
and visualization tools. With such data sizes, even simple data analysis tasks
(e.g. calculating a histogram or computing data minimum/maximum) may not be
achievable without access to a supercomputing facility.
To effectively handle such dataset sizes, which exceed today's single machine
memory and processing limits, we present a framework that exploits the
distributed power of GPUs and many-core CPUs, with a goal of providing data
analysis and visualizing tasks as a service for astronomers. By mixing shared
and distributed memory architectures, our framework effectively utilizes the
underlying hardware infrastructure handling both batched and real-time data
analysis and visualization tasks. Offering such functionality as a service in a
"software as a service" manner will reduce the total cost of ownership, provide
an easy to use tool to the wider astronomical community, and enable a more
optimized utilization of the underlying hardware infrastructure.Comment: 4 Pages, 1 figures, To appear in the proceedings of ADASS XXI, ed.
P.Ballester and D.Egret, ASP Conf. Serie
GPU-Based Volume Rendering of Noisy Multi-Spectral Astronomical Data
Traditional analysis techniques may not be sufficient for astronomers to make
the best use of the data sets that current and future instruments, such as the
Square Kilometre Array and its Pathfinders, will produce. By utilizing the
incredible pattern-recognition ability of the human mind, scientific
visualization provides an excellent opportunity for astronomers to gain
valuable new insight and understanding of their data, particularly when used
interactively in 3D. The goal of our work is to establish the feasibility of a
real-time 3D monitoring system for data going into the Australian SKA
Pathfinder archive.
Based on CUDA, an increasingly popular development tool, our work utilizes
the massively parallel architecture of modern graphics processing units (GPUs)
to provide astronomers with an interactive 3D volume rendering for
multi-spectral data sets. Unlike other approaches, we are targeting real time
interactive visualization of datasets larger than GPU memory while giving
special attention to data with low signal to noise ratio - two critical aspects
for astronomy that are missing from most existing scientific visualization
software packages. Our framework enables the astronomer to interact with the
geometrical representation of the data, and to control the volume rendering
process to generate a better representation of their datasets.Comment: 4 pages, 1 figure, to appear in the proceedings of ADASS XIX, Oct 4-8
2009, Sapporo, Japan (ASP Conf. Series
- …