132 research outputs found
VisIVO - Integrated Tools and Services for Large-Scale Astrophysical Visualization
VisIVO is an integrated suite of tools and services specifically designed for
the Virtual Observatory. This suite constitutes a software framework for
effective visual discovery in currently available (and next-generation) very
large-scale astrophysical datasets. VisIVO consists of VisiVO Desktop - a stand
alone application for interactive visualization on standard PCs, VisIVO Server
- a grid-enabled platform for high performance visualization and VisIVO Web - a
custom designed web portal supporting services based on the VisIVO Server
functionality. The main characteristic of VisIVO is support for
high-performance, multidimensional visualization of very large-scale
astrophysical datasets. Users can obtain meaningful visualizations rapidly
while preserving full and intuitive control of the relevant visualization
parameters. This paper focuses on newly developed integrated tools in VisIVO
Server allowing intuitive visual discovery with 3D views being created from
data tables. VisIVO Server can be installed easily on any web server with a
database repository. We discuss briefly aspects of our implementation of VisiVO
Server on a computational grid and also outline the functionality of the
services offered by VisIVO Web. Finally we conclude with a summary of our work
and pointers to future developments
scenery: Flexible Virtual Reality Visualization on the Java VM
Life science today involves computational analysis of a large amount and
variety of data, such as volumetric data acquired by state-of-the-art
microscopes, or mesh data from analysis of such data or simulations.
Visualization is often the first step in making sense of data, and a crucial
part of building and debugging analysis pipelines. It is therefore important
that visualizations can be quickly prototyped, as well as developed or embedded
into full applications. In order to better judge spatiotemporal relationships,
immersive hardware, such as Virtual or Augmented Reality (VR/AR) headsets and
associated controllers are becoming invaluable tools. In this work we introduce
scenery, a flexible VR/AR visualization framework for the Java VM that can
handle mesh and large volumetric data, containing multiple views, timepoints,
and color channels. scenery is free and open-source software, works on all
major platforms, and uses the Vulkan or OpenGL rendering APIs. We introduce
scenery's main features and example applications, such as its use in VR for
microscopy, in the biomedical image analysis software Fiji, or for visualizing
agent-based simulations.Comment: Added IEEE DOI, version published at VIS 201
Diva: A Declarative and Reactive Language for In-Situ Visualization
The use of adaptive workflow management for in situ visualization and
analysis has been a growing trend in large-scale scientific simulations.
However, coordinating adaptive workflows with traditional procedural
programming languages can be difficult because system flow is determined by
unpredictable scientific phenomena, which often appear in an unknown order and
can evade event handling. This makes the implementation of adaptive workflows
tedious and error-prone. Recently, reactive and declarative programming
paradigms have been recognized as well-suited solutions to similar problems in
other domains. However, there is a dearth of research on adapting these
approaches to in situ visualization and analysis. With this paper, we present a
language design and runtime system for developing adaptive systems through a
declarative and reactive programming paradigm. We illustrate how an adaptive
workflow programming system is implemented using our approach and demonstrate
it with a use case from a combustion simulation.Comment: 11 pages, 5 figures, 6 listings, 1 table, to be published in LDAV
2020. The article has gone through 2 major revisions: Emphasized
contributions, features and examples. Addressed connections between DIVA and
FRP. In sec. 3, we fixed a design flaw and addressed it in sec. 3.3-3.4.
Re-designed sec. 5 with a more concrete example and benchmark results.
Simplified the syntax of DIV
General Purpose Flow Visualization at the Exascale
Exascale computing, i.e., supercomputers that can perform 1018 math operations per second, provide significant opportunity for improving the computational sciences. That said, these machines can be difficult to use efficiently, due to their massive parallelism, due to the use of accelerators, and due to the diversity of accelerators used. All areas of the computational science stack need to be reconsidered to address these problems. With this dissertation, we consider flow visualization, which is critical for analyzing vector field data from simulations. We specifically consider flow visualization techniques that use particle advection, i.e., tracing particle trajectories, which presents performance and implementation challenges. The dissertation makes four primary contributions. First, it synthesizes previous work on particle advection performance and introduces a high-level analytical cost model. Second, it proposes an approach for performance portability across accelerators. Third, it studies expected speedups based on using accelerators, including the importance of factors such as duration, particle count, data set, and others. Finally, it proposes an exascale-capable particle advection system that addresses diversity in many dimensions, including accelerator type, parallelism approach, analysis use case, underlying vector field, and more
Recommended from our members
A Programmable Streaming Framework for Extreme-Scale Scientific Visualizations
Emerging computational and acquisition technologies are empowering scientists to conduct simulations and experiments on an unprecedented scale. These advancements can push the frontiers of science and technology with groundbreaking discoveries. However, they also pose significant challenges to traditional scientific visualization workflows. Firstly, the data generated by modern scientific studies using these technologies tends to be extremely large and complex, often resulting in slow processing and rendering times. This demands the development of visualization algorithms that can effectively scale with the size of the data. Secondly, state-of-the-art simulations and experiments produce data at extraordinary rates, complicating the task of generating valuable visualization results for scientists. Therefore, there's a pressing need for more adaptive and intelligent visualization workflows. Lastly, although new computer hardware and architecture can speed up the visualization process, significant performance variations still exist among visualization algorithms due to differing design choices. As a result, optimizing algorithms to better leverage emerging hardware features for enhanced efficiency remains an ongoing necessity.This dissertation addresses the aforementioned challenges by introducing a programmable streaming framework enhanced with implicit neural representation, designed for visualizing extreme-scale scientific data. Specifically, it unfolds three innovative methodologies:Firstly, the framework offers a reactive and declarative programming language for streamlining image generation, layout and interaction creation, and I/O processes, eliminating the need for users to manually control all visualization parameters and procedures. This language enables scientists to define highly adaptive visualization workflows through high-level, rule-based grammars. The system then automatically optimizes the low-level implementation according to these specifications, facilitating the creation of more efficient visualization workflows with simpler coding.Secondly, the framework features a scalable, hardware-accelerated streaming visualization system that allows visualization processes to run concurrently with I/O operations. This system not only achieves state-of-the-art scalability but can also effectively manages complex, multi-resolution data structures. It delivers accurate rendering outcomes, reduces memory usage, and leverages emerging hardware capabilities more efficiently.Finally, the framework integrates implicit neural representation (INR) techniques for data compression and interactive visualization. The use of INRs significantly reduces data size while preserving high-frequency details. Additionally, it enables direct access to spatial locations at any desired resolution, obviating the need for decompression or interpolation.In summary, this dissertation research addresses long-standing challenges inherent in extreme-scale scientific visualization by introducing novel designs and methodologies. The presented framework not only enables more efficient and adaptive visualization workflows but also leverages the latest hardware acceleration and data compression techniques. The implications of these advancements extend beyond mere technical improvements; they pave the way for deeper insights and discoveries across a broad spectrum of scientific studies. This research, therefore, represents a significant leap forward, with the potential to transform the landscape of scientific visualization
Distributed Parallel Extreme Event Analysis in Next Generation Simulation Architectures
Numerical simulations present challenges as they reach exascale because they generate petabyte-scale data that cannot be saved without interrupting the simulation
due to I/O constraints. Data scientists must be able to reduce, extract, and visualize the data while the simulation is running, which is essential for in transit and
post analysis. Next generation architectures in supercomputing include a burst buļ¬er
technology composed of SSDs primarily for the use of checkpointing the simulation
in case a restart is required. In the case of turbulence simulations, this checkpoint
provides an opportunity to perform analysis on the data without interrupting the
simulation.
First, we present a method of extracting velocity data in high vorticity regions.
This method requires calculating the vorticity of the entire dataset and identifying
regions where the threshold is above a speciļ¬ed value. Next we create a 3D stencil
from values above the threshold and dilate the stencil. Finally we use the stencil to
extract velocity data from the original dataset. The result is a dataset that is over
an order of magnitude smaller and contains all the data required to study extreme events and visualization of vorticity.
The next extraction utilizes the zfp lossy compressor to compress the entire velocity dataset. The compressed representation results in a dataset an order of magnitude
smaller than the raw simulation data. This provides the researcher approximate data
not captured by the velocity extraction. The error introduced is bounded, and results in a dataset that is visually indistinguishable from the original dataset.
Finally we present a modular distributed parallel extraction system. This system allows a data scientist to run the previously mentioned extraction algorithms in a
distributed parallel cluster of burst buļ¬er nodes. The extraction algorithms are built
as modules for the system and run in parallel on burst buļ¬er nodes. A feature extraction coordinator synchronizes the simulation with the extraction process. A data
scientist only needs to write one module that performs the extraction or visualization
on a single subset of data and the system will execute that module at scale on burst
buļ¬ers, managing all the communication, synchronization, and parallelism required
to perform the analysis
Steering in computational science: mesoscale modelling and simulation
This paper outlines the benefits of computational steering for high
performance computing applications. Lattice-Boltzmann mesoscale fluid
simulations of binary and ternary amphiphilic fluids in two and three
dimensions are used to illustrate the substantial improvements which
computational steering offers in terms of resource efficiency and time to
discover new physics. We discuss details of our current steering
implementations and describe their future outlook with the advent of
computational grids.Comment: 40 pages, 11 figures. Accepted for publication in Contemporary
Physic
Doctor of Philosophy
dissertationDataflow pipeline models are widely used in visualization systems. Despite recent advancements in parallel architecture, most systems still support only a single CPU or a small collection of CPUs such as a SMP workstation. Even for systems that are specifically tuned towards parallel visualization, their execution models only provide support for data-parallelism while ignoring taskparallelism and pipeline-parallelism. With the recent popularization of machines equipped with multicore CPUs and multi-GPU units, these visualization systems are undoubtedly falling further behind in reaching maximum efficiency. On the other hand, there exist several libraries that can schedule program executions on multiple CPUs and/or multiple GPUs. However, due to differences in executing a task graph and a pipeline along with their APIs being considerably low-level, it still remains a challenge to integrate these run-time libraries into current visualization systems. Thus, there is a need for a redesigned dataflow architecture to fully support and exploit the power of highly parallel machines in large-scale visualization. The new design must be able to schedule executions on heterogeneous platforms while at the same time supporting arbitrarily large datasets through the use of streaming data structures. The primary goal of this dissertation work is to develop a parallel dataflow architecture for streaming large-scale visualizations. The framework includes supports for platforms ranging from multicore processors to clusters consisting of thousands CPUs and GPUs. We achieve this in our system by introducing the notion of Virtual Processing Elements and Task-Oriented Modules along with a highly customizable scheduler that controls the assignment of tasks to elements dynamically. This creates an intuitive way to maintain multiple CPU/GPU kernels yet still provide coherency and synchronization across module executions. We have implemented these techniques into HyperFlow which is made of an API with all basic dataflow constructs described in the dissertation, and a distributed run-time library that can be used to deploy those pipelines on multicore, multi-GPU and cluster-based platforms
- ā¦