98,208 research outputs found
MOLNs: A cloud platform for interactive, reproducible and scalable spatial stochastic computational experiments in systems biology using PyURDME
Computational experiments using spatial stochastic simulations have led to
important new biological insights, but they require specialized tools, a
complex software stack, as well as large and scalable compute and data analysis
resources due to the large computational cost associated with Monte Carlo
computational workflows. The complexity of setting up and managing a
large-scale distributed computation environment to support productive and
reproducible modeling can be prohibitive for practitioners in systems biology.
This results in a barrier to the adoption of spatial stochastic simulation
tools, effectively limiting the type of biological questions addressed by
quantitative modeling. In this paper, we present PyURDME, a new, user-friendly
spatial modeling and simulation package, and MOLNs, a cloud computing appliance
for distributed simulation of stochastic reaction-diffusion models. MOLNs is
based on IPython and provides an interactive programming platform for
development of sharable and reproducible distributed parallel computational
experiments
nbodykit: an open-source, massively parallel toolkit for large-scale structure
We present nbodykit, an open-source, massively parallel Python toolkit for
analyzing large-scale structure (LSS) data. Using Python bindings of the
Message Passing Interface (MPI), we provide parallel implementations of many
commonly used algorithms in LSS. nbodykit is both an interactive and scalable
piece of scientific software, performing well in a supercomputing environment
while still taking advantage of the interactive tools provided by the Python
ecosystem. Existing functionality includes estimators of the power spectrum, 2
and 3-point correlation functions, a Friends-of-Friends grouping algorithm,
mock catalog creation via the halo occupation distribution technique, and
approximate N-body simulations via the FastPM scheme. The package also provides
a set of distributed data containers, insulated from the algorithms themselves,
that enable nbodykit to provide a unified treatment of both simulation and
observational data sets. nbodykit can be easily deployed in a high performance
computing environment, overcoming some of the traditional difficulties of using
Python on supercomputers. We provide performance benchmarks illustrating the
scalability of the software. The modular, component-based approach of nbodykit
allows researchers to easily build complex applications using its tools. The
package is extensively documented at http://nbodykit.readthedocs.io, which also
includes an interactive set of example recipes for new users to explore. As
open-source software, we hope nbodykit provides a common framework for the
community to use and develop in confronting the analysis challenges of future
LSS surveys.Comment: 18 pages, 7 figures. Feedback very welcome. Code available at
https://github.com/bccp/nbodykit and for documentation, see
http://nbodykit.readthedocs.i
Fast scalable visualization techniques for interactive billion-particle walkthrough
This research develops a comprehensive framework for interactive walkthrough involving one billion particles in an immersive virtual environment to enable interrogative visualization of large atomistic simulation data. As a mixture of scientific and engineering approaches, the framework is based on four key techniques: adaptive data compression based on space-filling curves, octree-based visibility and occlusion culling, predictive caching based on machine learning, and scalable data reduction based on parallel and distributed processing. In terms of parallel rendering, this system combines functional parallelism, data parallelism, and temporal parallelism to improve interactivity. The visualization framework will be applicable not only to material simulation, but also to computational biology, applied mathematics, mechanical engineering, and nanotechnology, etc
Developing Interactive PVM-based Parallel Programs on Distributed Computing Systems within AVS Framework
We discuss techniques in developing interactive Parallel Virtual Machine (PVM) concurrent programs on distributed systems under AVS framework. Using a stock option price modeling application as a case study, we demonstrate a simple, effective and modular approach to coupling network-based concurrent modules into an interactive remote visualization environment. A prototype simulation on-demand system is developed, in which parallel option pricing models locally implemented on two distributed systems, an Ethernet-connected IBM SP1 and a FDDI-based GIGswitch-connected DEC Alpha farm, are coupled with an interactive graphical user interface over a ATM-based wide area network. This integrated networking/visualization framework allows one to use a high level system software approach to combine AVS visualization modules with remote concurrent PVM tasks in fine-grain parallelism
Don't Repeat Yourself: Seamless Execution and Analysis of Extensive Network Experiments
This paper presents MACI, the first bespoke framework for the management, the
scalable execution, and the interactive analysis of a large number of network
experiments. Driven by the desire to avoid repetitive implementation of just a
few scripts for the execution and analysis of experiments, MACI emerged as a
generic framework for network experiments that significantly increases
efficiency and ensures reproducibility. To this end, MACI incorporates and
integrates established simulators and analysis tools to foster rapid but
systematic network experiments.
We found MACI indispensable in all phases of the research and development
process of various communication systems, such as i) an extensive DASH video
streaming study, ii) the systematic development and improvement of Multipath
TCP schedulers, and iii) research on a distributed topology graph pattern
matching algorithm. With this work, we make MACI publicly available to the
research community to advance efficient and reproducible network experiments
Developing interest management techniques in distributed interactive simulation using Java
Bandwidth consumption in distributed real time simulation, or networked real time simulation, is a major problem as the number of participants and the sophistication of joint simulation exercises grow in size. The paper briefly reviews distributed real time simulation and bandwidth reduction techniques and introduces the Generic Runtime Infrastructure for Distributed Simulation (GRIDS) as a research architecture for studying such problems. GRIDS uses Java abstract classes to promote distributed services called thin agents, a novel approach to implementing distributed simulation services, such as user defined bandwidth reduction mechanisms, and to distributing the executable code across the simulation. Thin agents offer the advantages of traditional agents without the overhead imposed by mobility or continuous state, which are unnecessary in this context. We present our implementation and some predicted results from message reduction studies using thin agent
From Big Data to Big Displays: High-Performance Visualization at Blue Brain
Blue Brain has pushed high-performance visualization (HPV) to complement its
HPC strategy since its inception in 2007. In 2011, this strategy has been
accelerated to develop innovative visualization solutions through increased
funding and strategic partnerships with other research institutions.
We present the key elements of this HPV ecosystem, which integrates C++
visualization applications with novel collaborative display systems. We
motivate how our strategy of transforming visualization engines into services
enables a variety of use cases, not only for the integration with high-fidelity
displays, but also to build service oriented architectures, to link into web
applications and to provide remote services to Python applications.Comment: ISC 2017 Visualization at Scale worksho
- …