83,582 research outputs found
PaPaS: A Portable, Lightweight, and Generic Framework for Parallel Parameter Studies
The current landscape of scientific research is widely based on modeling and
simulation, typically with complexity in the simulation's flow of execution and
parameterization properties. Execution flows are not necessarily
straightforward since they may need multiple processing tasks and iterations.
Furthermore, parameter and performance studies are common approaches used to
characterize a simulation, often requiring traversal of a large parameter
space. High-performance computers offer practical resources at the expense of
users handling the setup, submission, and management of jobs. This work
presents the design of PaPaS, a portable, lightweight, and generic workflow
framework for conducting parallel parameter and performance studies. Workflows
are defined using parameter files based on keyword-value pairs syntax, thus
removing from the user the overhead of creating complex scripts to manage the
workflow. A parameter set consists of any combination of environment variables,
files, partial file contents, and command line arguments. PaPaS is being
developed in Python 3 with support for distributed parallelization using SSH,
batch systems, and C++ MPI. The PaPaS framework will run as user processes, and
can be used in single/multi-node and multi-tenant computing systems. An example
simulation using the BehaviorSpace tool from NetLogo and a matrix multiply
using OpenMP are presented as parameter and performance studies, respectively.
The results demonstrate that the PaPaS framework offers a simple method for
defining and managing parameter studies, while increasing resource utilization.Comment: 8 pages, 6 figures, PEARC '18: Practice and Experience in Advanced
Research Computing, July 22--26, 2018, Pittsburgh, PA, US
Technical support for Life Sciences communities on a production grid infrastructure
Production operation of large distributed computing infrastructures (DCI)
still requires a lot of human intervention to reach acceptable quality of
service. This may be achievable for scientific communities with solid IT
support, but it remains a show-stopper for others. Some application execution
environments are used to hide runtime technical issues from end users. But they
mostly aim at fault-tolerance rather than incident resolution, and their
operation still requires substantial manpower. A longer-term support activity
is thus needed to ensure sustained quality of service for Virtual Organisations
(VO). This paper describes how the biomed VO has addressed this challenge by
setting up a technical support team. Its organisation, tooling, daily tasks,
and procedures are described. Results are shown in terms of resource usage by
end users, amount of reported incidents, and developed software tools. Based on
our experience, we suggest ways to measure the impact of the technical support,
perspectives to decrease its human cost and make it more community-specific.Comment: HealthGrid'12, Amsterdam : Netherlands (2012
Advances in the Design and Implementation of a Multi-Tier Architecture in the GIPSY Environment
We present advances in the software engineering design and implementation of
the multi-tier run-time system for the General Intensional Programming System
(GIPSY) by further unifying the distributed technologies used to implement the
Demand Migration Framework (DMF) in order to streamline distributed execution
of hybrid intensional-imperative programs using Java.Comment: 11 pages, 3 figure
- …