20,507 research outputs found
Supercomputing with MPI meets the Common Workflow Language standards: an experience report
Use of standards-based workflows is still somewhat unusual by
high-performance computing users. In this paper we describe the experience of
using the Common Workflow Language (CWL) standards to describe the execution,
in parallel, of MPI-parallelised applications. In particular, we motivate and
describe the simple extension to the specification which was required, as well
as our implementation of this within the CWL reference runner. We discuss some
of the unexpected benefits, such as simple use of HPC-oriented performance
measurement tools, and CWL software requirements interfacing with HPC module
systems. We close with a request for comment from the community on how these
features could be adopted within versions of the CWL standards.Comment: Submitted to 15th Workshop on Workflows in Support of Large-Scale
Science (WORKS20
A Taxonomy of Workflow Management Systems for Grid Computing
With the advent of Grid and application technologies, scientists and
engineers are building more and more complex applications to manage and process
large data sets, and execute scientific experiments on distributed resources.
Such application scenarios require means for composing and executing complex
workflows. Therefore, many efforts have been made towards the development of
workflow management systems for Grid computing. In this paper, we propose a
taxonomy that characterizes and classifies various approaches for building and
executing workflows on Grids. We also survey several representative Grid
workflow systems developed by various projects world-wide to demonstrate the
comprehensiveness of the taxonomy. The taxonomy not only highlights the design
and engineering similarities and differences of state-of-the-art in Grid
workflow systems, but also identifies the areas that need further research.Comment: 29 pages, 15 figure
- âŠ