1,253 research outputs found
Enhancing Workflow with a Semantic Description of Scientific Intent
Peer reviewedPreprin
Data Workflow - A Workflow Model for Continuous Data Processing
Online data or streaming data are getting more and more important for enterprise information systems, e.g. by integrating sensor data and workflows. The continuous flow of data provided e.g. by sensors requires new workflow models addressing the data perspective of these applications, since continuous data is potentially infinite while business process instances are always finite.\ud
In this paper a formal workflow model is proposed with data driven coordination and explicating properties of the continuous data processing. These properties can be used to optimize data workflows, i.e., reducing the computational power for processing the workflows in an engine by reusing intermediate processing results in several workflows
Automated data reduction workflows for astronomy
Data from complex modern astronomical instruments often consist of a large
number of different science and calibration files, and their reduction requires
a variety of software tools. The execution chain of the tools represents a
complex workflow that needs to be tuned and supervised, often by individual
researchers that are not necessarily experts for any specific instrument. The
efficiency of data reduction can be improved by using automatic workflows to
organise data and execute the sequence of data reduction steps. To realize such
efficiency gains, we designed a system that allows intuitive representation,
execution and modification of the data reduction workflow, and has facilities
for inspection and interaction with the data. The European Southern Observatory
(ESO) has developed Reflex, an environment to automate data reduction
workflows. Reflex is implemented as a package of customized components for the
Kepler workflow engine. Kepler provides the graphical user interface to create
an executable flowchart-like representation of the data reduction process. Key
features of Reflex are a rule-based data organiser, infrastructure to re-use
results, thorough book-keeping, data progeny tracking, interactive user
interfaces, and a novel concept to exploit information created during data
organisation for the workflow execution. Reflex includes novel concepts to
increase the efficiency of astronomical data processing. While Reflex is a
specific implementation of astronomical scientific workflows within the Kepler
workflow engine, the overall design choices and methods can also be applied to
other environments for running automated science workflows.Comment: 12 pages, 7 figure
- …