Benchmarking Workflow Discovery: A Case Study From Bioinformatics

Automation in science is increasingly marked by the use of workflow technology. The sharing of workflows through repositories supports the verifability, reproducibility and extensibility of computational experiments. However, the subsequent discovery of workflows remains a challenge, both from a sociological and technological viewpoint. Based on a survey with participants from 19 laboratories, we investigate current practices in workflow sharing, re-use and discovery amongst life scientists chiefly using the Taverna workflow management system. To address their perceived lack of effective workflow discovery tools, we go on to develop benchmarks for the evaluation of discovery tools, drawing on a series of practical exercises. We demonstrate the value of the benchmarks on two tools: one using graph matching, the other relying on text clustering

Workflow discovery: Requirements from e-science and a graph-based solution

Much has been written on the promise of Web service discovery and (semi-) automated composition. In this discussion, the value to practitioners of discovering and reusing existing service compositions, captured in workflows, is mostly ignored. We present the case for workflows and workflow discovery in science and develop one discovery solution. Through a survey with 21 scientists and developers from the myGrid/Taverna workflow environment, workflow discovery requirements are elicited. Through a user experiment with 13 scientists, an attempt is made to build a benchmark for workflow ranking. Through the design and implementation of a workflow discovery tool, a mechanism for ranking workflow fragments is provided based on graph sub-isomorphism detection. The tool evaluation, drawing on a corpus of 89 public workflows and the results of the user experiment, finds that, for a simple showcase, the average human ranking can largely be reproduced.</p

Discovering Scientific Workflows: The myExperiment Benchmarks

Automation in science is increasingly marked by the use of workflow technology. The sharing of workflows through publication mechanisms or repositories supports the verifiability, reproducibility and extensibility of computational experiments. However, the subsequent discovery of workflows remains a challenge, both from a technological and sociological viewpoint. We investigate current practices in workflow sharing, re-use and discovery amongst life scientists chiefly using the Taverna workflow management system. The study draws on two key sources: (i) a survey of researchers drawn from 19 research labs and (ii) an analysis of scientists’ behaviour on the myExperiment social network site, designed to encourage workflow exchange. The results reveal a multi-modal approach to workflow discovery, based on a mix of search on the content of the workflow and its situated context. We go on to develop a benchmark specifically for the evaluation of workflow discovery and to demonstrate it on two example approaches

Applying DLs to workflow reuse and repurposing

allowing a scientist to describe and enact their experimental processes in a structured, repeatable and verifiable way. The my Grid (www.mygrid.org.uk) workbench, a set of components to build workflows in bioinformatics, currently allows access to a thousand globally distributed services and a hundred workflows, some of which orchestrate up to fifty services. Figure 2 shows the example of a my Grid workflow which gathers information about genetic sequences in support of research on Williams-Beuren syndrome [10]. Much of the research geared towards the construction of on-line processes (i.e. workflows) is led by a vision of automatic composition of services based on extensive formalisation (see for example www.daml.org/services/owl-s/pubarchive.html). Such research can be complemented with techniques that exploit those cases where existing workflows and fragments of workflows can be reused, thereby benefitting from hard-won human experience in composing services. A workflow fragment is a piece of an experimental description that is a coherent sub-workflow that makes sense to a domain specialist. Each fragment forms