10,319 research outputs found
BioWorkbench: A High-Performance Framework for Managing and Analyzing Bioinformatics Experiments
Advances in sequencing techniques have led to exponential growth in
biological data, demanding the development of large-scale bioinformatics
experiments. Because these experiments are computation- and data-intensive,
they require high-performance computing (HPC) techniques and can benefit from
specialized technologies such as Scientific Workflow Management Systems (SWfMS)
and databases. In this work, we present BioWorkbench, a framework for managing
and analyzing bioinformatics experiments. This framework automatically collects
provenance data, including both performance data from workflow execution and
data from the scientific domain of the workflow application. Provenance data
can be analyzed through a web application that abstracts a set of queries to
the provenance database, simplifying access to provenance information. We
evaluate BioWorkbench using three case studies: SwiftPhylo, a phylogenetic tree
assembly workflow; SwiftGECKO, a comparative genomics workflow; and RASflow, a
RASopathy analysis workflow. We analyze each workflow from both computational
and scientific domain perspectives, by using queries to a provenance and
annotation database. Some of these queries are available as a pre-built feature
of the BioWorkbench web application. Through the provenance data, we show that
the framework is scalable and achieves high-performance, reducing up to 98% of
the case studies execution time. We also show how the application of machine
learning techniques can enrich the analysis process
An automated workflow for parallel processing of large multiview SPIM recordings
Multiview light sheet fluorescence microscopy (LSFM) allows to image
developing organisms in 3D at unprecedented temporal resolution over long
periods of time. The resulting massive amounts of raw image data requires
extensive processing interactively via dedicated graphical user interface (GUI)
applications. The consecutive processing steps can be easily automated and the
individual time points can be processed independently, which lends itself to
trivial parallelization on a high performance cluster (HPC). Here we introduce
an automated workflow for processing large multiview, multi-channel,
multi-illumination time-lapse LSFM data on a single workstation or in parallel
on a HPC. The pipeline relies on snakemake to resolve dependencies among
consecutive processing steps and can be easily adapted to any cluster
environment for processing LSFM data in a fraction of the time required to
collect it.Comment: 13 pages with supplement, LATEX; 1 table, 1 figure, 2 supplementary
figures, 2 supplementary lists, 2 supplementary tables; corrected error in
results table, results unchange
Enhancing Workflow with a Semantic Description of Scientific Intent
Peer reviewedPreprin
An Open Framework for Extensible Multi-Stage Bioinformatics Software
In research labs, there is often a need to customise software at every step
in a given bioinformatics workflow, but traditionally it has been difficult to
obtain both a high degree of customisability and good performance.
Performance-sensitive tools are often highly monolithic, which can make
research difficult. We present a novel set of software development principles
and a bioinformatics framework, Friedrich, which is currently in early
development. Friedrich applications support both early stage experimentation
and late stage batch processing, since they simultaneously allow for good
performance and a high degree of flexibility and customisability. These
benefits are obtained in large part by basing Friedrich on the multiparadigm
programming language Scala. We present a case study in the form of a basic
genome assembler and its extension with new functionality. Our architecture has
the potential to greatly increase the overall productivity of software
developers and researchers in bioinformatics.Comment: 12 pages, 1 figure, to appear in proceedings of PRIB 201
Keemei: cloud-based validation of tabular bioinformatics file formats in Google Sheets.
BackgroundBioinformatics software often requires human-generated tabular text files as input and has specific requirements for how those data are formatted. Users frequently manage these data in spreadsheet programs, which is convenient for researchers who are compiling the requisite information because the spreadsheet programs can easily be used on different platforms including laptops and tablets, and because they provide a familiar interface. It is increasingly common for many different researchers to be involved in compiling these data, including study coordinators, clinicians, lab technicians and bioinformaticians. As a result, many research groups are shifting toward using cloud-based spreadsheet programs, such as Google Sheets, which support the concurrent editing of a single spreadsheet by different users working on different platforms. Most of the researchers who enter data are not familiar with the formatting requirements of the bioinformatics programs that will be used, so validating and correcting file formats is often a bottleneck prior to beginning bioinformatics analysis.Main textWe present Keemei, a Google Sheets Add-on, for validating tabular files used in bioinformatics analyses. Keemei is available free of charge from Google's Chrome Web Store. Keemei can be installed and run on any web browser supported by Google Sheets. Keemei currently supports the validation of two widely used tabular bioinformatics formats, the Quantitative Insights into Microbial Ecology (QIIME) sample metadata mapping file format and the Spatially Referenced Genetic Data (SRGD) format, but is designed to easily support the addition of others.ConclusionsKeemei will save researchers time and frustration by providing a convenient interface for tabular bioinformatics file format validation. By allowing everyone involved with data entry for a project to easily validate their data, it will reduce the validation and formatting bottlenecks that are commonly encountered when human-generated data files are first used with a bioinformatics system. Simplifying the validation of essential tabular data files, such as sample metadata, will reduce common errors and thereby improve the quality and reliability of research outcomes
Data Workflow - A Workflow Model for Continuous Data Processing
Online data or streaming data are getting more and more important for enterprise information systems, e.g. by integrating sensor data and workflows. The continuous flow of data provided e.g. by sensors requires new workflow models addressing the data perspective of these applications, since continuous data is potentially infinite while business process instances are always finite.\ud
In this paper a formal workflow model is proposed with data driven coordination and explicating properties of the continuous data processing. These properties can be used to optimize data workflows, i.e., reducing the computational power for processing the workflows in an engine by reusing intermediate processing results in several workflows
Data Provenance Inference in Logic Programming: Reducing Effort of Instance-driven Debugging
Data provenance allows scientists in different domains validating their models and algorithms to find out anomalies and unexpected behaviors. In previous works, we described on-the-fly interpretation of (Python) scripts to build workflow provenance graph automatically and then infer fine-grained provenance information based on the workflow provenance graph and the availability of data. To broaden the scope of our approach and demonstrate its viability, in this paper we extend it beyond procedural languages, to be used for purely declarative languages such as logic programming under the stable model semantics. For experiments and validation, we use the Answer Set Programming solver oClingo, which makes it possible to formulate and solve stream reasoning problems in a purely declarative fashion. We demonstrate how the benefits of the provenance inference over the explicit provenance still holds in a declarative setting, and we briefly discuss the potential impact for declarative programming, in particular for instance-driven debugging of the model in declarative problem solving
- …