Data Workflow - A Workflow Model for Continuous Data Processing

Online data or streaming data are getting more and more important for enterprise information systems, e.g. by integrating sensor data and workflows. The continuous flow of data provided e.g. by sensors requires new workflow models addressing the data perspective of these applications, since continuous data is potentially infinite while business process instances are always finite.\ud In this paper a formal workflow model is proposed with data driven coordination and explicating properties of the continuous data processing. These properties can be used to optimize data workflows, i.e., reducing the computational power for processing the workflows in an engine by reusing intermediate processing results in several workflows

Workflows and service discovery: a mobile device approach

Bioinformatics has moved from command-line standalone programs to web-service based environments. Such trend has resulted in an enormous amount of online resources which can be hard to find and identify, let alone execute and exploit. Furthermore, these resources are aimed -in general- to solve specific tasks. Usually, this tasks need to be combined in order to achieve the desired results. In this line, finding the appropriate set of tools to build up a workflow to solve a problem with the services available in a repository is itself a complex exercise. Issues such as services discovering, composition and representation appear. On the technological side, mobile devices have experienced an incredible growth in the number of users and technical capabilities. Starting from this reality, in the present paper, we propose a solution for service discovering and workflow generation while distinct approaches of representing workflows in a mobile environment are reviewed and discussed. As a proof of concept, a specific use case has been developed: we have embedded an expanded version of our Magallanes search engine into mORCA, our mobile client for bioinformatics. Such composition delivers a powerful and ubiquitous solution that provides the user with a handy tool for not only generate and represent workflows, but also services, data types, operations and service types discoveryUniversidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Quality Flow : uma plataforma colaborativa orientada a qualidade para experimentos em eScience

Orientador: Claudia Maria Bauzer MedeirosDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Muitos procedimentos de pesquisa científica dependem da análise de dados obtidos de fontes de dados heterogêneas. A validade dos resultados de pesquisa depende, entre outros, da qualidade dos dados - um tópico recorrente na pesquisa em computação há décadas. Embora existam muitas propostas para a avaliação da qualidade de dados, ainda há problemas em aberto, como mecanismos flexíveis para a avaliação de qualidade e maneiras para derivar a qualidade dos dados. O objetivo desta dissertação é trabalhar nesses problemas. A principal contribuição da dissertação é a criação do QualityFlow: uma plataforma colaborativa para avaliação de qualidade para experimentos em eScience. As principais contribuições são: suportar à criação de workflows científicos com parâmetros de qualidade, permitindo a adição de atributos de qualidade a workflows, permitindo ao mesmo tempo que usuários disintos definam métricas de qualidade específicas para o mesmo workflow; permitir aos usuários manter o histórico de diferentes avaliações de qualidade para um mesmo processo, provendo assim melhor compreensão do real valor dos dados e workflows; e permitir aos cientistas customizar dimensões de qualidade de dados e métricas de qualidade colaborativamente. O QualityFlow foi desenvolvido como um protótipo web, e executado para dois experimentos ¿ um baseado em dados reais e o outro em um workflow de exemploAbstract: Many scientific research procedures rely upon the analysis of data obtained from heterogeneous sources. The validity of the research results depends, among others, on the quality of data. Data quality is a topic that has pervaded computer science research for decades. Though there are many proposals for data quality assessment, there are still open problems such as mechanisms to support flexible quality assessment and ways to derive data quality. The goal of this dissertation is to work on these issues. The main contribution of this dissertation is the proposal of QualityFlow: a quality-aware collaborative platform for experiments in eScience. The following contributions were accomplished: to support the creation of quality-aware scientific workflows, allowing the addition of quality attributes to workflows, while at the same time letting distinct users define their specific quality metrics for the same workflow; to allow users to keep track of different quality assessments for a given process, thereby providing insights into the actual value of data and workflow; and to allow scientists to customize data quality dimensions and quality metrics collaboratively. QualityFlow was developed as a web prototype, and executed in two experiments - one based upon a real problem and the other on a sample workflowMestradoCiência da ComputaçãoMestre em Ciência da Computaçã

Simulating sensor networks

Tese de mestrado em Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2010Nos últimos anos, as redes de sensores sem fios conheceram um grande impulso em variadas ´áreas, nomeadamente na monitorização industrial e ambiental e, mais recentemente, na logística e noutras aplicações que envolvem processos de negócio e a chamada Internet das Coisas e dos Serviços. Contudo, e apesar dos avanços que se têm verificado tanto em termos de hardware como de software, estas redes são difíceis de programar, testar e instalar. A simulação de redes de sensores é frequentemente utilizada para testar e depurar aplicações para redes de sensores, pois permite testar a execução de das aplicações em ambientes virtuais. Esta tese aborda um problema que diz respeito a testar estas redes através de simulação: a definição (manual) de modelos. A nossa abordagem aponta para a geração de modelos de simulação directamente a partir de aplicações redes de sensores, em particular, modelos para o simulador VisualSense criados a partir de aplicações escritas em Callas, uma linguagem de programação para as redes de sensores. Para tal, criamos uma ferramenta capaz de gerar modelos que ´e paramétrica pelos modelos de rede e modelos sensores da rede que se pretende modelar, e ainda por um conjunto extensível de parâmetros de simulação. As nossas experiências mostraram resultados encorajadores na simulação de redes de grande escala, uma vez que conseguimos executar simulações com até 5000 nós. À medida que as redes de sensores sem fios começam a ser utilizadas em processos de negócio, a informação que recolhem do ambiente tem cada vez mais influência no decurso dos fluxos de trabalho associados aos processos de negócio. De um modo geral, os testes levados a cabo em fluxos de trabalho fazem uso de informação gravada em fluxos de trabalho executados previamente, tornando difícil testar o sistema como um todo. Em alternativa, e como uma segunda proposta desta tese, propomos testar fluxos de trabalho através da incorporação de resultados obtidos nas simulações das aplicações das redes de sensores. Além de cobrir os casos cobertos pela primeira abordagem, esta técnica permite testar novos fluxos de trabalho, bem como as mudanças ocorridas num determinado fluxo de trabalho por acontecimentos no ambiente.In recent years, Wireless Sensor Networks have gaining momentum in several fields, notably in industrial and environmental monitoring and, more recently, in logistics. However, and in spite of the advances in hardware and software, Wireless Sensor Networks are still hard to program, test, and deploy. Simulation is often used for testing and debugging sensor networks because they allow us to perform deployments in virtual environments. This paper addresses a key problem of testing such networks using simulation: (manual) model definition. Our approach is to generate simulation models directly from WSN applications, in particular, VisualSense simulator models from applications written in Callas, a programming language for WSN. For that purpose, we create a model generator tool that is parameter sable by network and sensor templates, and by an extensible set of simulation parameters. Our experiments show encouraging results on simulating large scale networks, as we are able to handle WSN with as many as 5000 nodes. As Wireless Sensor Networks begin to play some role in business processes, the information they gather from the environment influences the execution of workflows. Generally, the tests carried out on these systems make use of recorded information in earlier workflow executions, making it difficult to test the system as a whole. Alternatively, and as a second proposal of this thesis, we propose testing such workflows by incorporating results obtained from the simulation of sensor network applications. Besides covering the situations described in the first approach, this technique allows the testing of new workflows, as well as the changes made to a given workflow by events in the environment

Enhancing integrated environmental modelling by designing resource-oriented interfaces

Integrated environmental modelling is gaining momentum for addressing grand scientific challenges such as monitoring the environment for change detection and forecasting environmental conditions along with the consequences for society. Such challenges can only be addressed by a multi-disciplinary approach, in which socio-economic, geospatial, and environmental information becomes inter-connected. However, existing solutions cannot be seamlessly integrated and current interaction paradigms prevent mainstream usage of the existing technology. In particular, it is still difficult to access and join harmonized data and processing algorithms that are provided by different environmental information infrastructures. In this paper we take a novel approach for integrated environmental modelling based on the notion of inter-linked resources on the Web. We present design practices for creating resource-oriented interfaces, driven by an interaction protocol built on the combination of valid linkages to enhance resource integration, accompanied by associated recommendations for implementation. The suggested resource-oriented approach provides a solution to the problems identified above, but still requires intense prototyping and experimentation. We discuss the central open issues and present a roadmap for future research

Data-Intensive architecture for scientific knowledge discovery

This paper presents a data-intensive architecture that demonstrates the ability to support applications from a wide range of application domains, and support the different types of users involved in defining, designing and executing data-intensive processing tasks. The prototype architecture is introduced, and the pivotal role of DISPEL as a canonical language is explained. The architecture promotes the exploration and exploitation of distributed and heterogeneous data and spans the complete knowledge discovery process, from data preparation, to analysis, to evaluation and reiteration. The architecture evaluation included large-scale applications from astronomy, cosmology, hydrology, functional genetics, imaging processing and seismology