Towards a service-oriented e-infrastructure for multidisciplinary environmental research

Research e-infrastructures are considered to have generic and thematic parts. The generic part provids high-speed networks, grid (large-scale distributed computing) and database systems (digital repositories and data transfer systems) applicable to all research commnities irrespective of discipline. Thematic parts are specific deployments of e-infrastructures to support diverse virtual research communities. The needs of a virtual community of multidisciplinary envronmental researchers are yet to be investigated. We envisage and argue for an e-infrastructure that will enable environmental researchers to develop environmental models and software entirely out of existing components through loose coupling of diverse digital resources based on the service-oriented achitecture. We discuss four specific aspects for consideration for a future e-infrastructure: 1) provision of digital resources (data, models & tools) as web services, 2) dealing with stateless and non-transactional nature of web services using workflow management systems, 3) enabling web servce discovery, composition and orchestration through semantic registries, and 4) creating synergy with existing grid infrastructures

A Service-Oriented Architecture enabling dynamic services grouping for optimizing distributed workflows execution

International audienceIn this paper, we describe a Service-Oriented Architecture allowing the optimization of the execution of service workflows. We discuss the advantages of the service-oriented approach with regard to the enactment of scientific applications on a grid infrastructure. Based on the development of a generic Web-Services wrapper, we show how the flexibility of our architecture enables dynamic service grouping for optimizing the application execution time. We demonstrate performance results on a real medical imaging application. On a production grid infrastructure, the optimization proposed introduces a significant speed-up (from 1.2 to 2.9) when compared to a traditional execution

PaPaS: A Portable, Lightweight, and Generic Framework for Parallel Parameter Studies

The current landscape of scientific research is widely based on modeling and simulation, typically with complexity in the simulation's flow of execution and parameterization properties. Execution flows are not necessarily straightforward since they may need multiple processing tasks and iterations. Furthermore, parameter and performance studies are common approaches used to characterize a simulation, often requiring traversal of a large parameter space. High-performance computers offer practical resources at the expense of users handling the setup, submission, and management of jobs. This work presents the design of PaPaS, a portable, lightweight, and generic workflow framework for conducting parallel parameter and performance studies. Workflows are defined using parameter files based on keyword-value pairs syntax, thus removing from the user the overhead of creating complex scripts to manage the workflow. A parameter set consists of any combination of environment variables, files, partial file contents, and command line arguments. PaPaS is being developed in Python 3 with support for distributed parallelization using SSH, batch systems, and C++ MPI. The PaPaS framework will run as user processes, and can be used in single/multi-node and multi-tenant computing systems. An example simulation using the BehaviorSpace tool from NetLogo and a matrix multiply using OpenMP are presented as parameter and performance studies, respectively. The results demonstrate that the PaPaS framework offers a simple method for defining and managing parameter studies, while increasing resource utilization.Comment: 8 pages, 6 figures, PEARC '18: Practice and Experience in Advanced Research Computing, July 22--26, 2018, Pittsburgh, PA, US

Extending OWL-S for the Composition of Web Services Generated With a Legacy Application Wrapper

Despite numerous efforts by various developers, web service composition is still a difficult problem to tackle. Lot of progressive research has been made on the development of suitable standards. These researches help to alleviate and overcome some of the web services composition issues. However, the legacy application wrappers generate nonstandard WSDL which hinder the progress. Indeed, in addition to their lack of semantics, WSDLs have sometimes different shapes because they are adapted to circumvent some technical implementation aspect. In this paper, we propose a method for the semi automatic composition of web services in the context of the NeuroLOG project. In this project the reuse of processing tools relies on a legacy application wrapper called jGASW. The paper describes the extensions to OWL-S in order to introduce and enable the composition of web services generated using the jGASW wrapper and also to implement consistency checks regarding these services.Comment: ICIW 2012, The Seventh International Conference on Internet and Web Applications and Services, Stuttgart : Germany (2012

Full Metadata Object profiling for flexible geoprocessing workflows

The design and running of complex geoprocessing workflows is an increasingly common geospatial modelling and analysis task. The Business Process Model and Notation (BPMN) standard, which provides a graphical representation of a workflow, allows stakeholders to discuss the scientific conceptual approach behind this modelling while also defining a machine-readable encoding in XML. Previous research has enabled the orchestration of Open Geospatial Consortium (OGC) Web Processing Services (WPS) with a BPMN workflow engine. However, the need for direct access to pre-defined data inputs and outputs results in a lack of flexibility during composition of the workflow and of efficiency during execution. This article develops metadata profiling approaches, described as two possible configurations, which enable workflow management at the meta-level through a coupling with a metadata catalogue. Specifically, a WPS profile and a BPMN profile are developed and tested using open-source components to achieve this coupling. A case study in the context of an event mapping task applied within a big data framework and based on analysis of the Global Database of Event Language and Tone (GDELT) database illustrates the two different architectures

GridCertLib: a Single Sign-on Solution for Grid Web Applications and Portals

This paper describes the design and implementation of GridCertLib, a Java library leveraging a Shibboleth-based authentication infrastructure and the SLCS online certificate signing service, to provide short-lived X.509 certificates and Grid proxies. The main use case envisioned for GridCertLib, is to provide seamless and secure access to Grid/X.509 certificates and proxies in web applications and portals: when a user logs in to the portal using Shibboleth authentication, GridCertLib can automatically obtain a Grid/X.509 certificate from the SLCS service and generate a VOMS proxy from it. We give an overview of the architecture of GridCertLib and briefly describe its programming model. Its application to some deployment scenarios is outlined, as well as a report on practical experience integrating GridCertLib into portals for Bioinformatics and Computational Chemistry applications, based on the popular P-GRADE and Django softwares.Comment: 18 pages, 1 figure; final manuscript accepted for publication by the "Journal of Grid Computing

Process Oriented Collaboration in Grid-Environments: A Case Study in the Construction Industry

This paper addresses the process-oriented collaboration based on a grid-based platform for the support of virtual organizations (VO), illustrated on the example of the construction industry. Distributed, organizational and IT-structures of teams involved in vintage complex projects cannot be managed with conventional methods in an appropriate manner. Both using a grid platform and grid-based services, in conjunction with semantic methods for consistency saving and goal-oriented process management can increase the efficiency of collaboration processes in large-scale projects. A hybrid grid- and web service-based architecture for the next generation of VO service and a gateway solution was developed integrating the process-oriented perspective and prototypically implemented. The problem, as well as the solution on the basis of the hybrid system architecture combing the benefits of the cutting-edge technologies, the methodical concept for modeling VO processes and their automated execution on a grid platform are discussed in detail