Transparent Orchestration of Task-based Parallel Applications in Containers Platforms

This paper presents a framework to easily build and execute parallel applications in container-based distributed computing platforms in a user-transparent way. The proposed framework is a combination of the COMP Superscalar (COMPSs) programming model and runtime, which provides a straightforward way to develop task-based parallel applications from sequential codes, and containers management platforms that ease the deployment of applications in computing environments (as Docker, Mesos or Singularity). This framework provides scientists and developers with an easy way to implement parallel distributed applications and deploy them in a one-click fashion. We have built a prototype which integrates COMPSs with different containers engines in different scenarios: i) a Docker cluster, ii) a Mesos cluster, and iii) Singularity in an HPC cluster. We have evaluated the overhead in the building phase, deployment and execution of two benchmark applications compared to a Cloud testbed based on KVM and OpenStack and to the usage of bare metal nodes. We have observed an important gain in comparison to cloud environments during the building and deployment phases. This enables better adaptation of resources with respect to the computational load. In contrast, we detected an extra overhead during the execution, which is mainly due to the multi-host Docker networking.This work is partly supported by the Spanish Government through Programa Severo Ochoa (SEV-2015-0493), by the Spanish Ministry of Science and Technology through TIN2015-65316 project, by the Generalitat de Catalunya under contracts 2014-SGR-1051 and 2014-SGR-1272, and by the European Union through the Horizon 2020 research and innovation program under grant 690116 (EUBra-BIGSEA Project). Results presented in this paper were obtained using the Chameleon testbed supported by the National Science Foundation.Peer ReviewedPostprint (author's final draft

Towards a Reference Architecture with Modular Design for Large-scale Genotyping and Phenotyping Data Analysis: A Case Study with Image Data

With the rapid advancement of computing technologies, various scientific research communities have been extensively using cloud-based software tools or applications. Cloud-based applications allow users to access software applications from web browsers while relieving them from the installation of any software applications in their desktop environment. For example, Galaxy, GenAP, and iPlant Colaborative are popular cloud-based systems for scientific workflow analysis in the domain of plant Genotyping and Phenotyping. These systems are being used for conducting research, devising new techniques, and sharing the computer assisted analysis results among collaborators. Researchers need to integrate their new workflows/pipelines, tools or techniques with the base system over time. Moreover, large scale data need to be processed within the time-line for more effective analysis. Recently, Big Data technologies are emerging for facilitating large scale data processing with commodity hardware. Among the above-mentioned systems, GenAp is utilizing the Big Data technologies for specific cases only. The structure of such a cloud-based system is highly variable and complex in nature. Software architects and developers need to consider totally different properties and challenges during the development and maintenance phases compared to the traditional business/service oriented systems. Recent studies report that software engineers and data engineers confront challenges to develop analytic tools for supporting large scale and heterogeneous data analysis. Unfortunately, less focus has been given by the software researchers to devise a well-defined methodology and frameworks for flexible design of a cloud system for the Genotyping and Phenotyping domain. To that end, more effective design methodologies and frameworks are an urgent need for cloud based Genotyping and Phenotyping analysis system development that also supports large scale data processing. In our thesis, we conduct a few studies in order to devise a stable reference architecture and modularity model for the software developers and data engineers in the domain of Genotyping and Phenotyping. In the first study, we analyze the architectural changes of existing candidate systems to find out the stability issues. Then, we extract architectural patterns of the candidate systems and propose a conceptual reference architectural model. Finally, we present a case study on the modularity of computation-intensive tasks as an extension of the data-centric development. We show that the data-centric modularity model is at the core of the flexible development of a Genotyping and Phenotyping analysis system. Our proposed model and case study with thousands of images provide a useful knowledge-base for software researchers, developers, and data engineers for cloud based Genotyping and Phenotyping analysis system development

A new approach for publishing workflows: abstractions, standards, and linked data

In recent years, a variety of systems have been developed that export the workflows used to analyze data and make them part of published articles. We argue that the workflows that are published in current approaches are dependent on the specific codes used for execution, the specific workflow system used, and the specific workflow catalogs where they are published. In this paper, we describe a new approach that addresses these shortcomings and makes workflows more reusable through: 1) the use of abstract workflows to complement executable workflows to make them reusable when the execution environment is different, 2) the publication of both abstract and executable workflows using standards such as the Open Provenance Model that can be imported by other workflow systems, 3) the publication of workflows as Linked Data that results in open web accessible workflow repositories. We illustrate this approach using a complex workflow that we re-created from an influential publication that describes the generation of 'drugomes'

Towards the Implementation of an Intelligent ERP System: Guidelines for Building Intelligent ERP Systems

Dissertation presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Information Systems and Technologies ManagementThe digital age has forced companies to change the way they operate their businesses and adapt quickly to the digital transformation driven by increased global competitiveness in recent years. To remain competitive, organizations must implement management solutions that allow them to efficiently control all business areas through an Enterprise Resource Planning (ERP) system. Management systems have had to evolve to keep up with technological advancements by incorporating intelligent tools. As a result, ERP companies have created new systems known as intelligent ERP (i-ERP). Given the variety of improvement opportunities, it has become necessary to develop a series of guidelines for i-ERP manufacturing as well as for companies that want to implement intelligent solutions in their different business areas, in order to assist technical and non-technical people selecting the best existing option. A design science research (DSR) methodology was used to accomplish the study's goal. It was mandatory to start by defining what an i-ERP system is. Furthermore, their seven capabilities have been clarified, such as intelligent behaviour, learning management, advanced analytics, process automation, intelligent interfaces, dark analytics, and simplification of customization. These capabilities are based on technologies such as artificial intelligence, machine learning, big data, and cloud computing. The guidelines were based on these seven capabilities and were applied to the four major modules of an ERP, which are financial, purchasing, sales, and human resources. As a result, it was possible to create a table with recommendations in general by i-ERP capabilities, followed by guidelines focusing on the financial, purchasing, sales, and human resources areas, and an assessment tool that allowed creating measures to evaluate an ERP system, considering its level of intelligence based on the recommendations created. Finally, the evaluation system was used to rate the latest system developed by SAP SE, SAP S4/HANA, demonstrating its usefulness, followed by expert interviews to validate the recommendations for the four areas identified in terms of their use and acceptance. The relevant literature review and my personal work experience were used as the basis for this master's thesis. It is expected that this study will contribute to the scientific community's understanding of intelligent information systems as well as arouse curiosity in future studies.A era digital forçou as empresas a mudarem a forma como operam os seus negócios e a adaptarem-se rapidamente à transformação digital impulsionada pelo aumento da competitividade global nos últimos anos. Para se manterem competitivas, as organizações devem implementar soluções de gestão que lhes permitam controlar eficazmente todas as áreas de negócio através de um sistema de planeamento de recursos corporativos (ERP). Os sistemas de gestão tiveram de evoluir para acompanhar os avanços tecnológicos, incorporando ferramentas inteligentes. Como resultado, as empresas de sistemas ERP criaram produtos conhecidos como ERP inteligentes (i-ERP). Dada a variedade de oportunidades de melhoria, tornou-se necessário desenvolver uma série de orientações para fabricantes de i-ERP bem como para empresas que pretendam implementar soluções inteligentes nas diversas áreas de negócio, a fim de ajudar as pessoas técnicas e não técnicas na seleção da melhor opção existente. Uma metodologia de desenho de investigação científica (DSR) foi utilizada para atingir o objetivo do estudo. Foi obrigatório começar por definir o que é um sistema i-ERP bem como as suas sete capacidades identificadas, como ter um comportamento inteligente, gestão da aprendizagem, análise avançada, automatização de processos, interfaces inteligentes, análise escura, e simplificação da personalização, que têm como base tecnologias como inteligência artificial, aprendizagem de máquinas, grandes dados e armazenamento em nuvem. As orientações utilizaram como base estas sete capacidades e foram aplicadas aos quatro principais módulos de um ERP, que são o financeiro, compras, logística e recursos humanos. Como resultado foi possível criar uma tabela de recomendações gerais por capacidades de um i-ERP, seguida de recomendações com foco na área financeira, compras, logística e recursos humanos e por último uma ferramenta de avaliação que permitiu criar medidas para avaliar um sistema ERP, considerando o seu nível de inteligência com base nas recomendações criadas. Por último, o sistema de avaliação foi utilizado para classificar o mais recente sistema desenvolvido pela SAP SE, o SAP S4/HANA, demonstrando a sua utilidade, seguido de entrevistas a especialistas para validar as recomendações para as quatro áreas identificadas em termos de respetiva utilização e aceitação. Uma relevante revisão bibliográfica e a minha experiência profissional foram utilizadas como base para esta tese de mestrado. Espera-se que este estudo contribua para a compreensão dos sistemas de informação inteligentes pela comunidade científica, assim como despertar curiosidade em estudos futuros

Automatic generation of software interfaces for supporting decisionmaking processes. An application of domain engineering & machine learning

[EN] Data analysis is a key process to foster knowledge generation in particular domains or fields of study. With a strong informative foundation derived from the analysis of collected data, decision-makers can make strategic choices with the aim of obtaining valuable benefits in their specific areas of action. However, given the steady growth of data volumes, data analysis needs to rely on powerful tools to enable knowledge extraction. Information dashboards offer a software solution to analyze large volumes of data visually to identify patterns and relations and make decisions according to the presented information. But decision-makers may have different goals and, consequently, different necessities regarding their dashboards. Moreover, the variety of data sources, structures, and domains can hamper the design and implementation of these tools. This Ph.D. Thesis tackles the challenge of improving the development process of information dashboards and data visualizations while enhancing their quality and features in terms of personalization, usability, and flexibility, among others. Several research activities have been carried out to support this thesis. First, a systematic literature mapping and review was performed to analyze different methodologies and solutions related to the automatic generation of tailored information dashboards. The outcomes of the review led to the selection of a modeldriven approach in combination with the software product line paradigm to deal with the automatic generation of information dashboards. In this context, a meta-model was developed following a domain engineering approach. This meta-model represents the skeleton of information dashboards and data visualizations through the abstraction of their components and features and has been the backbone of the subsequent generative pipeline of these tools. The meta-model and generative pipeline have been tested through their integration in different scenarios, both theoretical and practical. Regarding the theoretical dimension of the research, the meta-model has been successfully integrated with other meta-model to support knowledge generation in learning ecosystems, and as a framework to conceptualize and instantiate information dashboards in different domains. In terms of the practical applications, the focus has been put on how to transform the meta-model into an instance adapted to a specific context, and how to finally transform this later model into code, i.e., the final, functional product. These practical scenarios involved the automatic generation of dashboards in the context of a Ph.D. Programme, the application of Artificial Intelligence algorithms in the process, and the development of a graphical instantiation platform that combines the meta-model and the generative pipeline into a visual generation system. Finally, different case studies have been conducted in the employment and employability, health, and education domains. The number of applications of the meta-model in theoretical and practical dimensions and domains is also a result itself. Every outcome associated to this thesis is driven by the dashboard meta-model, which also proves its versatility and flexibility when it comes to conceptualize, generate, and capture knowledge related to dashboards and data visualizations