Towards Cloud Application Description Templates Supporting Quality of Service

Typical scientific, industrial and public sector applications require resource scalability and efficient resource utilization in order to serve a variable number of customers. Cloud computing provides an ideal solution to support such applications. However, the dynamic and intelligent utilization of cloud infrastructure resources from the perspective of cloud applications is not trivial. Although there have been several efforts to support the intelligent and coordinated deployment, and to a smaller extent also the run-time orchestration of cloud applications, no comprehensive solution has emerged until now that successfully leverages large scale near operational levels and ease of use. COLA is a European research project to provide a reference implementation of a generic and pluggable framework that supports the optimal and secure deployment and run-time orchestration of cloud applications. Such applications can then be embedded into workflows or science gateway frameworks to support complex application scenarios from user-friendly interfaces. A specific aspect of the cloud orchestration framework developed by COLA is the ability to describe complex application architectures incorporating several services. Besides the description of service components, the framework will also support the definition of various Quality of Service (QoS) parameters related to performance, economic viability and security. This paper concentrates on this latter aspect analysing how such application description templates can be developed based on existing standards and technologies

Modelling the behaviour of management operations in cloud-based applications

How to flexibly manage complex applications over heterogeneous clouds is one of the emerging problems in the cloud era. The OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) aims at solving this problem by providing a language to describe and manage complex cloud applications in a portable, vendoragnostic way. TOSCA permits to define an application as an orchestration of nodes, whose types can specify states, requirements, capabilities and management operations — but not how they interact each another. In this paper we first propose how to extend TOSCA to specify the behaviour of management operations and their relations with states, requirements, and capabilities. We then illustrate how such behaviour can be naturally modelled, in a compositional way, by means of open Petri nets. The proposed modelling permits to automate different analyses, such as determining whether a deployment plan is valid, which are its effects, or which plans allow to reach certain system configurations

Finding available services in TOSCA-compliant clouds

The OASIS TOSCA specification aims at enhancing the por-ta-bility of cloud applications by defining a language to describe and manage them across heterogeneous clouds. A service template is defined as an orchestration of typed nodes, which can be instantiated by matching other service templates. In this paper, we define and implement the notions of {em exact} and {it plug-in matching} between TOSCA service templates and node types. We then define two other types of matching ({em flexible} and {em white-box}), each permitting to ignore larger sets of non-relevant syntactic differences when type-checking service templates with respect to node types. The paper also describes how a service template that plug-in, flexibly or white-box matches a node type can be suitably adapted so as to exactly match it

The SEC-system : reuse support for scheduling system development

Recently, in a joint cooperation of Stichting VNA, SAL Apotheken, the Faculty of Management and Organization, and the University Centre for Pharmacy, University of Groningen in the Netherlands, a Ph.D-study started regarding Apot(he)ek, Organization and Management (APOM). The APOM-project deals with the structuring and steering of pharmacy organization. The manageability of the internal pharmacy organization, and the manageability of the direct environment of pharmacy organization is the subject matter. The theoretical background of the APOM-project is described. A literature study was made to find mixes of objectives. Three mixes of objectives in pharmacy organization are postulated; the product mix, the process mix, and the customer mix. The typology will be used as a basic starting point for the empirical study in the next phase of the APOM-project.

INDIGO-DataCloud: a Platform to Facilitate Seamless Access to E-Infrastructures

[EN] This paper describes the achievements of the H2020 project INDIGO-DataCloud. The project has provided e-infrastructures with tools, applications and cloud framework enhancements to manage the demanding requirements of scientific communities, either locally or through enhanced interfaces. The middleware developed allows to federate hybrid resources, to easily write, port and run scientific applications to the cloud. In particular, we have extended existing PaaS (Platform as a Service) solutions, allowing public and private e-infrastructures, including those provided by EGI, EUDAT, and Helix Nebula, to integrate their existing services and make them available through AAI services compliant with GEANT interfederation policies, thus guaranteeing transparency and trust in the provisioning of such services. Our middleware facilitates the execution of applications using containers on Cloud and Grid based infrastructures, as well as on HPC clusters. Our developments are freely downloadable as open source components, and are already being integrated into many scientific applications.INDIGO-Datacloud has been funded by the European Commision H2020 research and innovation program under grant agreement RIA 653549.Salomoni, D.; Campos, I.; Gaido, L.; Marco, J.; Solagna, P.; Gomes, J.; Matyska, L.... (2018). INDIGO-DataCloud: a Platform to Facilitate Seamless Access to E-Infrastructures. Journal of Grid Computing. 16(3):381-408. https://doi.org/10.1007/s10723-018-9453-3S381408163García, A.L., Castillo, E.F.-d., Puel, M.: Identity federation with VOMS in cloud infrastructures. In: 2013 IEEE 5Th International Conference on Cloud Computing Technology and Science, pp 42–48 (2013)Chadwick, D.W., Siu, K., Lee, C., Fouillat, Y., Germonville, D.: Adding federated identity management to OpenStack. Journal of Grid Computing 12(1), 3–27 (2014)Craig, A.L.: A design space review for general federation management using keystone. In: Proceedings of the 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing, pp 720–725. IEEE Computer Society (2014)Pustchi, N., Krishnan, R., Sandhu, R.: Authorization federation in iaas multi cloud. In: Proceedings of the 3rd International Workshop on Security in Cloud Computing, pp 63–71. ACM (2015)Lee, C.A., Desai, N., Brethorst, A.: A Keystone-Based Virtual Organization Management System. In: 2014 IEEE 6Th International Conference On Cloud Computing Technology and Science (Cloudcom), pp 727–730. IEEE (2014)Castillo, E.F.-d., Scardaci, D., García, A.L.: The EGI Federated Cloud e-Infrastructure. Procedia Computer Science 68, 196–205 (2015)AARC project: AARC Blueprint Architecture, see https://aarc-project.eu/architecture . Technical report (2016)Oesterle, F., Ostermann, S., Prodan, R., Mayr, G.J.: Experiences with distributed computing for meteorological applications: grid computing and cloud computing. Geosci. Model Dev. 8(7), 2067–2078 (2015)Plasencia, I.C., Castillo, E.F.-d., Heinemeyer, S., García, A.L., Pahlen, F., Borges, G.: Phenomenology tools on cloud infrastructures using OpenStack. The European Physical Journal C 73(4), 2375 (2013)Boettiger, C.: An introduction to docker for reproducible research. ACM SIGOPS Operating Systems Review 49(1), 71–79 (2015)Docker: http://www.docker.com (2013)Gomes, J., Campos, I., Bagnaschi, E., David, M., Alves, L., Martins, J., Pina, J., Alvaro, L.-G., Orviz, P.: Enabling rootless linux containers in multi-user environments: the udocker tool. Computing Physics Communications. https://doi.org/10.1016/j.cpc.2018.05.021 (2018)Zhang, Z., Chuan, W., Cheung, D.W.L.: A survey on cloud interoperability taxonomies, standards, and practice. SIGMETRICS perform. Eval. Rev. 40(4), 13–22 (2013)Lorido-Botran, T., Miguel-Alonso, J., Lozano, J.A.: A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments. Journal of Grid Computing 12(4), 559–592 (2014)Nyrén, R., Metsch, T., Edmonds, A., Papaspyrou, A.: Open Cloud Computing Interface–Core. Technical report, Open Grid Forum (2010)Metsch, T., Edmonds, A.: Open Cloud Computing Interface-Infrastructure. Technical report, Open Grid Forum (2010)Metsch, T., Edmonds, A.: Open Cloud Computing Interface-RESTful HTTP Rendering. Technical report, Open Grid Forum (2011)(Ca Technologies) Lipton, P., (Ibm) Moser, S., (Vnomic) Palma, D., (Ibm) Spatzier, T.: Topology and Orchestration Specification for Cloud Applications. Technical report, OASIS Standard (2013)Teckelmann, R., Reich, C., Sulistio, A.: Mapping of cloud standards to the taxonomy of interoperability in IaaS. In: Proceedings - 2011 3rd IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2011, pp 522–526 (2011)García, A.L., Castillo, E.F.-d., Fernández, P.O.: Standards for enabling heterogeneous IaaS cloud federations. Computer Standards & Interfaces 47, 19–23 (2016)Caballer, M., Zala, S., García, A.L., Montó, G., Fernández, P.O., Velten, M.: Orchestrating complex application architectures in heterogeneous clouds. Journal of Grid Computing 16 (1), 3–18 (2018)Hardt, M., Jejkal, T., Plasencia, I.C., Castillo, E.F.-d., Jackson, A., Weiland, M., Palak, B., Plociennik, M., Nielsson, D.: Transparent Access to Scientific and Commercial Clouds from the Kepler Workflow Engine. Computing and Informatics 31(1), 119 (2012)Fakhfakh, F., Kacem, H.H., Kacem, A.H.: Workflow Scheduling in Cloud Computing a Survey. In: IEEE 18Th International Enterprise Distributed Object Computing Conference Workshops and Demonstrations (EDOCW), 2014, Vol. 71, pp. 372–378. Springer, New York (2014)Stockton, D.B., Santamaria, F.: Automating NEURON simulation deployment in cloud resources. Neuroinformatics 15(1), 51–70 (2017)Plóciennik, M., Fiore, S., Donvito, G., Owsiak, M., Fargetta, M., Barbera, R., Bruno, R., Giorgio, E., Williams, D.N., Aloisio, G.: Two-level Dynamic Workflow Orchestration in the INDIGO DataCloud for Large-scale, Climate Change Data Analytics Experiments. Procedia Computer Science 80, 722–733 (2016)Moreno-Vozmediano, R., Montero, R.S., Llorente, I.M.: Multicloud deployment of computing clusters for loosely coupled mtc applications. IEEE transactions on parallel and distributed systems 22(6), 924–930 (2011)Katsaros, G., Menzel, M., Lenk, A.: Cloud Service Orchestration with TOSCA, Chef and Openstack. In: Ic2e (2014)Garcia, A.L., Zangrando, L., Sgaravatto, M., Llorens, V., Vallero, S., Zaccolo, V., Bagnasco, S., Taneja, S., Dal Pra, S., Salomoni, D., Donvito, G.: Improved Cloud resource allocation: how INDIGO-DataCloud is overcoming the current limitations in Cloud schedulers. J. Phys. Conf. Ser. 898(9), 92010 (2017)Singh, S., Chana, I.: A survey on resource scheduling in cloud computing issues and challenges. Journal of Grid Computing, pp. 1–48 (2016)García, A.L., Castillo, E.F.-d., Fernández, P.O., Plasencia, I.C., de Lucas, J.M.: Resource provisioning in Science Clouds: Requirements and challenges. Software: Practice and Experience 48(3), 486–498 (2018)Chauhan, M.A., Babar, M.A., Benatallah, B.: Architecting cloud-enabled systems: a systematic survey of challenges and solutions. Software - Practice and Experience 47(4), 599–644 (2017)Somasundaram, T.S., Govindarajan, K.: CLOUDRB A Framework for scheduling and managing High-Performance Computing (HPC) applications in science cloud. Futur. Gener. Comput. Syst. 34, 47–65 (2014)Sotomayor, B., Keahey, K., Foster, I.: Overhead Matters: A Model for Virtual Resource Management. In: Proceedings of the 2nd International Workshop on Virtualization Technology in Distributed Computing SE - VTDC ’06, p 5. IEEE Computer Society, Washington (2006)SS, S.S., Shyam, G.K., Shyam, G.K.: Resource management for Infrastructure as a Service (IaaS) in cloud computing SS Manvi A survey. J. Netw. Comput. Appl. 41, 424–440 (2014)INDIGO-DataCloud consortium: Initial requirements from research communities - d2.1, see https://www.indigo-datacloud.eu/documents/initial-requirements-research-communities-d21 https://www.indigo-datacloud.eu/documents/initial-requirements-research-communities-d21 https://www.indigo-datacloud.eu/documents/initial-requirements-research-communities-d21 . Technical report (2015)Europen open science cloud: https://ec.europa.eu/research/openscience (2015)Proot: https://proot-me.github.io/ (2014)Runc: https://github.com/opencontainers/runc (2016)Fakechroot: https://github.com/dex4er/fakechroot (2015)Pérez, A., Moltó, G., Caballer, M., Calatrava, A.: Serverless computing for container-based architectures Future Generation Computer Systems (2018)de Vries, K.J.: Global fits of supersymmetric models after LHC run 1. Phd thesis Imperial College London (2015)Openstack: https://www.openstack.org/ (2015)See http://argus-documentation.readthedocs.io/en/stable/argus_introduction.html (2017)See https://en.wikipedia.org/wiki/xacml (2013)See http://www.simplecloud.info (2014)Opennebula: http://opennebula.org/ (2018)Redhat openshift: http://www.opencityplatform.eu (2011)The cloud foundry foundation: https://www.cloudfoundry.org/ (2015)Caballer, M., Blanquer, I., Moltó, G., de Alfonso, C.: Dynamic management of virtual infrastructures. Journal of Grid Computing 13(1), 53–70 (2015)See http://www.infoq.com/articles/scaling-docker-with-kubernetes http://www.infoq.com/articles/scaling-docker-with-kubernetes (2014)Prisma project: http://www.ponsmartcities-prisma.it/ (2010)Opencitiy platform: http://www.opencityplatform.eu (2014)Onedata: https://onedata.org/ (2018)Dynafed: http://lcgdm.web.cern.ch/dynafed-dynamic-federation-project http://lcgdm.web.cern.ch/dynafed-dynamic-federation-project (2011)Fts3: https://svnweb.cern.ch/trac/fts3 (2011)Fernández, P.O., García, A.L., Duma, D.C., Donvito, G., David, M., Gomes, J.: A set of common software quality assurance baseline criteria for research projects, see http://hdl.handle.net/10261/160086 . Technical reportHttermann, M.: Devops for developers Apress (2012)EOSC-Hub: ”Integrating and managing services for the European Open Science Cloud” Funded by H2020 research and innovation pr ogramme under grant agreement No. 777536. See http://eosc-hub.eu (2018)Apache License: author = https://www.apache.org/licenses/LICENSE-2.0 (2004)INDIGO Package Repo: http://repo.indigo-datacloud.eu/ (2017)INDIGO DockerHub: https://hub.docker.com/u/indigodatacloud/ https://hub.docker.com/u/indigodatacloud/ (2015)Indigo gitbook: https://indigo-dc.gitbooks.io/indigo-datacloud-releases https://indigo-dc.gitbooks.io/indigo-datacloud-releases (2017)Van Zundert, G.C., Bonvin, A.M.: Disvis: quantifying and visualizing the accessible interaction space of distance restrained biomolecular complexes. Bioinformatics 31(19), 3222–3224 (2015)Van Zundert, G.C., Bonvin, A.M.: Fast and sensitive rigid–body fitting into cryo–em density maps with powerfit. AIMS Biophys. 2(0273), 73–87 (2015

Persistence and discovery of reusable cloud application topologies

Due to the benefits introduced by the Cloud computing paradigm and the increase of available Cloud services (VM- and non VM-oriented), in the last years the number of application developers strongly supporting a partial or complete migration of application component to Cloud environments has significantly increased. For example, it is possible to host the application's database off-premise (e.g. in a DBaaS solution) while keeping the remaining components (presentation or business logic components) on-premise. However, the previous application deployment is only one possible distribution alternative, and the existence of further alternatives allows the generation of a wide variety of distribution combinations. In addition, the challenges for application developers to efficiently select optimal strategy of application's deployment by considering evolving application performance with fluctuating workload has increased rapidly. How to select, configure and deploy an application optimally to satisfy functional and non-functional requirements of business and operation has been a research area in both academic and industry domains. In this Master thesis, basing on the approaches proposed in previous work, we first conduct a research on existing approaches and technologies about how to persist, retrieve and build typed graph-based Cloud application topologies leveraging the benefits introduced and developed in graph databases and graph database technologies, respectively. Consequently, we develop the core algorithms for persisting and discovering application topologies focusing on their similar characteristics. Such conceptual models relate to the required structural aspects representing the relationship between the application topologies, their performance aspects, and their evolving workload. As a result of this thesis, a prototypical implementation of a RESTful-based framework to support discovering and building reusable viable topologies of Cloud application w.r.t. evolving functional and non-functional aspects is provided, e.g. taking into account its performance, its corresponding profile and its corresponding evolving workload

Recognition of resource patterns in human-centric processes : a case study

Business experts need to improve business processes by increasing process efficiency and reducing process costs. To achieve this, a common method is to capture a series of repeatedly conducted process activities and their structure, i.e. the business logic of the process, and then enact process execution based on it. However, there exist informal processes, which are human-centric processes that are highly dynamic. Since this approach assumes the existence of predictable business logic of the process, it does not apply for management of informal processes. The Informal Process Essentials (IPE) model is a modeling approach for informal processes. This model depicts informal processes by documenting resources used in these process. Through this approach, we are able to retain best practice and knowledge accumulated in the processes. Based on this approach, there is also the InProXec method to enable the application of the IPE approach in organizations. In this thesis work, we want to validate the concepts introduced in the InProXec method using a case study on the jclouds project. To achieve this aim, we introduce the concept of a generic mapping mechanism and an evolving correlation coefficient function. Based on these concepts, we present the Informal Process Discoverer (IPD) service. The IPD service is an implementation of the discovery of IPE models. The test results of the IPD service have shown that the service is successful in discovering the IPE model and giving process recommendations. For example, using an informal process model with includes 7 human resources and 2 GitHub repositories as input, we are able to discover 74 other resources that participate in the process including 65 human resources and 9 Git repositories

Characterizing and providing interoperability to function as a service platforms

Dissertação para obtenção do Grau de Mestre em Engenharia Informática e de ComputadoresA computação sem servidor abstrai o controlo da infraestrutura dos programadores e executa código a pedido com escalonamento automático onde apenas se é cobrado pela quantidade de recursos consumidos. Um dos serviços mais populares da computação sem servidor é a Função como Serviço (Function-as-a-Service ou FaaS), onde os programadores são muitas vezes confrontados com requisitos específicos dos prestadores de serviços de nuvem. Requisitos de assinatura das funções, e o uso de bibliotecas exclusivas ao prestador de serviços, foram identificados como sendo as principais causas de problemas de portabilidade das aplicações FaaS. O controlo reduzido da infraestrutura e a elevada dependência para com o prestador de serviços dá origem a diversos problemas de aprisionamento tecnológico. Neste trabalho, introduzimos o QuickFaaS, uma ferramenta para desktop de interoperabilidade multi-cloud com foco principal no desenvolvimento de funções agnósticas à nuvem e na criação das mesmas na respetiva plataforma. O QuickFaaS permite melhorar substancialmente a produtividade, flexibilidade e agilidade no desenvolvimento de soluções sem servidor para múltiplos prestadores de serviços, sem o requisito de instalar software adicional. A abordagem agnóstica à nuvem irá permitir que os programadores reutilizem as suas funções em diferentes prestadores de serviços sem terem a necessidade de reescrever código. A solução visa a minimizar o aprisionamento tecnológico nas plataformas FaaS através do aumento da portabilidade das funções sem servidor, incentivando assim programadores e organizações a apostarem em diferentes prestadores de serviços em troca de um benefício funcional.Serverless computing hides infrastructure management from developers and runs code on-demand automatically scaled and billed during code’s execution time. One of the most popular serverless backend services is called Function-as-a-Service (FaaS), in which developers are many times confronted with cloud-specific requirements. Function signature requirements, and the usage of custom libraries that are unique to cloud providers, were identified as the two main reasons for portability issues in FaaS applications. Such reduced control over the infrastructure and tight-coupling with cloud services amplifies various vendor lock-in problems. In this work, we introduce QuickFaaS, a multi-cloud interoperability desktop tool targeting cloud-agnostic functions development and FaaS deployments. QuickFaaS substantially improves developers’ productivity, flexibility and agility when creating serverless solutions to multiple cloud providers, without requiring the installation of extra software. The proposed cloud-agnostic approach enables developers to reuse their serverless functions in different cloud providers with no need to rewrite code. The solution aims to minimize vendor lock-in in FaaS platforms by increasing the portability of serverless functions, which will, therefore, encourage developers and organizations to target different providers in exchange for a functional benefit.N/

Automatic deployment and reproducibility of workflow on the Cloud using container virtualization

PhD ThesisCloud computing is a service-oriented approach to distributed computing that has many attractive features, including on-demand access to large compute resources. One type of cloud applications are scientific work ows, which are playing an increasingly important role in building applications from heterogeneous components. Work ows are increasingly used in science as a means to capture, share, and publish computational analysis. Clouds can offer a number of benefits to work ow systems, including the dynamic provisioning of the resources needed for computation and storage, which has the potential to dramatically increase the ability to quickly extract new results from the huge amounts of data now being collected. However, there are increasing number of Cloud computing platforms, each with different functionality and interfaces. It therefore becomes increasingly challenging to de ne work ows in a portable way so that they can be run reliably on different clouds. As a consequence, work ow developers face the problem of deciding which Cloud to select and - more importantly for the long-term - how to avoid vendor lock-in. A further issue that has arisen with work ows is that it is common for them to stop being executable a relatively short time after they were created. This can be due to the external resources required to execute a work ow - such as data and services - becoming unavailable. It can also be caused by changes in the execution environment on which the work ow depends, such as changes to a library causing an error when a work ow service is executed. This "work ow decay" issue is recognised as an impediment to the reuse of work ows and the reproducibility of their results. It is becoming a major problem, as the reproducibility of science is increasingly dependent on the reproducibility of scientific work ows. In this thesis we presented new solutions to address these challenges. We propose a new approach to work ow modelling that offers a portable and re-usable description of the work ow using the TOSCA specification language. Our approach addresses portability by allowing work ow components to be systematically specifed and automatically - v - deployed on a range of clouds, or in local computing environments, using container virtualisation techniques. To address the issues of reproducibility and work ow decay, our modelling and deployment approach has also been integrated with source control and container management techniques to create a new framework that e ciently supports dynamic work ow deployment, (re-)execution and reproducibility. To improve deployment performance, we extend the framework with number of new optimisation techniques, and evaluate their effect on a range of real and synthetic work ows.Ministry of Higher Education and Scientific Research in Iraq and Mosul Universit

Deployment and Operation of Complex Software in Heterogeneous Execution Environments

This open access book provides an overview of the work developed within the SODALITE project, which aims at facilitating the deployment and operation of distributed software on top of heterogeneous infrastructures, including cloud, HPC and edge resources. The experts participating in the project describe how SODALITE works and how it can be exploited by end users. While multiple languages and tools are available in the literature to support DevOps teams in the automation of deployment and operation steps, still these activities require specific know-how and skills that cannot be found in average teams. The SODALITE framework tackles this problem by offering modelling and smart editing features to allow those we call Application Ops Experts to work without knowing low level details about the adopted, potentially heterogeneous, infrastructures. The framework offers also mechanisms to verify the quality of the defined models, generate the corresponding executable infrastructural code, automatically wrap application components within proper execution containers, orchestrate all activities concerned with deployment and operation of all system components, and support on-the-fly self-adaptation and refactoring