A modeling language for multi-tenant data architecture evolution in cloud applications

Multi-tenancy enables efficient resource utilization by sharing application resources across multiple customers (i.e., tenants). Hence, applications built using this pat- tern can be offered at a lower price and reduce maintenance effort as less application instances and supporting cloud resources must be maintained. These properties en- courage cloud application providers to adopt multi-tenancy to their existing applications, yet introducing this pattern requires significant changes in the application structure to address multi-tenancy requirements such as isolation of tenants, extensibility of the application, and scalability of the solution. In cloud applications, the data layer is often the prime candidate for multi-tenancy, and it usually comprises a combination of different cloud storage solutions such as blob storage, relational and non-relational databases. These storage types are conceptually and tangibly divergent, each requiring its own partitioning schemes to meet multi-tenancy requirements. Currently, multi-tenant data architectures are implemented using manual coding methods, at times following guidance and patterns offered by cloud providers. However, such manual implementation approach tends to be time consuming and error prone. Several modeling methods based on Model-Driven Engineer- ing (MDE) and Software Product Line Engineering (SPLE) have been proposed to capture multi-tenancy in cloud applications. These methods mainly generate cloud deployment configurations from an application model, though they do not automate implementation or evolution of applications. This thesis aims to facilitate development of multi-tenant cloud data architectures using model-driven engineering techniques. This is achieved by designing and implementing a novel modeling language, CadaML, that provides concepts and notations to model multi-tenant cloud data architectures in an abstract way. CadaML also provides a set of tools to validate the data architecture and automatically produce corresponding data access layer code. The thesis demonstrates the feasibility of the modeling language in a practical setting and adequacy of multi-tenancy implementation by the generated code on an industrial business process analyzing application. Moreover, the modeling language is empirically compared against manual implementation methods to inspect its effect on developer productivity, development effort, reliability of the application code, and usability of the language. These outcomes provide a strong argument that the CadaML modeling language effectively mitigates the high overhead of manual implementation of multi-tenant cloud data layers, significantly reducing the required development complexity and time

Cloud resource orchestration in the multi-cloud landscape: a systematic review of existing frameworks

The number of both service providers operating in the cloud market and customers consuming cloud-based services is constantly increasing, proving that the cloud computing paradigm has successfully delivered its potential. Nevertheless, the unceasing growth of the cloud market is posing hard challenges on its participants. On the provider side, the capability of orchestrating resources in order to maximise profits without failing customers’ expectations is a matter of concern. On the customer side, the efficient resource selection from a plethora of similar services advertised by a multitude of providers is an open question. In such a multi-cloud landscape, several research initiatives advocate the employment of software frameworks (namely, cloud resource orchestration frameworks - CROFs) capable of orchestrating the heterogeneous resources offered by a multitude of cloud providers in a way that best suits the customer’s need. The objective of this paper is to provide the reader with a systematic review and comparison of the most relevant CROFs found in the literature, as well as to highlight the multi-cloud computing open issues that need to be addressed by the research community in the near future

Using DSML for Handling Multi-tenant Evolution in Cloud Applications

Multi-tenancy is sharing a single application's resources to serve more than a single group of users (i.e. tenant). Cloud application providers are encouraged to adopt multi-tenancy as it facilitates increased resource utilization and ease of maintenance, translating into lower operational and energy costs. However, introducing multi-tenancy to a single-tenant application requires significant changes in its structure to ensure tenant isolation, configurability and extensibility. In this paper, we analyse and address the different challenges associated with evolving an application's architecture to a multi-tenant cloud deployment. We focus specifically on multi-tenant data architectures, commonly the prime candidate for consolidation and multi-tenancy. We present a Domain-Specific Modeling language (DSML) to model a multi-tenant data architecture, and automatically generate source code that handles the evolution of the application's data layer. We apply the DSML on a representative case study of a single-tenant application evolving to become a multi-tenant cloud application under two resource sharing scenarios. We evaluate the costs associated with using this DSML against the state of the art and against manual evolution, reporting specifically on the gained benefits in terms of development effort and reliability

Infrastructure as Runtime Models - Towards Model-Driven Resource Management

International audienceThe importance of continuous delivery and the emergence of tools allowing to treat infrastructure configurations programmatically have revolutionized the way computing resources and software systems are managed. However, these tools keep lacking an explicit model representation of underlying resources making it difficult to introspect, verify or reconfigure the system in response to external events.In this paper, we outline a novel approach that treats system infrastructure as explicit runtime models. A key benefit of using such [email protected] representation is that it provides a uniform semantic foundation for resources monitoring and reconfiguration. Adopting models at runtime allows one to integrate different aspects of system management, such as resource monitoring and subsequent verification into an unified view which would otherwise have to be done manually and require to use different tools. It also simplifies the development of various self-adaptation strategies without requiring the engineers and researchers to cope with low-level system complexities

Model-based Continuous Deployment of SIS

This chapter is organized as follows. Section 4.2 provides an overview of the current state of the art and of the practice for the automatic deployment of SIS. Section 4.3 introduces our solutions for the automatic deployment of SIS, first describing how they can be integrated in order to form a coherent deployment bundle and then detailing each our two enablers: GENESIS and DivENACT. Section 4.4 focus on the support offered by our solutions to ensure the trustworthiness deployment of SIS. Finally, Section 4.5 draws some conclusions.publishedVersio

A systematic review on cloud testing

A systematic literature review is presented that surveyed the topic of cloud testing over the period (2012-2017). Cloud testing can refer either to testing cloud-based systems (testing of the cloud), or to leveraging the cloud for testing purposes (testing in the cloud): both approaches (and their combination into testing of the cloud in the cloud) have drawn research interest. An extensive paper search was conducted by both automated query of popular digital libraries and snowballing, which resulted into the final selection of 147 primary studies. Along the survey a framework has been incrementally derived that classifies cloud testing research along six main areas and their topics. The paper includes a detailed analysis of the selected primary studies to identify trends and gaps, as well as an extensive report of the state of art as it emerges by answering the identified Research Questions. We find that cloud testing is an active research field, although not all topics have received so far enough attention, and conclude by presenting the most relevant open research challenges for each area of the classification framework.This paper describes research work mostly undertaken in the context of the European Project H2020 731535: ElasTest. This work has also been partially supported by: the Italian MIUR PRIN 2015 Project: GAUSS; the Regional Government of Madrid (CM) under project Cloud4BigData (S2013/ICE-2894) cofunded by FSE & FEDER; and the Spanish Government under project LERNIM (RTC-2016-4674-7) cofunded by the Ministry of Economy and Competitiveness, FEDER & AEI

Resources Events Agents (REA), a text DSL for OMNIA Entities

The Numbersbelieve has been developing the OMNIA platform. This is a web application platform for developing applications using Low-code principles, using Agile approaches. Modeling Entities is an application that is used on the platform to create new entities. The OMNIA Entity concept has the following properties: Agents, Commitments, Documents, Events, entities, Resources or Series. Most of these concepts are in accordance with the Resources Events Agents (REA) ontology but are not formalized. One of the goals of Numbersbelieve is a formalization of the REA concepts according to the ontology for the application that creates entities on OMNIA platform and later for other applications. REA defines an enterprise ontology developed by McCarthy (1979, 1982) has its origin in accounting database systems. Later Geerts and McCarthy (2002, 2006) extended the original model with new concepts. To formalize the concepts of the REA ontology, this research shows the development of a textual Domain-Specific Language (DSL) based on the development methodology Model Driven Engineering (MDE) which focuses software development on models. This simplifies the engineering processes as it represents the actions and behaviors of a system even before the start of the coding phase. This research is structured according to the Design Science Research Methodology (DSRM). The Design Science (DS) is a methodology for solving problems that seek to innovate by creating useful artifacts that define practices, projects and implementations and is therefore suitable for this research. This research developed three artifacts for the formalization of the DSL, a meta-model the abstract syntax, a textual language the concrete syntax and a Json file for interaction with OMNIA. The first phase of DSRM was to identify the problem that was mentioned above. The following focuses on the identification of requirements which identified the REA concepts to be included in the meta-model and textual language. Subsequently, the development of the artifacts and the editor of the language. The editor allows use cases, provided by the Numbersbelieve team, to be defined with the DSL language, correct faults and improve the language. The results were evaluated according the objectives and requirements, all successfully completed. Based on the analysis of the artifacts, the use of the language and the interaction with the OMNIA platform, through the Json file, it is concluded that the use of the DSL language is suitable to interact with the OMNIA platform through the Application Program Interface (API) and helped demonstrate that other applications on the platform could be modeled using a REA approach.A Numbersbelieve tem vindo a desenvolver a plataforma OMNIA. Esta plataforma é uma aplicação web para o desenvolvimento de aplicações usando princípios Low-code, usando abordagens Agile. Modeling Entities é a aplicação que é usada na plataforma para criar novas entidades. O conceito OMNIA de Entidade tem as seguintes propriedades: Agents, Commitments, Documents, Events, Generic entities, Resources or Series. A maior parte destes conceitos estão de acordo com a ontologia REA mas não estão formalizados. Um dos objetivos da Numbersbelieve é ter uma formalização dos conceitos REA de acordo com a ontologia para a aplicação que cria as entidades na plataforma OMNIA e posteriormente para as outras aplicações. REA define uma ontologia empresarial desenvolvida por McCarthy (1979, 1982) tem sua origem nos sistemas de base de dados para contabilidade. Mais tarde Geerts and McCarthy (2002, 2006) estenderam o modelo original com novos conceitos. Para formalizar os conceitos da ontologia REA, esta pesquisa mostra o desenvolvimento de uma DSL textual com base na metodologia de desenvolvimento MDE que foca o desenvolvimento de software no modelo. Esta simplifica os processos de engenharia pois representa as ações e comportamentos de um sistema mesmo antes do início da fase de codificação. A pesquisa está estruturada de acordo com a DSRM. O DS é uma metodologia para resolver problemas que procuram inovar criando artefactos úteis que definem práticas, projetos e implementações e por isso é adequado a esta pesquisa que desenvolveu três artefactos para a formalização da DSL, um meta-modelo a sintaxe abstrata, uma linguagem textual a sintaxe concreta e um ficheiro Json para interação com a plataforma OMNIA. A primeira fase do DSRM foi identificar o problema que foi referido em cima. A seguinte concentra-se na identificação dos requisitos que identificaram os conceitos REA a serem incluídos no meta-modelo e na linguagem textual. Posteriormente, é feito o desenvolvimento dos artefactos e do editor da linguagem. O editor permite definir, com a DSL, os casos de uso fornecidos pela equipa da Numbersbelieve, corrigir falhas e melhorar a linguagem. Os resultados foram avaliados de acordo com o cumprimento dos requisitos. Foram todos foram concluídos com êxito. Com base na análise dos artefactos, do uso da linguagem e da interação com a plataforma OMNIA, através do ficheiro Json, conclui-se que a utilização da linguagem é adequada para interagir com a plataforma OMNIA através da sua API e ajudou a demonstrar que outras aplicações da plataforma podem ser modeladas usando uma abordagem REA

Dagstuhl News January - December 2011

"Dagstuhl News" is a publication edited especially for the members of the Foundation "Informatikzentrum Schloss Dagstuhl" to thank them for their support. The News give a summary of the scientific work being done in Dagstuhl. Each Dagstuhl Seminar is presented by a small abstract describing the contents and scientific highlights of the seminar as well as the perspectives or challenges of the research topic

MINIMIZATION OF RESOURCE CONSUMPTION THROUGH WORKLOAD CONSOLIDATION IN LARGE-SCALE DISTRIBUTED DATA PLATFORMS

The rapid increase in the data volumes encountered in many application domains has led to widespread adoption of parallel and distributed data management systems like parallel databases and MapReduce-based frameworks (e.g., Hadoop) in recent years. Use of such parallel and distributed frameworks is expected to accelerate in the coming years, putting further strain on already-scarce resources like compute power, network bandwidth, and energy. To reduce total execution times, there is a trend towards increasing execution parallelism by spreading out data across a large number of machines. However, this often increases the total resource consumption, and especially energy consumption, significantly because of process startup costs and other overheads (e.g., communication overheads). In this dissertation, we develop several data management techniques to minimize resource consumption through workload consolidation. In this dissertation, we introduce a key metric called query span, i.e., number of machines involved in the execution of a query or a job. In order to minimize the per query resource consumption we propose to minimize query span. To that end, we develop several workload-driven data partitioning and replica selection algorithms that attempt to minimize the average query span by exploiting the fact that most distributed environments need to use replication for fault tolerance. Extensive experiments on various datasets show that judicious data placement and replication can dramatically reduce the average query spans resulting in significant reductions in resource consumption. We show our results primarily on two applications, distributed data warehouse system and distributed information retrieval. In the first case, we show that minimizing average query spans can minimize overall resource consumption for a given workload and can also improve the performance of complex analytical queries. In the second case, our approach minimizes the overall search cost as well as effectively trades off search cost with load imbalance. The best case of resource efficiency for any underlying data processing system is achieved when the job or the query can be run efficiently on a single machine (i.e., query span=1). In the final part of dissertation, we discuss an in-memory MapReduce system optimized for performing complex analytics tasks on input data sizes that fit in a single machine's memory. We argue that systems like Hadoop that are designed to operate across a large number of machines are not optimal in performance for small and medium sized complex analytics tasks because of high startup costs, heavy disk activity, and wasteful checkpointing. We have developed a prototype runtime called HONE that is API compatible with standard (distributed) Hadoop. In other words, we can take existing Hadoop code and run it, without modification, on a multi-core shared memory machine. This allows us to take existing Hadoop algorithms and find the most suitable runtime environment for execution on datasets of varying sizes. Overall, in this dissertation, our key contributions in this work include identification of key metric query span and its relationship with overall resource consumption in scale-out architectures. We introduce several workload-aware techniques to optimize this key metric. We go on to demonstrate the effectiveness of query span minimization on different application scenarios. In order to take advantage of scale-up architectures effectively we develop novel in-memory MapReduce system HONE for single machine. Our thorough experiments on real and synthetic datasets demonstrate the efficacy of our proposed approaches