Implementation and evaluation of a container-based software architecture

Recent advances in fields such as Cloud Computing, Web Systems, Internet of Things and Distributed NoSQL DBMS are enabling the development of innovative enterprise information systems that significantly increase the productivity of end users and developers. The aim of this thesis is to explore the new opportunities that these new technologies are bringing to the enterprise world. The new opportunities are explored by investigating the scenario of a medium-sized worldwide-trading company, Fiorital S.p.A. The thesis presents the design of a software architecture for the future information system of the company. The architecture is based on the usage of the Container technology and of the Microservice architectural style. Containers have empowered the usage of Microservices architectures by being lightweight, providing fast start-up times, and having low overhead. Candidate technologies for the implementation of the proposed software architecture are singled out, and the selection rationale is presented. This thesis provides an evaluation of both the candidate architecture and the technologies through the implementation of a prototype and the application of synthetic workloads that mimic stressful use scenarios. The results show that, in spite of the relative immaturity of some of the candidate technologies, the information system's candidate architecture is appropriate and that a company like Fiorital would considerably benefit from it

CoAP Infrastructure for IoT

The Internet of Things (IoT) can be seen as a large-scale network of billions of smart devices. Often IoT devices exchange data in small but numerous messages, which requires IoT services to be more scalable and reliable than ever. Traditional protocols that are known in the Web world does not fit well in the constrained environment that these devices operate in. Therefore many lightweight protocols specialized for the IoT have been studied, among which the Constrained Application Protocol (CoAP) stands out for its well-known REST paradigm and easy integration with existing Web. On the other hand, new paradigms such as Fog Computing emerges, attempting to avoid the centralized bottleneck in IoT services by moving computations to the edge of the network. Since a node of the Fog essentially belongs to relatively constrained environment, CoAP fits in well. Among the many attempts of building scalable and reliable systems, Erlang as a typical concurrency-oriented programming (COP) language has been battle tested in the telecom industry, which has similar requirements as the IoT. In order to explore the possibility of applying Erlang and COP in general to the IoT, this thesis presents an Erlang based CoAP server/client prototype ecoap with a flexible concurrency model that can scale up to an unconstrained environment like the Cloud and scale down to a constrained environment like an embedded platform. The flexibility of the presented server renders the same architecture applicable from Fog to Cloud. To evaluate its performance, the proposed server is compared with the mainstream CoAP implementation on an Amazon Web Service (AWS) Cloud instance and a Raspberry Pi 3, representing the unconstrained and constrained environment respectively. The ecoap server achieves comparable throughput, lower latency, and in general scales better than the other implementation in the Cloud and on the Raspberry Pi. The thesis yields positive results and demonstrates the value of the philosophy of Erlang in the IoT space

Toward Customizable Multi-tenant SaaS Applications

abstract: Nowadays, Computing is so pervasive that it has become indeed the 5th utility (after water, electricity, gas, telephony) as Leonard Kleinrock once envisioned. Evolved from utility computing, cloud computing has emerged as a computing infrastructure that enables rapid delivery of computing resources as a utility in a dynamically scalable, virtualized manner. However, the current industrial cloud computing implementations promote segregation among different cloud providers, which leads to user lockdown because of prohibitive migration cost. On the other hand, Service-Orented Computing (SOC) including service-oriented architecture (SOA) and Web Services (WS) promote standardization and openness with its enabling standards and communication protocols. This thesis proposes a Service-Oriented Cloud Computing Architecture by combining the best attributes of the two paradigms to promote an open, interoperable environment for cloud computing development. Mutil-tenancy SaaS applicantions built on top of SOCCA have more flexibility and are not locked down by a certain platform. Tenants residing on a multi-tenant application appear to be the sole owner of the application and not aware of the existence of others. A multi-tenant SaaS application accommodates each tenant’s unique requirements by allowing tenant-level customization. A complex SaaS application that supports hundreds, even thousands of tenants could have hundreds of customization points with each of them providing multiple options, and this could result in a huge number of ways to customize the application. This dissertation also proposes innovative customization approaches, which studies similar tenants’ customization choices and each individual users behaviors, then provides guided semi-automated customization process for the future tenants. A semi-automated customization process could enable tenants to quickly implement the customization that best suits their business needs.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Processamento de eventos complexos como serviço em ambientes multi-nuvem

Orientadores: Luiz Fernando Bittencourt, Miriam Akemi Manabe CapretzTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: O surgimento das tecnologias de dispositivos móveis e da Internet das Coisas, combinada com avanços das tecnologias Web, criou um novo mundo de Big Data em que o volume e a velocidade da geração de dados atingiu uma escala sem precedentes. Por ser uma tecnologia criada para processar fluxos contínuos de dados, o Processamento de Eventos Complexos (CEP, do inglês Complex Event Processing) tem sido frequentemente associado a Big Data e aplicado como uma ferramenta para obter informações em tempo real. Todavia, apesar desta onda de interesse, o mercado de CEP ainda é dominado por soluções proprietárias que requerem grandes investimentos para sua aquisição e não proveem a flexibilidade que os usuários necessitam. Como alternativa, algumas empresas adotam soluções de baixo nível que demandam intenso treinamento técnico e possuem alto custo operacional. A fim de solucionar esses problemas, esta pesquisa propõe a criação de um sistema de CEP que pode ser oferecido como serviço e usado através da Internet. Um sistema de CEP como Serviço (CEPaaS, do inglês CEP as a Service) oferece aos usuários as funcionalidades de CEP aliadas às vantagens do modelo de serviços, tais como redução do investimento inicial e baixo custo de manutenção. No entanto, a criação de tal serviço envolve inúmeros desafios que não são abordados no atual estado da arte de CEP. Em especial, esta pesquisa propõe soluções para três problemas em aberto que existem neste contexto. Em primeiro lugar, para o problema de entender e reusar a enorme variedade de procedimentos para gerência de sistemas CEP, esta pesquisa propõe o formalismo Reescrita de Grafos com Atributos para Gerência de Processamento de Eventos Complexos (AGeCEP, do inglês Attributed Graph Rewriting for Complex Event Processing Management). Este formalismo inclui modelos para consultas CEP e transformações de consultas que são independentes de tecnologia e linguagem. Em segundo lugar, para o problema de avaliar estratégias de gerência e processamento de consultas CEP, esta pesquisa apresenta CEPSim, um simulador de sistemas CEP baseado em nuvem. Por fim, esta pesquisa também descreve um sistema CEPaaS fundamentado em ambientes multi-nuvem, sistemas de gerência de contêineres e um design multiusuário baseado em AGeCEP. Para demonstrar sua viabilidade, o formalismo AGeCEP foi usado para projetar um gerente autônomo e um conjunto de políticas de auto-gerenciamento para sistemas CEP. Além disso, o simulador CEPSim foi minuciosamente avaliado através de experimentos que demonstram sua capacidade de simular sistemas CEP com acurácia e baixo custo adicional de processamento. Por fim, experimentos adicionais validaram o sistema CEPaaS e demonstraram que o objetivo de oferecer funcionalidades CEP como um serviço escalável e tolerante a falhas foi atingido. Em conjunto, esses resultados confirmam que esta pesquisa avança significantemente o estado da arte e também oferece novas ferramentas e metodologias que podem ser aplicadas à pesquisa em CEPAbstract: The rise of mobile technologies and the Internet of Things, combined with advances in Web technologies, have created a new Big Data world in which the volume and velocity of data generation have achieved an unprecedented scale. As a technology created to process continuous streams of data, Complex Event Processing (CEP) has been often related to Big Data and used as a tool to obtain real-time insights. However, despite this recent surge of interest, the CEP market is still dominated by solutions that are costly and inflexible or too low-level and hard to operate. To address these problems, this research proposes the creation of a CEP system that can be offered as a service and used over the Internet. Such a CEP as a Service (CEPaaS) system would give its users CEP functionalities associated with the advantages of the services model, such as no up-front investment and low maintenance cost. Nevertheless, creating such a service involves challenges that are not addressed by current CEP systems. This research proposes solutions for three open problems that exist in this context. First, to address the problem of understanding and reusing existing CEP management procedures, this research introduces the Attributed Graph Rewriting for Complex Event Processing Management (AGeCEP) formalism as a technology- and language-agnostic representation of queries and their reconfigurations. Second, to address the problem of evaluating CEP query management and processing strategies, this research introduces CEPSim, a simulator of cloud-based CEP systems. Finally, this research also introduces a CEPaaS system based on a multi-cloud architecture, container management systems, and an AGeCEP-based multi-tenant design. To demonstrate its feasibility, AGeCEP was used to design an autonomic manager and a selected set of self-management policies. Moreover, CEPSim was thoroughly evaluated by experiments that showed it can simulate existing systems with accuracy and low execution overhead. Finally, additional experiments validated the CEPaaS system and demonstrated it achieves the goal of offering CEP functionalities as a scalable and fault-tolerant service. In tandem, these results confirm this research significantly advances the CEP state of the art and provides novel tools and methodologies that can be applied to CEP researchDoutoradoCiência da ComputaçãoDoutor em Ciência da Computação140920/2012-9CNP

POLICY-BASED MIDDLEWARE FOR MOBILE CLOUD COMPUTING

Mobile devices are the dominant interface for interacting with online services as well as an efficient platform for cloud data consumption. Cloud computing allows the delivery of applications/functionalities as services over the internet and provides the software/hardware infrastructure to host these services in a scalable manner. In mobile cloud computing, the apps running on the mobile device use cloud hosted services to overcome resource constraints of the host device. This approach allows mobile devices to outsource the resource-consuming tasks. Furthermore, as the number of devices owned by a single user increases, there is the growing demand for cross-platform application deployment to ensure a consistent user experience. However, the mobile devices communicate through unstable wireless networks, to access the data and services hosted in the cloud. The major challenges that mobile clients face when accessing services hosted in the cloud, are network latency and synchronization of data. To address the above mentioned challenges, this research proposed an architecture which introduced a policy-based middleware that supports user to access cloud hosted digital assets and services via an application across multiple mobile devices in a seamless manner. The major contribution of this thesis is identifying different information, used to configure the behavior of the middleware towards reliable and consistent communication among mobile clients and the cloud hosted services. Finally, the advantages of the using policy-based middleware architecture are illustrated by experiments conducted on a proof-of-concept prototype

Web-IDE for Low-Code Development in OutSystems

Due to the growing popularity of cloud computing and its numerous benefits, many desktop applications have been, and will continue to be, migrated into the cloud and made available through the web. These applications can then be accessed through any device that has access to a browser and internet connection, eliminating the need for installation or managing dependencies. Moreover, the process of introduction to the product is much simpler, faster and collaboration aspects are facilitated. OutSystems is a company that provides software that enables, through an Integrated Development Environment (IDE) and a specific Low-Code language, users to securely and rapidly build robust applications. However, there are only available desktop versions of this IDE. For this reason, the objective of the proposed thesis is to understand what would be the best path for developing a Web-based version of the IDE. To achieve this, it is important not only to understand the OutSystems Platform and, more specifically, the architecture of the Service Studio IDE, which is the component IDE provided by the product, but also to explore the state-of-the-art technologies that could prove to be beneficial for the development of the project. The goal of this work is to debate different architectural possibilities to implement the project in question and present a conclusion as to what the adequate course of action, given the context of the problem. After distinguishing what are the biggest uncertainties and relevant points, a proof of concept is to be presented accompanied with the respective implementation details. Finally, this work intends to determine what would be a viable technological architecture to build a Web-based IDE that is capable of maintaining an acceptable performance, similarly to Service Studio IDE, while also insuring that the this system is scalable, in order to be able to provide the service to a large amount of users. That is to say, to present a conclusion regarding the feasibility of the project proposed.Devido ao aumento de popularidade de tecnologias de computação cloud e as suas inúmeras vantagens, aplicações desktop estão e vão continuar a ser migradas para a cloud para que possam ser acedidas através da web. Estas aplicações podem ser acedidas através de qualquer dispositivo que tenha acesso à internet, eliminando a necessidade de instalação e gestão de dependências. Além disso, o processo de introdução ao produto é simplificado, mais rápido e a colaboração é facilitada. A OutSystems é uma empresa que disponibiliza um software que faz com que utilizadores, através de um IDE e uma linguagem de baixo nível, possam criar aplicações robustas de forma rápida e segura. No entanto, atualmente só existem versões deste IDE para desktop. Como tal, o objetivo da tese proposta é perceber qual será a melhor forma de desenvolver uma versão do IDE sobre a Web. Para alcançar isto, é importante não só compreender a Plataforma OutSystems e, mais especificamente, a arquitetura do Service Studio IDE, que é o principal componente disponibilizado pelo produto, mas também explorar as tecnologias estado de arte que podem ser benéficas para o desenvolvimento do projeto. O objetivo deste trabalho é debater diferentes arquiteturas possíveis para a implementação do projeto e concluir qual será o curso de ação adequado, dado o contexto do problema. Após distinguir quais são os maiores pontos de incerteza, uma prova de conceito é apresentada juntamente com os respetivos detalhes de implementação. Finalmente, este trabalho tem como intenção detalhar uma arquitetura tecnológica viável para construir um IDE na web capaz de manter uma performance aceitável, semelhante à do Service Studio IDE, e garantir a escalabilidade do sistema, de forma a conseguir oferecer o serviço a um número elevado de utilizadores. Por outras palavras, apresentar uma conclusão em relação à viabilidade do projeto proposto

Triggerflow: Trigger-based Orchestration of Serverless Workflows

As more applications are being moved to the Cloud thanks to serverless computing, it is increasingly necessary to support native life cycle execution of those applications in the data center. But existing systems either focus on short-running workflows (like IBM Composer or Amazon Express Workflows) or impose considerable overheads for synchronizing massively parallel jobs (Azure Durable Functions, Amazon Step Functions, Google Cloud Composer). None of them are open systems enabling extensible interception and optimization of custom workflows. We present Triggerflow: an extensible Trigger-based Orchestration architecture for serverless workflows built on top of Knative Eventing and Kubernetes technologies. We demonstrate that Triggerflow is a novel serverless building block capable of constructing different reactive schedulers (State Machines, Directed Acyclic Graphs, Workflow as code). We also validate that it can support high-volume event processing workloads, auto-scale on demand and transparently optimize scientific workflows.Comment: The 14th ACM International Conference on Distributed and Event-based Systems (DEBS 2020

Complex Event Processing as a Service in Multi-Cloud Environments

The rise of mobile technologies and the Internet of Things, combined with advances in Web technologies, have created a new Big Data world in which the volume and velocity of data generation have achieved an unprecedented scale. As a technology created to process continuous streams of data, Complex Event Processing (CEP) has been often related to Big Data and used as a tool to obtain real-time insights. However, despite this recent surge of interest, the CEP market is still dominated by solutions that are costly and inflexible or too low-level and hard to operate. To address these problems, this research proposes the creation of a CEP system that can be offered as a service and used over the Internet. Such a CEP as a Service (CEPaaS) system would give its users CEP functionalities associated with the advantages of the services model, such as no up-front investment and low maintenance cost. Nevertheless, creating such a service involves challenges that are not addressed by current CEP systems. This research proposes solutions for three open problems that exist in this context. First, to address the problem of understanding and reusing existing CEP management procedures, this research introduces the Attributed Graph Rewriting for Complex Event Processing Management (AGeCEP) formalism as a technology- and language-agnostic representation of queries and their reconfigurations. Second, to address the problem of evaluating CEP query management and processing strategies, this research introduces CEPSim, a simulator of cloud-based CEP systems. Finally, this research also introduces a CEPaaS system based on a multi-cloud architecture, container management systems, and an AGeCEP-based multi-tenant design. To demonstrate its feasibility, AGeCEP was used to design an autonomic manager and a selected set of self-management policies. Moreover, CEPSim was thoroughly evaluated by experiments that showed it can simulate existing systems with accuracy and low execution overhead. Finally, additional experiments validated the CEPaaS system and demonstrated it achieves the goal of offering CEP functionalities as a scalable and fault-tolerant service. In tandem, these results confirm this research significantly advances the CEP state of the art and provides novel tools and methodologies that can be applied to CEP research