A multidimensional model for monitoring cloud services

The complexity of monitoring cloud environments and the lack of standards so far urge for a careful analysis, systematizing and understanding of key points involved when assessing the services provided. In this context, this paper proposes a layered model for Cloud Services monitoring, identifying the multiple dimensions of monitoring, while combining the perspectives of service providers and customers. This process involves the identification of relevant parameters and metrics for each monitoring dimension, focusing on monitoring of resources, quality of service, security and service contracts. Taking a stratified view of the problem, this study contributes to achieve a clearer and more efficient approach to cloud services monitoring.Fundação para a Ciência e a Tecnologia (FCT

GMonE: a complete approach to cloud monitoring

The inherent complexity of modern cloud infrastructures has created the need for innovative monitoring approaches, as state-of-the-art solutions used for other large-scale environments do not address specific cloud features. Although cloud monitoring is nowadays an active research field, a comprehensive study covering all its aspects has not been presented yet. This paper provides a deep insight into cloud monitoring. It proposes a unified cloud monitoring taxonomy, based on which it defines a layered cloud monitoring architecture. To illustrate it, we have implemented GMonE, a general-purpose cloud monitoring tool which covers all aspects of cloud monitoring by specifically addressing the needs of modern cloud infrastructures. Furthermore, we have evaluated the performance, scalability and overhead of GMonE with Yahoo Cloud Serving Benchmark (YCSB), by using the OpenNebula cloud middleware on the Grid’5000 experimental testbed. The results of this evaluation demonstrate the benefits of our approach, surpassing the monitoring performance and capabilities of cloud monitoring alternatives such as those present in state-of-the-art systems such as Amazon EC2 and OpenNebula

A Descriptive Literature Review and Classification of Cloud Computing Research

We present a descriptive literature review and classification scheme for cloud computing research. This includes 205 refereed journal articles published since the inception of cloud computing research. The articles are classified based on a scheme that consists of four main categories: technological issues, business issues, domains and applications, and conceptualising cloud computing. The results show that although current research is still skewed towards technological issues, new research themes regarding social and organisational implications are emerging. This review provides a reference source and classification scheme for IS researchers interested in cloud computing, and to indicate under-researched areas as well as future directions

Bisnesmetriikan kerääminen ja visualisointi pilvipohjaisessa kehitysympäristössä

Monitoring cloud computing resources is a straightforward and common task for any cloud application developer. The problem with current monitoring solutions is that they only focus on infrastructure resources. Many companies on the other hand would need data about the business side of their applications. This thesis extends the current monitoring solutions to capture business metrics from within applications. The metrics are then visualized to quickly allow for better analysis of the data. The tool is composed of three main components. The metrics are captured with a Node.js library that is imported in the monitored application. The library sends the captured data to InfluxDB timeseries database. The data is visualized with Grafana which implements tables, graphs, and gauges. The provided command-line tool creates a file that can be imported in Grafana to create a new dashboard with graphs in it. The requirements for the tool were created through the needs of software developers and clients of web- and mobile-developer Codemate. An architectural design was made based on the requirements and then implemented on the AWS cloud platform on top of Kubernetes. The implementation was evaluated by testing it in a real production server. The tool is functional and it works as intended. The results from the evaluation prove that the tool created in this thesis can help companies gain better information about their products. Future work includes adding the metrics capture for other languages such as Go and Ruby as well as integrating the tool to Codemate’s new development environment. Further research can be done especially in improving performance of the solution in large systems.Pilviresurssien monitorointi on selkeä ja yleinen tehtävä jokaiselle pilvipalvelun kehittäjälle. Monitorointisovellukset keskittyvät vain infrastruktuuriresursseihin, vaikka monet nykyajan yritykset tarvitsisivat tarkempaa tietoa sovellusten bisnespuolesta. Tämä diplomityö laajentaa nykyisiä monitorointisovelluksia kattamaan bisnesmetriikan keräämisen applikaatioiden sisältä sekä visualisoi datan paremman analyysin mahdollistamiseksi. Diplomityössä kehitetty työkalu koostuu kolmesta osasta. Metriikat kerätään sovelluksista Node.js-kirjaston avulla, joka lisätään sovelluksen koodiin. Kirjasto lähettää dataa InfluxDB-tietokantaan, josta se visualisoidaan Grafanalla interaktiivisten kuvaajien sekä taulukoiden avulla. Grafanaan voidaan lisäksi luoda työpöytiä diplomityötä varten luodulla ohjelmalla. Bisnesmetriikan keräämiseen ja visualisointiin luotu työkalu määriteltiin ohjelmistokehittäjä Codematen ohjelmistoinsinöörien sekä asiakkaiden tarpeiden mukaan. Määrittelyä käytettiin työkalun arkkitehtuurin luomiseen, joka ohjasi käytännön toteutusta. Työkalu rakennettiin Amazonin AWS-palveluun Kuberneteksen päälle. Toteutetun työkalun toimivuus testattiin lopuksi asiakasympäristössä tuotantopalvelimella. Työkalun todettiin toimivan tarkoituksenmukaisesti. Testauksesta saadut tulokset osoittavat, että työkalu voi auttaa yrityksiä saamaan parempaa informaatiota ohjelmistotuotteistaan sekä niiden käytöstä. Työkalun kehitystä voidaan jatkaa laajentamalla sen toimintaa Go- ja Ruby-kielille sekä integroimalla se tiiviimmin Codematen uuteen kehitysympäristöön. Lisätutkimus erityisesti suorituskyvyn parantamiseen laajoissa järjestelmissä on tarpeen

Utilising stream reasoning techniques to underpin an autonomous framework for cloud application platforms

As cloud application platforms (CAPs) are reaching the stage where the human effort required to maintain them at an operational level is unsupportable, one of the major challenges faced by the cloud providers is to develop appropriate mechanisms for run-time monitoring and adaptation, to prevent cloud application platforms from quickly dissolving into a non-reliable environment. In this context, the application of intelligent approaches to Autonomic Clouds may offer promising opportunities. In this paper we present an approach to providing cloud platforms with autonomic capabilities, utilising techniques from the Semantic Web and Stream Reasoning research fields. The main idea of this approach is to encode values, monitored within cloud application platforms, using Semantic Web languages, which then allows us to integrate semantically-enriched observation streams with static ontological knowledge and apply intelligent reasoning. Using such run-time reasoning capabilities, we have developed a conceptual architecture for an autonomous framework and describe a prototype solution we have constructed which implements this architecture. Our prototype is able to perform analysis and failure diagnosis, and suggest further adaptation actions. We report our experience in utilising the Stream Reasoning technique in this context as well as further challenges that arise out of our work

An Actor-Centric, Asset-Based Monitor Deployment Model for Cloud Computing

Effective monitoring is essential for the security of cloud systems. Although many monitoring tools exist in the cloud domain, there is little guidance on how to deploy monitors to make the most of collected monitor data and increase the likelihood of detecting breaches of security. We introduce an actor-centric, asset-based monitor deployment model for the cloud that enables practitioners to reason about monitor deployment in terms of the security of the cloud assets that they own. We define an actor model that consolidates several roles in the literature to three roles that are motivated by security. We then develop an architectural model that identifies the assets that can be owned by each of those actors, and use it to drive an asset-based cloud threat model. Using our threat model, we claim that a cloud practitioner can reason about monitor deployment to more efficiently deploy monitors and increase its chances of detecting intrusions. We demonstrate the utility of our model with a cloud scenario based on Netflix’s use of Amazon Web Services.Air Force Research Laboratory & Air Force Office of Scientific Research/FA8750-11-2-0084Ope

Foundations and Technological Landscape of Cloud Computing

The cloud computing paradigm has brought the benefits of utility computing to a global scale. It has gained paramount attention in recent years. Companies are seriously considering to adopt this new paradigm and expecting to receive significant benefits. In fact, the concept of cloud computing is not a revolution in terms of technology; it has been established based on the solid ground of virtualization, distributed system, and web services. To comprehend cloud computing, its foundations and technological landscape need to be adequately understood. This paper provides a comprehensive review on the building blocks of cloud computing and relevant technological aspects. It focuses on four key areas including architecture, virtualization, data management, and security issues

Uma abordagem estratificada à monitorização de serviços Cloud

Dissertação de mestrado em Redes e Serviços de ComunicaçõesA monitorização é uma tarefa essencial na gestão e engenharia das redes de comunicações atuais. Face a paradigmas como Cloud Computing e Cloud Services, os desafios colocados à monitorização de redes e serviços são ainda mais variados e exigentes. Cloud Computing inclui modelos de serviços distintos (IaaS, PaaS, SaaS), compartilhando algumas necessidades comuns na medição de infraestruturas, mas com especificidades de acordo com o tipo de serviço prestado e recursos envolvidos. A complexidade associada à monitorização destes ambientes e a falta de normas até agora, exige uma análise cuidadosa e sistematizada, de modo a obter uma melhor compreensão acerca dos pontos-chave envolvidos na avaliação dos serviços prestados. Neste contexto, esta dissertação apresenta uma abordagem estratificada à monitorização de Serviços Cloud. O objetivo principal prende-se com a identificação das várias dimensões da monitorização de serviços Cloud, combinando as perspetivas do fornecedor de infraestruturas e de serviços, e dos clientes. Consequentemente, a monitorização do estado dos recursos, da qualidade de serviço, qualidade de experiência e contratos de serviço são aspetos a cobrir. Este processo envolve a identificação de parâmetros e métricas relevantes para cada dimensão monitorizada. Através de uma visão estratificada do problema, este estudo contribui para alcançar uma abordagem mais clara e eficiente à monitorização de serviços Cloud.Monitoring is an essential task in the management and engineering of today’s communication networks. To face paradigms such as cloud computing and cloud services, the challenges of monitoring networks and services are even more varied and demanding. Cloud Computing includes distinct service models (IaaS, PaaS, SaaS), sharing common needs in measuring the infrastructure, but with specificities according to the type of service and resources involved. The complexity of monitoring these environments and the lack of standards so far urge for a careful analysis, systematizing and understanding the key points involved when assessing the services provided. In this context, this dissertation presents a layered approach to Cloud Services monitoring. The main objective concerns the identification of the multiple dimensions of monitoring cloud services, combining the perspectives of providers and customers. Consequently, the monitoring of resources, quality of service, quality of experience and service contracts are aspects to cover. This process involves the identification of relevant parameters and metrics for each monitoring dimension. Taking a stratified view of the problem, this study contributes to achieve a clearer and more efficient approach to cloud services monitoring

Foundations and Technological Landscape of Cloud Computing

The cloud computing paradigm has brought the benefits of utility computing to a global scale. It has gained paramount attention in recent years. Companies are seriously considering to adopt this new paradigm and expecting to receive significant benefits. In fact, the concept of cloud computing is not a revolution in terms of technology; it has been established based on the solid ground of virtualization, distributed system, and web services. To comprehend cloud computing, its foundations and technological landscape need to be adequately understood. This paper provides a comprehensive review on the building blocks of cloud computing and relevant technological aspects. It focuses on four key areas including architecture, virtualization, data management, and security issues

Systematic support for accountability in the cloud

PhD ThesisCloud computing offers computational resources such as processing, networking, and storage to customers. Infrastructure as a Service (IaaS) consists of a cloud-based infrastructure to offer consumers raw computation resources such as storage and networking. These resources are billed using a pay-per-use cost model. However, IaaS is far from being a secure cloud infrastructure as the seven main security threats defined by the Cloud Security Alliance (CSA) indicate. Use of logging systems can provide evidence to support accountability for an IaaS cloud. An accountability helps when mitigating known threats. However, previous accountability with logging systems solutions are provided without systematic approaches. These solutions are usually either for the cloud customer side or for the cloud provider side, not for both of them. Moreover, the solutions also lack descriptions of logging systems in the context of a design pattern of the systems' components. This design pattern facilitates analysis of logging systems in terms of their quality. Additionally, there is a number of benefits of this pattern. They could be: to promote the reusability of design and development of logging systems; that designers can access this pattern more easily; to assist a designer adopts design approaches which make a logging system reusable and not to choose approaches which do not concern reusability concepts; and to enhance the documentation and maintenance of existing logging systems. Thus, the aim of this thesis is to provide support for accountability in the cloud with systematic approaches to assist in mitigating the risks associated with real world CSA threats, to benefit both customers and providers. We research the extent to which such logging systems help us to mitigate risks associated with the threats identified by the CSA. The thesis also presents a way of identifying the reference components of logging systems and how they may be arranged to satisfy logging requirements. 'Generic logging components' for logging systems are proposed. These components encompass all possible instantiations of logging solutions for IaaS cloud. The generic logging components can be used to map existing logging systems for the purposes of analysis of the systems' security. Based on the generic components, the thesis identifies design patterns in the context of logging in IaaS cloud. We believe that these identified patterns facilitate analysis of logging systems in terms of their quality. We also argue that: these identified patterns could increase reusability of the design and development of logging systems; designers should access these patterns more easily; the patterns could assist a designer adopts design approaches which make a logging system reusable and not to choose approaches which do not concern reusability concepts; and they can enhance the documentation and maintenance of existing logging systems. We identify a logging solution which is based on the generic logging components to mitigate the risks associated with CSA threat number one. An example of the threat is malicious activities, for example spamming, which are performed in consumers' virtual machines or VMs. We argue that the generic logging components we suggest could be used to perform a systematic analysis of logging systems in terms of security before deploying them in production systems. To assist in mitigating the risks associated with this threat to benefit both customers and providers, we investigate how CSA threat number one can affect the security of both consumers and providers. Then we propose logging solutions based on the generic logging components and the identified patterns. We systematically design and implement a prototype system of the proposed logging solutions in an IaaS to record history of customer's files. This prototype system can be also modified in order to record VMs' process behaviour log files. This system can record the log files while having a smaller trusted computing base, compared to previous work. Additionally, the system can be seen as possible solutions that could tackle the dificult problem of logging file and process activities in the IaaS. Thus, the proposed logging solutions can assist in mitigating the risks associated with the CSA threats to benefit both consumers and providers. This could promote systematic support for accountability in the cloud