Modeling and analysis of high availability techniques in a virtualized system

Availability evaluation of a virtualized system is critical to the wide deployment of cloud computing services. Time-based, prediction-based rejuvenation of virtual machines (VM) and virtual machine monitors, VM failover and live VM migration are common high-availability (HA) techniques in a virtualized system. This paper investigates the effect of combination of these availability techniques on VM availability in a virtualized system where various software and hardware failures may occur. For each combination, we construct analytic models rejuvenation mechanisms to improve VM availability; (2) prediction-based rejuvenation enhances VM availability much more than time-based VM rejuvenation when prediction successful probability is above 70%, regardless failover and/or live VM migration is also deployed; (3) failover mechanism outperforms live VM migration, although they can work together for higher availability of VM. In addition, they can combine with software rejuvenation mechanisms for even higher availability; (4) and time interval setting is critical to a time-based rejuvenation mechanism. These analytic results provide guidelines for deploying and parameter setting of HA techniques in a virtualized system

Fault-Tolerance in the Scope of Cloud Computing

Fault-tolerance methods are required to ensure high availability and high reliability in cloud computing environments. In this survey, we address fault-tolerance in the scope of cloud computing. Recently, cloud computing-based environments have presented new challenges to support fault-tolerance and opened new paths to develop novel strategies, architectures, and standards. We provide a detailed background of cloud computing to establish a comprehensive understanding of the subject, from basic to advanced. We then highlight fault-tolerance components and system-level metrics and identify the needs and applications of fault-tolerance in cloud computing. Furthermore, we discuss state-of-the-art proactive and reactive approaches to cloud computing fault-tolerance. We further structure and discuss current research efforts on cloud computing fault-tolerance architectures and frameworks. Finally, we conclude by enumerating future research directions specific to cloud computing fault-tolerance development.publishe

Proactive cloud service assurance framework for fault remediation in cloud environment

Cloud resiliency is an important issue in successful implementation of cloud computing systems. Handling cloud faults proactively, with a suitable remediation technique having minimum cost is an important requirement for a fault management system. The selection of best applicable remediation technique is a decision making problem and considers parameters such as i) Impact of remediation technique ii) Overhead of remediation technique ii) Severity of fault and iv) Priority of the application. This manuscript proposes an analytical model to measure the effectiveness of a remediation technique for various categories of faults, further it demonstrates the implementation of an efficient fault remediation system using a rule-based expert system. The expert system is designed to compute an utility value for each remediation technique in a novel way and select the best remediation technique from its knowledgebase. A prototype is developed for experimentation purpose and the results shows improved availability with less overhead as compared to a reactive fault management system

Proactive software rejuvenation solution for web enviroments on virtualized platforms

The availability of the Information Technologies for everything, from everywhere, at all times is a growing requirement. We use information Technologies from common and social tasks to critical tasks like managing nuclear power plants or even the International Space Station (ISS). However, the availability of IT infrastructures is still a huge challenge nowadays. In a quick look around news, we can find reports of corporate outage, affecting millions of users and impacting on the revenue and image of the companies. It is well known that, currently, computer system outages are more often due to software faults, than hardware faults. Several studies have reported that one of the causes of unplanned software outages is the software aging phenomenon. This term refers to the accumulation of errors, usually causing resource contention, during long running application executions, like web applications, which normally cause applications/systems to hang or crash. Gradual performance degradation could also accompany software aging phenomena. The software aging phenomena are often related to memory bloating/ leaks, unterminated threads, data corruption, unreleased file-locks or overruns. We can find several examples of software aging in the industry. The work presented in this thesis aims to offer a proactive and predictive software rejuvenation solution for Internet Services against software aging caused by resource exhaustion. To this end, we first present a threshold based proactive rejuvenation to avoid the consequences of software aging. This first approach has some limitations, but the most important of them it is the need to know a priori the resource or resources involved in the crash and the critical condition values. Moreover, we need some expertise to fix the threshold value to trigger the rejuvenation action. Due to these limitations, we have evaluated the use of Machine Learning to overcome the weaknesses of our first approach to obtain a proactive and predictive solution. Finally, the current and increasing tendency to use virtualization technologies to improve the resource utilization has made traditional data centers turn into virtualized data centers or platforms. We have used a Mathematical Programming approach to virtual machine allocation and migration to optimize the resources, accepting as many services as possible on the platform while at the same time, guaranteeing the availability (via our software rejuvenation proposal) of the services deployed against the software aging phenomena. The thesis is supported by an exhaustive experimental evaluation that proves the effectiveness and feasibility of our proposals for current systems

Multi-perspective Evaluation of Self-Healing Systems Using Simple Probabilistic Models

Quantifying the efficacy of self-healing systems is a challenging but important task, which has implications for increasing designer, operator and end-user confidence in these systems. During design system architects benefit from tools and techniques that enhance their understanding of the system, allowing them to reason about the tradeoffs of proposed or existing self-healing mechanisms and the overall effectiveness of the system as a result of different mechanism-compositions. At deployment time, system integrators and operators need to understand how the selfhealing mechanisms work and how their operation impacts the system's reliability, availability and serviceability (RAS) in order to cope with any limitations of these mechanisms when the system is placed into production. In this paper we construct an evaluation framework for selfhealing systems around simple, yet powerful, probabilistic models that capture the behavior of the system's selfhealing mechanisms from multiple perspectives (designer, operator, and end-user). We combine these analytical models with runtime fault-injection to study the operation of VM-Rejuv — a virtual machine based rejuvenation scheme for web-application servers. We use the results from the fault-injection experiments and model-analysis to reason about the efficacy of VM-Rejuv, its limitations and strategies for managing/mitigating these limitations in system deployments. Whereas we use VM-Rejuv as the subject of our evaluation in this paper, our main contribution is a practical evaluation approach that can be generalized to other self-healing systems

Multi-perspective Evaluation of Self-Healing Systems Using Simple Probabilistic Models

Quantifying the efficacy of self-healing systems is a challenging but important task, which has implications for increasing designer, operator and end-user confidence in these systems. During design system architects benefit from tools and techniques that enhance their understanding of the system, allowing them to reason about the tradeoffs of proposed or existing self-healing mechanisms and the overall effectiveness of the system as a result of different mechanism-compositions. At deployment time, system integrators and operators need to understand how the selfhealing mechanisms work and how their operation impacts the system's reliability, availability and serviceability (RAS) in order to cope with any limitations of these mechanisms when the system is placed into production. In this paper we construct an evaluation framework for selfhealing systems around simple, yet powerful, probabilistic models that capture the behavior of the system's selfhealing mechanisms from multiple perspectives (designer, operator, and end-user). We combine these analytical models with runtime fault-injection to study the operation of VM-Rejuv — a virtual machine based rejuvenation scheme for web-application servers. We use the results from the fault-injection experiments and model-analysis to reason about the efficacy of VM-Rejuv, its limitations and strategies for managing/mitigating these limitations in system deployments. Whereas we use VM-Rejuv as the subject of our evaluation in this paper, our main contribution is a practical evaluation approach that can be generalized to other self-healing systems

MACHS: Mitigating the Achilles Heel of the Cloud through High Availability and Performance-aware Solutions

Cloud computing is continuously growing as a business model for hosting information and communication technology applications. However, many concerns arise regarding the quality of service (QoS) offered by the cloud. One major challenge is the high availability (HA) of cloud-based applications. The key to achieving availability requirements is to develop an approach that is immune to cloud failures while minimizing the service level agreement (SLA) violations. To this end, this thesis addresses the HA of cloud-based applications from different perspectives. First, the thesis proposes a component’s HA-ware scheduler (CHASE) to manage the deployments of carrier-grade cloud applications while maximizing their HA and satisfying the QoS requirements. Second, a Stochastic Petri Net (SPN) model is proposed to capture the stochastic characteristics of cloud services and quantify the expected availability offered by an application deployment. The SPN model is then associated with an extensible policy-driven cloud scoring system that integrates other cloud challenges (i.e. green and cost concerns) with HA objectives. The proposed HA-aware solutions are extended to include a live virtual machine migration model that provides a trade-off between the migration time and the downtime while maintaining HA objective. Furthermore, the thesis proposes a generic input template for cloud simulators, GITS, to facilitate the creation of cloud scenarios while ensuring reusability, simplicity, and portability. Finally, an availability-aware CloudSim extension, ACE, is proposed. ACE extends CloudSim simulator with failure injection, computational paths, repair, failover, load balancing, and other availability-based modules

Diversity management in intrusion tolerant systems

Tese de mestrado em Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2011Uma aplicação importante dos protocolos de tolerância a faltas arbitrárias (ou Bizantinas) é a construção de sistemas tolerantes a intrusões, que são capazes de funcionar correctamente mesmo que alguns dos seus componentes sejam comprometidos. Estes sistemas são concretizados através da replicação de componentes e da utilização de protocolos capazes de tolerar faltas arbitrárias, quer na rede como num subconjunto de réplicas. Os protocolos garantem um comportamento correcto ainda que exista uma minoria (normalmente menor do que um terço) de componentes controlados por um adversário malicioso. Para cumprir esta condição os componentes do sistema têm de apresentar independência nas falhas. No entanto, quando estamos no contexto da segurança de sistemas, temos de admitir a possibilidade de ocorrerem ataques simultâneos contra várias réplicas. Se os vários componentes tiverem as mesmas vulnerabilidades, então podem ser comprometidos com um só ataque, o que destrói o propósito de se construir sistemas tolerantesa intrusões. Com o objectivo de reduzir a probabilidade de existirem vulnerabilidades comuns pretendemos utilizar diversidade: cada componente usa software distinto que fornece as mesmas funcionalidades, com a expectativa de que as diferenças vão reduzir o número de vulnerabilidades semelhantes. Reconhecemos também que isoladamente a tolerância a faltas arbitrárias tem algumas limitações uma vez que consideramos faltas maliciosas: uma das limitações mais importantes é que dado tempo suficiente o adversário pode comprometer f + 1 réplicas, e então violar a hipótese de que no máximo f componentes podem sofrer uma falta, levando os recursos do sistema à exaustão. Uma forma de lidar com esta limitação consiste em renovar periodicamente as réplicas, uma técnica denominada por Recuperação Proactiva. O propósito das recuperações é limpar o estado do sistema, reiniciando a replica com código disponível em armazenamento apenas com permissões de leitura (ex: CD-ROM) e validar/obter o estado de outro componente (que seja correcto). Num sistema tolerante a intrusões com recuperação proactiva o intervalo temporal que o adversário tem para comprometer f + 1 réplicas passa a ser uma pequena janela de vulnerabilidade, que compreende o tempo de recuperação do sistema todo. Apesar dos benefícios que as recuperações periódicas oferecem em termos de fiabilidade persiste a seguinte dificuldade: as vulnerabilidades exploradas nas execuções anteriores da replica podem ainda ser exploradas depois da recuperação. Esta limitação permite facilmente a um atacante criar um script que automaticamente compromete novamente a replica logo a seguir à sua recuperação pois as vulnerabilidades não são apagadas mas sim a falta (i.e., o estado incorrecto no componente devido à intrusão). Com o objectivo de melhorar o sistema introduzimos diversidade nas recuperações, mais precisamente em componentes off-the-shelf (OTS). Hoje em dia praticamente todo o software desenvolvido é baseado neste tipo de componentes, como por exemplo sistemas operativos (SO) e gestores de bases de dados. Isto deve-se principalmente à complexidade do desenvolvimento destes componentes em conjugação com os benefícios relacionados com o baixo custo, a instalação rápida e a variedade de opções disponíveis. No entanto, a maior parte dos componentes OTS não foram desenhados com segurança como prioridade, o que significa que em todos eles existem vulnerabilidades que podem ser maliciosamente exploradas. Por vezes, sistemas supostamente seguros são comprometidos através de uma componente critica na sua infraestrutura. Por outro lado, dada a quantidade de oferta das componentes OTS, utilizar diversidade nestes componentes é menos complexo e tem um menor custo do que desenvolver várias componentes de software diferentes. Um bom exemplo disto é o caso dos SO: as organizações na verdade preferem um sistema operativo OTS do que construir o seu próprio SO. Dada a variedade de sistemas operativos disponíveis e a criticidade do papel desempenhado por estes em qualquer computador, a diversidade ao nível dos SO pode ser uma forma razoável de garantir segurança contra vulnerabilidades comuns com um baixo custo adicional. O foco nas vulnerabilidades comuns é um aspecto importante deste trabalho. Visto que a tolerância intrusões ´e aplicada em sistemas críticos, ´e seguro afirmar que no sistema operativo vai ser assegurada a máxima segurança, aplicando todos os patches disponíveis. No entanto, mesmo com sistemas actualizados, o sistema pode ser comprometido através de vulnerabilidades que ainda não foram descobertas pelos programadores (vulnerabilidades de dia zero), visto que os patches aparecem normalmente depois da vulnerabilidade ser anunciada. Se uma vulnerabilidade de dia zero afectar o sistema existe uma janela de oportunidade para o atacante causar uma intrusão. A questão principal que tratamos na primeira parte desta tese é: Quais são os ganhos de se aplicar diversidade de SO num sistema tolerante a intrusões replicado? Para responder a esta questão, recolhemos e selecionámos dados sobre vulnerabilidades do NIST National Vulnerability Database (NVD) entre 1994 e 2010 para 11 sistemas operativos. Os dados do NVD relativamente aos SO são consideráveis, o que nos permite tirar algumas conclusões. Cada vulnerabilidade presente no NVD contém (entre outras coisas) informação sobre que produtos são afectados pela vulnerabilidade. Recolhemos estes dados e verificámos quantas vulnerabilidades afectam mais do que um sistema operativo. Constatámos que este número é relativamente pequeno para a maior parte de de sistemas operativos. Este estudo foi depois estendido a um número maior de SO, com conclusões semelhantes para esses conjuntos. Estes resultados sugerem que existem ganhos de segurança que podem ser alcançados recorrendo à utilização de sistemas operativos diferentes num sistema replicado. Como nota de cautela, não pretendemos afirmar que estes resultados são uma prova final sobre a ausência de vulnerabilidades comuns (embora sejam bastante promissores). Um dos principais problemas encontrados é que os relatórios focam-se nas vulnerabilidades e não em quantas intrusões ou exploits ocorreram para cada vulnerabilidade; isto faz com que a avaliação, em termos de segurança, seja mais difícil. A segunda parte da tese propõe uma arquitectura que explora a diversidade disponível nos sistemas operativos juntamente com mecanismos de recuperação proactiva. O objectivo principal é mudar a configuração das réplicas de forma a alterar o conjunto de vulnerabilidades após uma recuperação. Desenvolvemos também um algoritmo que seleciona entre os candidatos o melhor sistema operativo para ser usado numa recuperação, assegurando o maior nível de diversidade possível entre as réplicas que se encontram em execução.One of the key benefits of using intrusion-tolerant systems is the possibility of ensuring correct behavior in the presence of attacks and intrusions. These security gains are directly dependent on the components exhibiting failure diversity. To what extent failure diversity is observed in practical deployment depends on how diverse are the components that constitute the system. In this thesis we present a study with operating systems (OS) vulnerability reports from the NIST National Vulnerability Database. We have analyzed the vulnerabilities of 11 different OS over a period of roughly 15 years, to check how many of these vulnerabilities occur in more than one OS. We found this number to be low for several combinations of OS. Hence, our analysis provides a strong indication that building a system with diverse OS may be a useful technique to improve its intrusion tolerance capabilities. However, even with diversity the attacker eventually will find vulnerabilities in all OS replicas. To mitigate/eliminate this problem we introduce diverse proactive recovery on the replicas. Proactive recovery is a technique that periodically rejuvenates the components of a replicated system. When used in the context of intrusiontolerant systems, in which faulty replicas may be under control of some malicious user, it allows the removal of intrusions from the compromised replicas. We propose that after each recovery a replica starts to run a different software. The selection of the new replica configuration is a non-trivial problem, as we will explain, since we would like to maximize the diversity of the system under the constraint of the available configurations