Design, Implementation and Experiments for Moving Target Defense Framework

The traditional defensive security strategy for distributed systems employs well-established defensive techniques such as; redundancy/replications, firewalls, and encryption to prevent attackers from taking control of the system. However, given sufficient time and resources, all these methods can be defeated, especially when dealing with sophisticated attacks from advanced adversaries that leverage zero-day exploits

High availability using virtualization

High availability has always been one of the main problems for a data center. Till now high availability was achieved by host per host redundancy, a highly expensive method in terms of hardware and human costs. A new approach to the problem can be offered by virtualization. Using virtualization, it is possible to achieve a redundancy system for all the services running on a data center. This new approach to high availability allows to share the running virtual machines over the servers up and running, by exploiting the features of the virtualization layer: start, stop and move virtual machines between physical hosts. The system (3RC) is based on a finite state machine with hysteresis, providing the possibility to restart each virtual machine over any physical host, or reinstall it from scratch. A complete infrastructure has been developed to install operating system and middleware in a few minutes. To virtualize the main servers of a data center, a new procedure has been developed to migrate physical to virtual hosts. The whole Grid data center SNS-PISA is running at the moment in virtual environment under the high availability system. As extension of the 3RC architecture, several storage solutions have been tested to store and centralize all the virtual disks, from NAS to SAN, to grant data safety and access from everywhere. Exploiting virtualization and ability to automatically reinstall a host, we provide a sort of host on-demand, where the action on a virtual machine is performed only when a disaster occurs.Comment: PhD Thesis in Information Technology Engineering: Electronics, Computer Science, Telecommunications, pp. 94, University of Pisa [Italy

Elastic Resource Management in Distributed Clouds

The ubiquitous nature of computing devices and their increasing reliance on remote resources have driven and shaped public cloud platforms into unprecedented large-scale, distributed data centers. Concurrently, a plethora of cloud-based applications are experiencing multi-dimensional workload dynamics---workload volumes that vary along both time and space axes and with higher frequency. The interplay of diverse workload characteristics and distributed clouds raises several key challenges for efficiently and dynamically managing server resources. First, current cloud platforms impose certain restrictions that might hinder some resource management tasks. Second, an application-agnostic approach might not entail appropriate performance goals, therefore, requires numerous specific methods. Third, provisioning resources outside LAN boundary might incur huge delay which would impact the desired agility. In this dissertation, I investigate the above challenges and present the design of automated systems that manage resources for various applications in distributed clouds. The intermediate goal of these automated systems is to fully exploit potential benefits such as reduced network latency offered by increasingly distributed server resources. The ultimate goal is to improve end-to-end user response time with novel resource management approaches, within a certain cost budget. Centered around these two goals, I first investigate how to optimize the location and performance of virtual machines in distributed clouds. I use virtual desktops, mostly serving a single user, as an example use case for developing a black-box approach that ranks virtual machines based on their dynamic latency requirements. Those with high latency sensitivities have a higher priority of being placed or migrated to a cloud location closest to their users. Next, I relax the assumption of well-provisioned virtual machines and look at how to provision enough resources for applications that exhibit both temporal and spatial workload fluctuations. I propose an application-agnostic queueing model that captures the resource utilization and server response time. Building upon this model, I present a geo-elastic provisioning approach---referred as geo-elasticity---for replicable multi-tier applications that can spin up an appropriate amount of server resources in any cloud locations. Last, I explore the benefits of providing geo-elasticity for database clouds, a popular platform for hosting application backends. Performing geo-elastic provisioning for backend database servers entails several challenges that are specific to database workload, and therefore requires tailored solutions. In addition, cloud platforms offer resources at various prices for different locations. Towards this end, I propose a cost-aware geo-elasticity that combines a regression-based workload model and a queueing network capacity model for database clouds. In summary, hosting a diverse set of applications in an increasingly distributed cloud makes it interesting and necessary to develop new, efficient and dynamic resource management approaches

Migration of networks

Tese de mestrado em Engenharia Informática, Universidade de Lisboa, Faculdade de Ciências, 2021A forma como os recursos computacionais são geridos, mais propriamente os alojados nos grandes centros de dados, tem vindo, nos últimos anos, a evoluir. As soluções iniciais que passavam por aplicações a correr em grandes servidores físicos, comportavam elevados custos não só de aquisição, mas também, e principalmente, de manutenção. A razão chave por trás deste facto prendia-se em grande parte com uma utilização largamente ineficiente dos recursos computacionais disponíveis. No entanto, o surgimento de tecnologias de virtualização de servidores foi o volte-face necessário para alterar radicalmente o paradigma até aqui existente. Isto não só levou a que os operadores dos grandes centros de dados pudessem passar a alugar os seus recursos computacionais, criando assim uma interessante oportunidade de negócio, mas também permitiu potenciar (e facilitar) negócios dos clientes. Do ponto de vista destes, os benefícios são evidentes: poder alugar recursos, num modelo pay-as-you-go, evita os elevados custos de capital necessários para iniciar um novo serviço. A este novo conceito baseado no aluguer e partilha de recursos computacionais a terceiros dá-se o nome de computação em nuvem (“cloud computing”). Como referimos anteriormente, nada disto teria sido possível sem o aparecimento de tecnologias de virtualização, que permitem o desacoplamento dos serviços dos utilizadores do hardware que os suporta. Esta tecnologia tem-se revelado uma ferramenta fundamental na administração e manutenção dos recursos disponíveis em qualquer centro de dados. Por exemplo, a migração de máquinas virtuais facilita tarefas como a manutenção das infraestruturas, a distribuição de carga, a tolerância a faltas, entre outras primitivas operacionais, graças ao desacoplamento entre as máquinas virtuais e as máquinas físicas, e à consequente grande mobilidade que lhes é assim conferida. Atualmente, muitas aplicações e serviços alojados na nuvem apresentam dimensão e complexidade considerável. O serviço típico é composto por diversos componentes que se complementam de forma a cumprir um determinado propósito. Por exemplo, diversos serviços são baseados numa topologia de vários níveis, composta por múltiplos servidores web, balanceadores de carga e bases de dados distribuídas e replicadas. Daqui resulta uma forte ligação e dependência dos vários elementos deste sistema e das infraestruturas de comunicação e de rede que os suportam. Esta forte dependência da rede vem limitar grandemente a flexibilidade e mobilidade das máquinas virtuais, o que, por sua vez, restringe inevitavelmente o seu reconhecido potencial. Esta dependência é particularmente afetada pela reduzida flexibilidade que a gestão e o controlo das redes apresentam atualmente, levando a que o processo de migração de máquinas virtuais se torne num demorado processo que apresenta restrições que obrigam à reconfiguração da rede, operação esta que, muitas vezes, é assegurada por um operador humano (de que pode resultar, por exemplo, a introdução de falhas). Num cenário ideal, a infraestrutura de redes de que depende a comunicação entre as máquinas virtuais seria também ela virtual, abstraindo os recursos necessários à comunicação, o que conferiria à globalidade do sistema uma maior flexibilidade e mobilidade que, por sua vez, permitiria a realização de uma migração conjunta das referidas máquinas virtuais e da infraestrutura de rede que as suporta. Neste contexto, surgem as redes definidas por software (SDN) [34], uma nova abordagem às redes de computadores que propõe separar a infraestrutura responsável pelo encaminhamento do tráfego (o plano de dados) do plano de controlo, planos que, até aqui, se encontravam acoplados nos elementos de rede (switches e routers). O controlo passa assim para um grupo de servidores, o que permite criar uma centralização lógica do controlo da rede. Uma SDN consegue então oferecer uma visão global da rede e do seu respetivo estado, característica fundamental para permitir o desacoplamento necessário entre a infraestrutura física e virtual. Recentemente, várias soluções de virtualização de rede foram propostas (e.g., VMware NSX [5], Microsoft AccelNet [21] e Google Andromeda [2]), ancoradas na centralização oferecida por uma SDN. No entanto, embora estas plataformas permitam virtualizar a rede, nenhuma delas trata o problema da migração dos seus elementos, limitando a sua flexibilidade. O objetivo desta dissertação passa então por implementar e avaliar soluções de migração de redes recorrendo a SDNs. A ideia é migrar um dispositivo de rede (neste caso, um switch virtual), escolhido pelo utilizador, de modo transparente, quer para os serviços que utilizam a rede, evitando causar disrupção, quer para as aplicações de controlo SDN da rede. O desafio passa por migrar o estado mantido no switch de forma consistente e sem afetar o normal funcionamento da rede. Com esse intuito, implementámos e avaliámos três diferentes abordagens à migração ( freeze and copy, move e clone) e discutimos as vantagens e desvantagens de cada uma. É de realçar que a solução baseada em clonagem se encontra incorporada como um módulo do virtualizador de rede Sirius.The way computational resources are managed, specifically those in big data centers, has been evolving in the last few years. One of the big stepping-stones for this was the emergence of server virtualization technologies that, given their ability to decouple software from the hardware, allowed for big data center operators to rent their resources, which, in its turn, represented an interesting business opportunity for both the operators and their potential customers. This new concept that consists in renting computational resources is called cloud computing. Furthermore, with the possibility that later arose of live migrating virtual machines, be it by customer request (for example, to move their service closer to the target consumer) or by provider decision (for example, to execute scheduled rack maintenances without downtimes), this new paradigm presented really strong arguments in comparison with traditional hosting solutions. Today, most cloud applications have considerable dimension and complexity. This complexity results in a strong dependency between the system elements and the communication infrastructure that lays underneath. This strong network dependency greatly limits the flexibility and mobility of the virtual machines (VMs). This dependency is mainly due to the reduced flexibility of current network management and control, turning the VM migration process into a long and error prone procedure. From a network’s perspective however, software-defined networks (SDNs) [34] manage to provide tools and mechanisms that can go a long way to mitigate this limitation. SDN proposes the separation of the forwarding infrastructure from the control plane as a way to tackle the flexibility problem. Recently, several network virtualization solutions were proposed (e.g., VMware NSX [5], Microsoft AccelNet [21] and Google Andromeda [2]), all supported on the logical centralization offered by an SDN. However, while allowing for network virtualization, none of these platforms addressed the problem of migrating the virtual networks, which limits their functionality. The goal of this dissertation is to implement and evaluate network migration solutions using SDNs. These solutions should allow for the migration of a network element (a virtual switch), chosen by the user, transparently, both for the services that are actively using the network and for the SDN applications that control the network. The challenge is to migrate the virtual element’s state in a consistent manner, whilst not affecting the normal operation of the network. With that in mind, we implemented and evaluated three different migration approaches (freeze and copy, move and clone), and discussed their respective advantages and disadvantages. It is relevant to mention that the cloning approach we implemented and evaluated is incorporated as a module of the network virtualization platform Sirius

Live migration of user environments across wide area networks

A complex challenge in mobile computing is to allow the user to migrate her highly customised environment while moving to a different location and to continue work without interruption. I motivate why this is a highly desirable capability and conduct a survey of the current approaches towards this goal and explain their limitations. I then propose a new architecture to support user mobility by live migration of a user’s operating system instance over the network. Previous work includes the Collective and Internet Suspend/Resume projects that have addressed migration of a user’s environment by suspending the running state and resuming it at a later time. In contrast to previous work, this work addresses live migration of a user’s operating system instance across wide area links. Live migration is done by performing most of the migration while the operating system is still running, achieving very little downtime and preserving all network connectivity. I developed an initial proof of concept of this solution. It relies on migrating whole operating systems using the Xen virtual machine and provides a way to perform live migration of persistent storage as well as the network connections across subnets. These challenges have not been addressed previously in this scenario. In a virtual machine environment, persistent storage is provided by virtual block devices. The architecture supports decentralized virtual block device replication across wide area network links, as well as migrating network connection across subnetworks using the Host Identity Protocol. The proposed architecture is compared against existing solutions and an initial performance evaluation of the prototype implementation is presented, showing that such a solution is a promising step towards true seamless mobility of fully fledged computing environments

Diversity management in intrusion tolerant systems

Tese de mestrado em Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2011Uma aplicação importante dos protocolos de tolerância a faltas arbitrárias (ou Bizantinas) é a construção de sistemas tolerantes a intrusões, que são capazes de funcionar correctamente mesmo que alguns dos seus componentes sejam comprometidos. Estes sistemas são concretizados através da replicação de componentes e da utilização de protocolos capazes de tolerar faltas arbitrárias, quer na rede como num subconjunto de réplicas. Os protocolos garantem um comportamento correcto ainda que exista uma minoria (normalmente menor do que um terço) de componentes controlados por um adversário malicioso. Para cumprir esta condição os componentes do sistema têm de apresentar independência nas falhas. No entanto, quando estamos no contexto da segurança de sistemas, temos de admitir a possibilidade de ocorrerem ataques simultâneos contra várias réplicas. Se os vários componentes tiverem as mesmas vulnerabilidades, então podem ser comprometidos com um só ataque, o que destrói o propósito de se construir sistemas tolerantesa intrusões. Com o objectivo de reduzir a probabilidade de existirem vulnerabilidades comuns pretendemos utilizar diversidade: cada componente usa software distinto que fornece as mesmas funcionalidades, com a expectativa de que as diferenças vão reduzir o número de vulnerabilidades semelhantes. Reconhecemos também que isoladamente a tolerância a faltas arbitrárias tem algumas limitações uma vez que consideramos faltas maliciosas: uma das limitações mais importantes é que dado tempo suficiente o adversário pode comprometer f + 1 réplicas, e então violar a hipótese de que no máximo f componentes podem sofrer uma falta, levando os recursos do sistema à exaustão. Uma forma de lidar com esta limitação consiste em renovar periodicamente as réplicas, uma técnica denominada por Recuperação Proactiva. O propósito das recuperações é limpar o estado do sistema, reiniciando a replica com código disponível em armazenamento apenas com permissões de leitura (ex: CD-ROM) e validar/obter o estado de outro componente (que seja correcto). Num sistema tolerante a intrusões com recuperação proactiva o intervalo temporal que o adversário tem para comprometer f + 1 réplicas passa a ser uma pequena janela de vulnerabilidade, que compreende o tempo de recuperação do sistema todo. Apesar dos benefícios que as recuperações periódicas oferecem em termos de fiabilidade persiste a seguinte dificuldade: as vulnerabilidades exploradas nas execuções anteriores da replica podem ainda ser exploradas depois da recuperação. Esta limitação permite facilmente a um atacante criar um script que automaticamente compromete novamente a replica logo a seguir à sua recuperação pois as vulnerabilidades não são apagadas mas sim a falta (i.e., o estado incorrecto no componente devido à intrusão). Com o objectivo de melhorar o sistema introduzimos diversidade nas recuperações, mais precisamente em componentes off-the-shelf (OTS). Hoje em dia praticamente todo o software desenvolvido é baseado neste tipo de componentes, como por exemplo sistemas operativos (SO) e gestores de bases de dados. Isto deve-se principalmente à complexidade do desenvolvimento destes componentes em conjugação com os benefícios relacionados com o baixo custo, a instalação rápida e a variedade de opções disponíveis. No entanto, a maior parte dos componentes OTS não foram desenhados com segurança como prioridade, o que significa que em todos eles existem vulnerabilidades que podem ser maliciosamente exploradas. Por vezes, sistemas supostamente seguros são comprometidos através de uma componente critica na sua infraestrutura. Por outro lado, dada a quantidade de oferta das componentes OTS, utilizar diversidade nestes componentes é menos complexo e tem um menor custo do que desenvolver várias componentes de software diferentes. Um bom exemplo disto é o caso dos SO: as organizações na verdade preferem um sistema operativo OTS do que construir o seu próprio SO. Dada a variedade de sistemas operativos disponíveis e a criticidade do papel desempenhado por estes em qualquer computador, a diversidade ao nível dos SO pode ser uma forma razoável de garantir segurança contra vulnerabilidades comuns com um baixo custo adicional. O foco nas vulnerabilidades comuns é um aspecto importante deste trabalho. Visto que a tolerância intrusões ´e aplicada em sistemas críticos, ´e seguro afirmar que no sistema operativo vai ser assegurada a máxima segurança, aplicando todos os patches disponíveis. No entanto, mesmo com sistemas actualizados, o sistema pode ser comprometido através de vulnerabilidades que ainda não foram descobertas pelos programadores (vulnerabilidades de dia zero), visto que os patches aparecem normalmente depois da vulnerabilidade ser anunciada. Se uma vulnerabilidade de dia zero afectar o sistema existe uma janela de oportunidade para o atacante causar uma intrusão. A questão principal que tratamos na primeira parte desta tese é: Quais são os ganhos de se aplicar diversidade de SO num sistema tolerante a intrusões replicado? Para responder a esta questão, recolhemos e selecionámos dados sobre vulnerabilidades do NIST National Vulnerability Database (NVD) entre 1994 e 2010 para 11 sistemas operativos. Os dados do NVD relativamente aos SO são consideráveis, o que nos permite tirar algumas conclusões. Cada vulnerabilidade presente no NVD contém (entre outras coisas) informação sobre que produtos são afectados pela vulnerabilidade. Recolhemos estes dados e verificámos quantas vulnerabilidades afectam mais do que um sistema operativo. Constatámos que este número é relativamente pequeno para a maior parte de de sistemas operativos. Este estudo foi depois estendido a um número maior de SO, com conclusões semelhantes para esses conjuntos. Estes resultados sugerem que existem ganhos de segurança que podem ser alcançados recorrendo à utilização de sistemas operativos diferentes num sistema replicado. Como nota de cautela, não pretendemos afirmar que estes resultados são uma prova final sobre a ausência de vulnerabilidades comuns (embora sejam bastante promissores). Um dos principais problemas encontrados é que os relatórios focam-se nas vulnerabilidades e não em quantas intrusões ou exploits ocorreram para cada vulnerabilidade; isto faz com que a avaliação, em termos de segurança, seja mais difícil. A segunda parte da tese propõe uma arquitectura que explora a diversidade disponível nos sistemas operativos juntamente com mecanismos de recuperação proactiva. O objectivo principal é mudar a configuração das réplicas de forma a alterar o conjunto de vulnerabilidades após uma recuperação. Desenvolvemos também um algoritmo que seleciona entre os candidatos o melhor sistema operativo para ser usado numa recuperação, assegurando o maior nível de diversidade possível entre as réplicas que se encontram em execução.One of the key benefits of using intrusion-tolerant systems is the possibility of ensuring correct behavior in the presence of attacks and intrusions. These security gains are directly dependent on the components exhibiting failure diversity. To what extent failure diversity is observed in practical deployment depends on how diverse are the components that constitute the system. In this thesis we present a study with operating systems (OS) vulnerability reports from the NIST National Vulnerability Database. We have analyzed the vulnerabilities of 11 different OS over a period of roughly 15 years, to check how many of these vulnerabilities occur in more than one OS. We found this number to be low for several combinations of OS. Hence, our analysis provides a strong indication that building a system with diverse OS may be a useful technique to improve its intrusion tolerance capabilities. However, even with diversity the attacker eventually will find vulnerabilities in all OS replicas. To mitigate/eliminate this problem we introduce diverse proactive recovery on the replicas. Proactive recovery is a technique that periodically rejuvenates the components of a replicated system. When used in the context of intrusiontolerant systems, in which faulty replicas may be under control of some malicious user, it allows the removal of intrusions from the compromised replicas. We propose that after each recovery a replica starts to run a different software. The selection of the new replica configuration is a non-trivial problem, as we will explain, since we would like to maximize the diversity of the system under the constraint of the available configurations

Reliable and energy efficient resource provisioning in cloud computing systems

Cloud Computing has revolutionized the Information Technology sector by giving computing a perspective of service. The services of cloud computing can be accessed by users not knowing about the underlying system with easy-to-use portals. To provide such an abstract view, cloud computing systems have to perform many complex operations besides managing a large underlying infrastructure. Such complex operations confront service providers with many challenges such as security, sustainability, reliability, energy consumption and resource management. Among all the challenges, reliability and energy consumption are two key challenges focused on in this thesis because of their conflicting nature. Current solutions either focused on reliability techniques or energy efficiency methods. But it has been observed that mechanisms providing reliability in cloud computing systems can deteriorate the energy consumption. Adding backup resources and running replicated systems provide strong fault tolerance but also increase energy consumption. Reducing energy consumption by running resources on low power scaling levels or by reducing the number of active but idle sitting resources such as backup resources reduces the system reliability. This creates a critical trade-off between these two metrics that are investigated in this thesis. To address this problem, this thesis presents novel resource management policies which target the provisioning of best resources in terms of reliability and energy efficiency and allocate them to suitable virtual machines. A mathematical framework showing interplay between reliability and energy consumption is also proposed in this thesis. A formal method to calculate the finishing time of tasks running in a cloud computing environment impacted with independent and correlated failures is also provided. The proposed policies adopted various fault tolerance mechanisms while satisfying the constraints such as task deadlines and utility values. This thesis also provides a novel failure-aware VM consolidation method, which takes the failure characteristics of resources into consideration before performing VM consolidation. All the proposed resource management methods are evaluated by using real failure traces collected from various distributed computing sites. In order to perform the evaluation, a cloud computing framework, 'ReliableCloudSim' capable of simulating failure-prone cloud computing systems is developed. The key research findings and contributions of this thesis are: 1. If the emphasis is given only to energy optimization without considering reliability in a failure prone cloud computing environment, the results can be contrary to the intuitive expectations. Rather than reducing energy consumption, a system ends up consuming more energy due to the energy losses incurred because of failure overheads. 2. While performing VM consolidation in a failure prone cloud computing environment, a significant improvement in terms of energy efficiency and reliability can be achieved by considering failure characteristics of physical resources. 3. By considering correlated occurrence of failures during resource provisioning and VM allocation, the service downtime or interruption is reduced significantly by 34% in comparison to the environments with the assumption of independent occurrence of failures. Moreover, measured by our mathematical model, the ratio of reliability and energy consumption is improved by 14%

Diverse Intrusion-tolerant Systems

Over the past 20 years, there have been indisputable advances on the development of Byzantine Fault-Tolerant (BFT) replicated systems. These systems keep operational safety as long as at most f out of n replicas fail simultaneously. Therefore, in order to maintain correctness it is assumed that replicas do not suffer from common mode failures, or in other words that replicas fail independently. In an adversarial setting, this requires that replicas do not include similar vulnerabilities, or otherwise a single exploit could be employed to compromise a significant part of the system. The thesis investigates how this assumption can be substantiated in practice by exploring diversity when managing the configurations of replicas. The thesis begins with an analysis of a large dataset of vulnerability information to get evidence that diversity can contribute to failure independence. In particular, we used the data from a vulnerability database to devise strategies for building groups of n replicas with different Operating Systems (OS). Our results demonstrate that it is possible to create dependable configurations of OSes, which do not share vulnerabilities over reasonable periods of time (i.e., a few years). Then, the thesis proposes a new design for a firewall-like service that protects and regulates the access to critical systems, and that could benefit from our diversity management approach. The solution provides fault and intrusion tolerance by implementing an architecture based on two filtering layers, enabling efficient removal of invalid messages at early stages in order to decrease the costs associated with BFT replication in the later stages. The thesis also presents a novel solution for managing diverse replicas. It collects and processes data from several data sources to continuously compute a risk metric. Once the risk increases, the solution replaces a potentially vulnerable replica by another one, trying to maximize the failure independence of the replicated service. Then, the replaced replica is put on quarantine and updated with the available patches, to be prepared for later re-use. We devised various experiments that show the dependability gains and performance impact of our prototype, including key benchmarks and three BFT applications (a key-value store, our firewall-like service, and a blockchain).Unidade de investigação LASIGE (UID/CEC/00408/2019) e o projeto PTDC/EEI-SCR/1741/2041 (Abyss

Resilient and Trustworthy Dynamic Data-driven Application Systems (DDDAS) Services for Crisis Management Environments

Future crisis management systems needresilient and trustworthy infrastructures to quickly develop reliable applications and processes, andensure end-to-end security, trust, and privacy. Due to the multiplicity and diversity of involved actors, volumes of data, and heterogeneity of shared information;crisis management systems tend to be highly vulnerable and subjectto unforeseen incidents. As a result, the dependability of crisis management systems can be at risk. This paper presents a cloud-based resilient and trustworthy infrastructure (known as rDaaS) to quickly develop secure crisis management systems. The rDaaS integrates the Dynamic Data-Driven Application Systems (DDDAS) paradigm into a service-oriented architecture over cloud technology and provides a set of resilient DDDAS-As-A Service (rDaaS) components to build secure and trusted adaptable crisis processes. The rDaaS also ensures resilience and security by obfuscating the execution environment and applying Behavior Software Encryption and Moving Technique Defense. A simulation environment for a nuclear plant crisis management case study is illustrated to build resilient and trusted crisis response processes

Resource-Efficient Replication and Migration of Virtual Machines.

Continuous replication and live migration of Virtual Machines (VMs) are two vital tools in a virtualized environment, but they are resource-expensive. Continuously replicating a VM's checkpointed state to a backup host maintains high-availability (HA) of the VM despite host failures, but checkpoint replication can generate significant network traffic. Each replicated VM also incurs a 100% memory overhead, since the backup unproductively reserves the same amount of memory to hold the redundant VM state. Live migration, though being widely used for load-balancing, power-saving, etc., can also generate excessive network traffic, by transferring VM state iteratively. In addition, it can incur a long completion time and degrade application performance. This thesis explores ways to replicate VMs for HA using resources efficiently, and to migrate VMs fast, with minimal execution disruption and using resources efficiently. First, we investigate the tradeoffs in using different compression methods to reduce the network traffic of checkpoint replication in a HA system. We evaluate gzip, delta and similarity compressions based on metrics that are specifically important in a HA system, and then suggest guidelines for their selection. Next, we propose HydraVM, a storage-based HA approach that eliminates the unproductive memory reservation made in backup hosts. HydraVM maintains a recent image of a protected VM in a shared storage by taking and consolidating incremental VM checkpoints. When a failure occurs, HydraVM quickly resumes the execution of a failed VM by loading a small amount of essential VM state from the storage. As the VM executes, the VM state not yet loaded is supplied on-demand. Finally, we propose application-assisted live migration, which skips transfer of VM memory that need not be migrated to execute running applications at the destination. We develop a generic framework for the proposed approach, and then use the framework to build JAVMM, a system that migrates VMs running Java applications skipping transfer of garbage in Java memory. Our evaluation results show that compared to Xen live migration, which is agnostic of running applications, JAVMM can reduce the completion time, network traffic and application downtime caused by Java VM migration, all by up to over 90%.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/111575/1/karenhou_1.pd