Stabilizing Server-Based Storage in Byzantine Asynchronous Message-Passing Systems

A stabilizing Byzantine single-writer single-reader (SWSR) regular register, which stabilizes after the first invoked write operation, is first presented. Then, new/old ordering inversions are eliminated by the use of a (bounded) sequence number for writes, obtaining a practically stabilizing SWSR atomic register. A practically stabilizing Byzantine single-writer multi-reader (SWMR) atomic register is then obtained by using several copies of SWSR atomic registers. Finally, bounded time-stamps, with a time-stamp per writer, together with SWMR atomic registers, are used to construct a practically stabilizing Byzantine multi-writer multi-reader (MWMR) atomic register. In a system of $n$ servers implementing an atomic register, and in addition to transient failures, the constructions tolerate t<n/8 Byzantine servers if communication is asynchronous, and t<n/3 Byzantine servers if it is synchronous. The noteworthy feature of the proposed algorithms is that (to our knowledge) these are the first that build an atomic read/write storage on top of asynchronous servers prone to transient failures, and where up to t of them can be Byzantine

Efficient and Modular Consensus-Free Reconfiguration for Fault-Tolerant Storage

Quorum systems are useful tools for implementing consistent and available storage in the presence of failures. These systems usually comprise of a static set of servers that provide a fault-tolerant read/write register accessed by a set of clients. We consider a dynamic variant of these systems and propose FreeStore, a set of fault-tolerant protocols that emulates a register in dynamic asynchronous systems in which processes are able to join/leave the set of servers during the execution. These protocols use a new abstraction called view generators, that captures the agreement requirements of reconfiguration and can be implemented in different system models with different properties. Particularly interesting, we present a reconfiguration protocol that is modular, efficient, consensus-free and loosely coupled with read/write protocols. An analysis and an experimental evaluation show that the proposed protocols improve the overall system performance when compared with previous solutions

Asynchronous Reconfiguration with Byzantine Failures

Replicated services are inherently vulnerable to failures and security breaches. In a long-running system, it is, therefore, indispensable to maintain a reconfiguration mechanism that would replace faulty replicas with correct ones. An important challenge is to enable reconfiguration without affecting the availability and consistency of the replicated data: the clients should be able to get correct service even when the set of service replicas is being updated. In this paper, we address the problem of reconfiguration in the presence of Byzantine failures: faulty replicas or clients may arbitrarily deviate from their expected behavior. We describe a generic technique for building asynchronous and Byzantine fault-tolerant reconfigurable objects: clients can manipulate the object data and issue reconfiguration calls without reaching consensus on the current configuration. With the help of forward-secure digital signatures, our solution makes sure that superseded and possibly compromised configurations are harmless, that slow clients cannot be fooled into reading stale data, and that Byzantine clients cannot cause a denial of service by flooding the system with reconfiguration requests. Our approach is modular and based on dynamic lattice agreement abstraction, and we discuss how to extend it to enable Byzantine fault-tolerant implementations of a large class of reconfigurable replicated services

Accountability and Reconfiguration: Self-Healing Lattice Agreement

An accountable distributed system provides means to detect deviations of system components from their expected behavior. It is natural to complement fault detection with a reconfiguration mechanism, so that the system could heal itself, by replacing malfunctioning parts with new ones. In this paper, we describe a framework that can be used to implement a large class of accountable and reconfigurable replicated services. We build atop the fundamental lattice agreement abstraction lying at the core of storage systems and cryptocurrencies. Our asynchronous implementation of accountable lattice agreement ensures that every violation of consistency is followed by an undeniable evidence of misbehavior of a faulty replica. The system can then be seamlessly reconfigured by evicting faulty replicas, adding new ones and merging inconsistent states. We believe that this paper opens a direction towards asynchronous "self-healing" systems that combine accountability and reconfiguration

Automatic Reconfiguration for Large-Scale Reliable Storage Systems

Byzantine-fault-tolerant replication enhances the availability and reliability of Internet services that store critical state and preserve it despite attacks or software errors. However, existing Byzantine-fault-tolerant storage systems either assume a static set of replicas, or have limitations in how they handle reconfigurations (e.g., in terms of the scalability of the solutions or the consistency levels they provide). This can be problematic in long-lived, large-scale systems where system membership is likely to change during the system lifetime. In this paper, we present a complete solution for dynamically changing system membership in a large-scale Byzantine-fault-tolerant system. We present a service that tracks system membership and periodically notifies other system nodes of membership changes. The membership service runs mostly automatically, to avoid human configuration errors; is itself Byzantine-fault-tolerant and reconfigurable; and provides applications with a sequence of consistent views of the system membership. We demonstrate the utility of this membership service by using it in a novel distributed hash table called dBQS that provides atomic semantics even across changes in replica sets. dBQS is interesting in its own right because its storage algorithms extend existing Byzantine quorum protocols to handle changes in the replica set, and because it differs from previous DHTs by providing Byzantine fault tolerance and offering strong semantics. We implemented the membership service and dBQS. Our results show that the approach works well, in practice: the membership service is able to manage a large system and the cost to change the system membership is low

Protocolos tolerantes a faltas bizantinas para sistemas distribuídos dinâmicos

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia de Automação e Sistemas, Florianópolis, 2011As novas tecnologias de comunicação e a maior disponibilidade de recursos em redes de comunicação vêm provocando profundas mudanças na forma de se projetar aplicações distribuídas. Estas mudanças possibilitaram o surgimento dos sistemas distribuídos dinâmicos, que se caracterizam por serem sistemas onde os componentes podem entrar e sair do mesmo em qualquer momento. Sendo assim, os protocolos desenvolvidos para estes sistemas devem detectar e tratar mudanças que ocorrem na composição da aplicação distribuída, permitindo sua reconfiguração em tempo de execução. Desta forma, os participantes destas aplicações são caracterizados principalmente pela heterogeneidade e não confiabilidade. No desenvolvimento de aplicações distribuídas seguras e confiáveis, vários problemas são identificados como peças fundamentais por formarem a base para a maioria das soluções empregadas nestas aplicações. Neste sentido, surge a necessidade do desenvolvimento de protocolos que resolvem estes problemas em sistemas distribuídos dinâmicos. Como os participantes destas computações não são confiáveis, torna-se essencial que estes protocolos suportem os atributos de segurança de funcionamento. Esta tese apresenta estudos e protocolos para a solução dos principais destes problemas fundamentais em sistemas distribuídos dinâmicos, os quais são: o problema do consenso, sistemas de quóruns e replicação Máquina de Estados. O problema do consenso é estudado em redes desconhecidas, onde são definidas as condições necessárias e suficientes para resolver o consenso. Estas condições especificam o grau de conhecimento sobre a composição do sistema dinâmico que deve ser obtido pelos participantes e o nível de sincronia que deve ser observado no mesmo. A segunda contribuição desta tese é formada principalmente por um conjunto de protocolos para reconfiguração de sistemas de quóruns, os quais podem ser divididos em: (1) algoritmos para inicialização da reconfiguração; (2) algoritmos para geração de novas visões do sistema; e (3) algoritmos para instalação das visões atualizadas. Várias combinações destes algoritmos são possíveis, resultando em um sistema com diferentes características e garantias. Estes protocolos são completamente desacoplados dos protocolos de leitura e escrita no registrador, facilitanto a integração dos mesmos com os mais variados sistemas de quóruns encontrados na literatura, além de aumentar o desempenho do sistema. A última grande contribuição desta tese refere-se à adição de suporte à reconfiguração em replicação Máquina de Estados. Nestas reconfigurações, tanto o conjunto de participantes do sistema quanto parâmetros da replicação podem ser alterados, resultando em um sistema bastante robusto e flexível. Todas as soluções propostas nesta tese suportam a presença de participantes maliciosos no sistema e fornecem protocolos que incorporam os atributos de confiabilidade, disponibilidade e integridade para suas aplicaçõe