Incremental Consistency Guarantees for Replicated Objects

Programming with replicated objects is difficult. Developers must face the fundamental trade-off between consistency and performance head on, while struggling with the complexity of distributed storage stacks. We introduce Correctables, a novel abstraction that hides most of this complexity, allowing developers to focus on the task of balancing consistency and performance. To aid developers with this task, Correctables provide incremental consistency guarantees, which capture successive refinements on the result of an ongoing operation on a replicated object. In short, applications receive both a preliminary---fast, possibly inconsistent---result, as well as a final---consistent---result that arrives later. We show how to leverage incremental consistency guarantees by speculating on preliminary values, trading throughput and bandwidth for improved latency. We experiment with two popular storage systems (Cassandra and ZooKeeper) and three applications: a Twissandra-based microblogging service, an ad serving system, and a ticket selling system. Our evaluation on the Amazon EC2 platform with YCSB workloads A, B, and C shows that we can reduce the latency of strongly consistent operations by up to 40% (from 100ms to 60ms) at little cost (10% bandwidth increase, 6% throughput drop) in the ad system. Even if the preliminary result is frequently inconsistent (25% of accesses), incremental consistency incurs a bandwidth overhead of only 27%.Comment: 16 total pages, 12 figures. OSDI'16 (to appear

Implementation of a Distributed Computation Framework

This project aims at building a dynamic and scalable framework for Byzantine fault tolerant distributed computation. By organizing nodes of a distributed system in fault-tolerant clusters, we mask Byzantine failures of single nodes and provide an abstractions of reliable and highly available units at the level of these clusters

Scalable Byzantine Reliable Broadcast

Byzantine reliable broadcast is a powerful primitive that allows a set of processes to agree on a message from a designated sender, even if some processes (including the sender) are Byzantine. Existing broadcast protocols for this setting scale poorly, as they typically build on quorum systems with strong intersection guarantees, which results in linear per-process communication and computation complexity. We generalize the Byzantine reliable broadcast abstraction to the probabilistic setting, allowing each of its properties to be violated with a fixed, arbitrarily small probability. We leverage these relaxed guarantees in a protocol where we replace quorums with stochastic samples. Compared to quorums, samples are significantly smaller in size, leading to a more scalable design. We obtain the first Byzantine reliable broadcast protocol with logarithmic per-process communication and computation complexity. We conduct a complete and thorough analysis of our protocol, deriving bounds on the probability of each of its properties being compromised. During our analysis, we introduce a novel general technique that we call adversary decorators. Adversary decorators allow us to make claims about the optimal strategy of the Byzantine adversary without imposing any additional assumptions. We also introduce Threshold Contagion, a model of message propagation through a system with Byzantine processes. To the best of our knowledge, this is the first formal analysis of a probabilistic broadcast protocol in the Byzantine fault model. We show numerically that practically negligible failure probabilities can be achieved with realistic security parameters

Trade-offs in Replicated Systems

Replicated systems provide the foundation for most of today’s large-scale services. Engineering such replicated system is an onerous task. The first—and often foremost—step in this task is to establish an appropriate set of design goals, such as availability or performance, which should synthesize all the underlying system properties. Mixing design goals, however, is fraught with dangers, given that many properties are antagonistic and fundamental trade-offs exist among them. Navigating the harsh landscape of trade-offs is difficult because these formulations use different notations and system models, so it is hard to get an all-encompassing understanding of the state of the art in this area. In this paper, we address this difficulty by providing a systematic overview of the most relevant trade- offs involved in building replicated systems. Starting from the well-known FLP result, we follow a long line of research and investigate different trade-offs, assembling a coherent perspective of these results. Among others, we consider trade-offs which examine the complex interactions between properties such as consistency, availability, low latency, partition-tolerance, churn, scalability, and visibility latency

Long-amplicon MinION-based sequencing study in a salt-contaminated twelfth century granite-built chapel

The irregular damp dark staining on the stonework of a salt-contaminated twelfth century granite-built chapel is thought to be related to a non-homogeneous distribution of salts and microbial communities. To enhance understanding of the role of microorganisms in the presence of salt and damp stains, we determined the salt content and identified the microbial ecosystem in several paving slabs and inner wall slabs (untreated and previously bio-desalinated) and in the exterior surrounding soil. Soluble salt analysis and culture-dependent approaches combined with archaeal and bacterial 16S rRNA and fungal ITS fragment as well as with the functional genes nirK, dsr, and soxB long-amplicon MinION-based sequencing were performed. State-of-the-art technology was used for microbial identification, providing information about the microbial diversity and phylogenetic groups present and enabling us to gain some insight into the biological cycles occurring in the community key genes involved in the different geomicrobiological cycles. A well-defined relationship between microbial data and soluble salts was identified, suggesting that poorly soluble salts (CaSO4) could fill the pores in the stone and lead to condensation and dissolution of highly soluble salts (Ca(NO3)2 and Mg(NO3)2) in the thin layer of water formed on the stonework. By contrast, no direct relationship between the damp staining and the salt content or related microbiota was established. Further analysis regarding organic matter and recalcitrant elements in the stonework should be carried outOpen Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. P. Sanmartín acknowledges receipt of a Ramón y Cajal contract (RYC2020-029987-I) financed by the Spanish Ministry of Science and Innovation (MICINN). The authors acknowledge the projects APVV-19–0059 and VEGA 2/099/2021 which also financed this study. The authors acknowledge CONSORCIO DE LA CIUDAD DE SANTIAGO funding for this researchS

National identity predicts public health support during a global pandemic

Changing collective behaviour and supporting non-pharmaceutical interventions is an important component in mitigating virus transmission during a pandemic. In a large international collaboration (Study 1, N = 49,968 across 67 countries), we investigated self-reported factors associated with public health behaviours (e.g., spatial distancing and stricter hygiene) and endorsed public policy interventions (e.g., closing bars and restaurants) during the early stage of the COVID-19 pandemic (April-May 2020). Respondents who reported identifying more strongly with their nation consistently reported greater engagement in public health behaviours and support for public health policies. Results were similar for representative and non-representative national samples. Study 2 (N = 42 countries) conceptually replicated the central finding using aggregate indices of national identity (obtained using the World Values Survey) and a measure of actual behaviour change during the pandemic (obtained from Google mobility reports). Higher levels of national identification prior to the pandemic predicted lower mobility during the early stage of the pandemic (r = −0.40). We discuss the potential implications of links between national identity, leadership, and public health for managing COVID-19 and future pandemics.publishedVersio

Scaling Byzantine Fault Tolerance

Online services are becoming more and more ubiquitous and keep growing in scale. At the same time, they are required to be highly available, secure, energy-efficient, and to achieve high performance. To ensure these (and many other) properties, replication and distribution of these services becomes inevitable. Indeed, todayâs online services often involve thousands of processes running on different machines interconnected by a communication network. These processes may experience various kinds of failures, from simply crashing to being compromised by a malicious (Byzantine) adversary. The classic method for dealing with Byzantine faults is state machine replication (SMR). However, SMR fundamentally relies on a solution of the consensus problem, which often proves to be a scalability bottleneck. This dissertation addresses the scalability of Byzantine fault-tolerant systems. We argue that, for a certain class of applications, consensus either does not need to be solved at all, or only needs to be solved among a limited number of processes. By circumventing the consensus problem where solving it is not necessary, we improve the scalability of these applications. We start by focusing on the particular problem of distributed asset transfer, where digital assets are being transferred between user accountsâa problem underlying many cryptocurrency systems, most of which address it using Byzantine fault-tolerant SMR (and thus consensus). We show that consensus is not required for asset transfer by defining it as a sequential object type in the shared memory model and proving that it has consensus number 1 in Herlihyâs hierarchy. We further generalize the asset transfer object type, allowing an account to be shared by up to k owners. We prove that the consensus number of such an object type is k. We also discuss the asset transfer problem in the message passing model. We devise a consensusless asset transfer algorithm that relies on a secure broadcast primitive that, unlike consensus, has fully asynchronous deterministic implementations. Furthermore, since deterministic implementations of secure broadcast have limited scalability, we propose probabilistic secure broadcast, a variant of secure broadcast where some properties are allowed to be violated with a bounded probability. We design a highly scalable randomized algorithm that implements probabilistic secure broadcast with an arbitrarily low bound on the failure probability. Finally, we present Atum, a system for scalable group communication in a Byzantine environment that supports high churn. Atum achieves scalability by partitioning the system into groups of logarithmic size, only executing a consensus protocol inside each group