6 research outputs found
Non-Uniform Replication
Replication is a key technique in the design of efficient and reliable distributed systems. As information grows, it becomes difficult or even impossible to store all information at every replica. A common approach to deal with this problem is to rely on partial replication, where each replica maintains only a part of the total system information. As a consequence, a remote replica might need to be contacted for computing the reply to some given query, which leads to high latency costs particularly in geo-replicated settings. In this work, we introduce the concept of non- uniform replication, where each replica stores only part of the information, but where all replicas store enough information to answer every query. We apply this concept to eventual consistency and conflict-free replicated data types. We show that this model can address useful problems and present two data types that solve such problems. Our evaluation shows that non-uniform replication is more efficient than traditional replication, using less storage space and network bandwidth
CRDTs in highly volatile environments
Publisher Copyright:
© 2022 The Author(s)The implementation of collaborative applications in highly volatile environments, such as the ones composed of mobile devices, requires low coordination mechanisms. The replication without coordination semantics of Conflict-Free Replicated Data Types (CRDTs) makes them a natural solution for these execution contexts. However, the current CRDT models require each replica to know all other replicas beforehand or to discover them on-the-fly. Such solutions are not compatible with the dynamic ingress and egress of nodes in volatile environments. To cope with this limitation, we propose the Publish/Subscribe Conflict-Free Replicated Data Type (PS-CRDT) model that combines CRDTs with the publish/subscribe interaction model, and, with that, enable the spatial and temporal decoupling of update propagation. We implemented PS-CRDTs in Thyme, a reactive storage system for mobile edge computing. Our experimental results show that PS-CRDTs require less communication than other CRDT-based solutions in volatile environments.publishersversionpublishe
Building fast and consistent (geo-)replicated systems : from principles to practice
Distributing data across replicas within a data center or across multiple data centers plays an important role in building Internet-scale services that provide a good user experience, namely low latency access and high throughput. This approach often compromises on strong consistency semantics, which helps maintain application-specific desired properties, namely, state convergence and invariant preservation. To relieve such inherent tension, in the past few years, many proposals have been designed to allow programmers to selectively weaken consistency levels of certain operations to avoid costly immediate coordination for concurrent user requests. However, these fail to provide principles to guide programmers to make a correct decision of assigning consistency levels to various operations so that good performance is extracted while the system behavior still complies with its specification.
The primary goal of this thesis work is to provide programmers with principles and tools for building fast and consistent (geo-) replicated systems by allowing programmers to think about various consistency levels in the same framework. The first step we took was to propose RedBlue consistency, which presents sufficient conditions that allow programmers to safely separate weakly consistent operations from strongly consistent ones in a coarse-grained manner. Second, to improve the practicality of RedBlue consistency, we built SIEVE - a tool that explores both Commutative Replicated Data Types and program analysis techniques to assign proper consistency levels to different operations and to maximize the weakly consistent operation space. Finally, we generalized the tradeoff between consistency and performance and proposed Partial Order-Restrictions consistency (or short, PoR consistency) - a generic consistency definition that captures various consistency levels in terms of visibility restrictions among pairs of operations and allows programmers to tune the restrictions to obtain a fine-grained control of their targeted consistency semantics.Daten auf mehrere Repliken in einem Datenzentrum oder über mehrere Datenzentren zu verteilen, nimmt einen hohen Stellenwert ein, um Internet-weite Services mit guter Nutzererfahrung, nsbesondere mit niedrigen Zugriffszeiten und hohem Datendurchsatz, zu implementieren. Diese Methode beeinträchtigt in der Regel die starke Konsitenzsemantik, die hilft gewünschte anwendungsspezifische Eigenschaften, die Zustandskonvergenz und Erhaltung von Invarianten, aufrechtzuerhalten. Um diesen Kompromiss zu mildern, wurde in den letzten Jahren mehrere Vorschläge entworfen, die es dem Programmierer ermöglichen für einzelne Operationen ein schwächeres Konsitenzlevel auszuwählen, um der aufwendigen Koordination paralleler Benutzeranfragen zu entgehen. Allerdings liefern diese Leitsätze für die Programmierer keine Lösungsansätze, wann welches Konsistenzlevel für eine Operation anzuwenden ist, so dass die höchstmögliche Leistung erreicht wird und gleichzeitig die Handlung des Systems die Spezifikation erfüllen.
Das Hauptziel dieser Doktorarbeit ist es Leitsätzen und Werkzeuge für Programmierer bereitzustellen, die die Entwicklung von leistungsstarken, konsistenten und (weltweit) replizierten Sytemen ermöglichen, in dem dem Programmierer mit Hilfe eines Frameworks gleichzeitig zwischen verschiedenen Konsistenzlevel wählen kann. Als ersten Schritt entwickelten wir RedBlue Konsistenz, welches die hinreichende Bedingungen erläutert, die es einem Programmierer erlauben zwischen schwacher Konsistenz und starker Konsistenz zu wählen. Um die Praktikabilität von RedBlue Konsistenz im zweiten Schritt weiter zu erhöhen, entwickelten wir SIEVE - ein Werkzeug, das sowohl kommutative, replizierte Datentypen und Programmanalyseverfahren verwendet, um den richtigen Konsistenzlevel zu verschiedenen Operationen zuzuordnen und dabei die schwach konsistenten Operationen zu maximieren. Abschliessend verallgemeinern wir den Kompromiss zwischen Konsistenz und Leistungsstärke und stellen die partiell, eingeschränkt geordnete Konsistenz vor (PoR Konsistenz) - eine generische Konsistenzdefinition, die verschiedene Konsistenz level, hinsichtlich der Einschränkung der Sichtbarkeit zwischen paaren von Operationen, umfasst und dem Programmierer erlaubt, die Einschränkungen zu justieren, um die gewünschte Konsistenzsemantik zu erzielen
Conflict-Free Replicated Data Types in Dynamic Environments
Over the years, mobile devices have become increasingly popular and gained improved
computation capabilities allowing them to perform more complex tasks such as
collaborative applications. Given the weak characteristic properties of mobile networks,
which represent highly dynamic environments where users may experience regular involuntary
disconnection periods, the big question arises of how to maintain data consistency.
This issue is most pronounced in collaborative environments where multiple users interact
with each other, sharing a replicated state that may diverge due to concurrency
conflicts and loss of updates.
To maintain consistency, one of today’s best solutions is Conflict-Free Replicated Data
Types (CRDTs), which ensure low latency values and automatic conflict resolution, guaranteeing
eventual consistency of the shared data. However, a limitation often found on
CRDTs and the systems that employ them is the need for the knowledge of the replicas
whom the state changes must be disseminated to. This constitutes a problem since it is
inconceivable to maintain said knowledge in an environment where clients may leave
and join at any given time and consequently get disconnected due to mobile network
communications unreliability.
In this thesis, we present the study and extension of the CRDT concept to dynamic
environments by introducing the developed P/S-CRDTs model, where CRDTs are coupled
with the publisher/subscriber interaction scheme and additional mechanisms to
ensure users are able to cooperate and maintain consistency whilst accounting for the
consequent volatile behaviors of mobile networks. The experimental results show that
in volatile scenarios of disconnection, mobile users in collaborative activity maintain
consistency among themselves and when compared to other available CRDT models, the
P/S-CRDTs model is able to decouple the required knowledge of whom the updates must
be disseminated to, while ensuring appropriate network traffic values
Verifying and Enforcing Application Constraints in Antidote SQL
Geo-replicated storage systems are currently a fundamental piece in the development of large-scale
applications where users are distributed across the world. To meet the high requirements regarding la-
tency and availability of these applications, these database systems are forced to use weak consistency
mechanisms. However, under these consistency models, there is no guarantee that the invariants are
preserved, which can jeopardise the correctness of applications. The most obvious alternative to solve
this problem would be to use strong consistency, but this would place a large burden on the system.
Since neither of these options was feasible, many systems have been developed to preserve the
invariants of the applications without sacrificing low latency and high availability. These systems,
based on the analysis of operations, make it possible to increase the guarantees of weak consistency
by introducing consistency at the level of operations that are potentially dangerous to the invariant.
Antidote SQL is a database system that, by combining strong with weak consistency mechanisms,
attempts to guarantee the preservation of invariants at the data level. In this way, and after defining
the concurrency semantics for the application, any operation can be performed without coordination
and without the risk of violating the invariant. However, this approach has some limitations, namely
the fact that it is not trivial for developers to define appropriate concurrency semantics.
In this document, we propose a methodology for the verification and validation of defined prop-
erties, such as invariants, for applications using Antidote SQL. The proposed methodology uses a
high-level programming language with automatic verification features called VeriFx and provides
guidelines for programmers who wish to implement and verify their own systems and specifications
using this tool.Os sistemas de armazenamento geo-replicados são atualmente uma peça fundamental no desenvolvi-
mento de aplicações de grande escala em que os utilizadores se encontram espalhados pelo mundo.
Com o objetivo de satisfazer os elevados requisitos em relação à latência e à disponibilidade destas
aplicações, estes sistemas de bases de dados vêem-se obrigados a recorrer a mecanismos de consistên-
cia fracos. No entanto, sob estes modelos de consistência não existe qualquer tipo de garantia de que os
invariantes são preservados, o que pode colocar em causa a correção das aplicações. A alternativa mais
óbvia para resolver este problema passaria por utilizar consistência forte, no entanto esta incutiria
uma grande sobrecarga no sistema.
Sendo que nenhuma destas opções é viável, muitos sistemas foram desenvolvidos no sentido de
preservar os invariantes das aplicações, sem contudo, abdicar de baixas latências e alta disponibilidade.
Estes sistemas, baseados na análise das operações, permitem aumentar as garantias de consistência
fraca com a introdução de consistência ao nível das operações potencialmente perigosas para o invari-
ante.
O Antidote SQL é um sistema de base de dados que através da combinação de mecanismos de
consistência fortes com mecanismos de consistência fracos tenta garantir a preservação dos invariantes
ao nível dos dados. Desta forma, e depois de definidas as semânticas de concorrência para a aplicação,
qualquer operação pode ser executada sem coordenação e sem perigo de quebra do invariante. No
entanto esta abordagem apresenta algumas limitações nomeadamente o facto de não ser trivial para
os programadores definirem as semânticas de concorrência adequadas.
Neste documento propomos uma metodologia para a verificação e validação de propriedades defi-
nidas, como os invariantes, para aplicações que usam o Antidote SQL. A metodologia proposta utiliza
uma linguagem de programação de alto nível com capacidade de verificação automática designada
por VeriFx, e fornece as diretrizes a seguir para que o programador consiga implementar e verificar
os seus próprios sistemas e especificações, utilizando a ferramenta