On Verifying Causal Consistency

Causal consistency is one of the most adopted consistency criteria for distributed implementations of data structures. It ensures that operations are executed at all sites according to their causal precedence. We address the issue of verifying automatically whether the executions of an implementation of a data structure are causally consistent. We consider two problems: (1) checking whether one single execution is causally consistent, which is relevant for developing testing and bug finding algorithms, and (2) verifying whether all the executions of an implementation are causally consistent. We show that the first problem is NP-complete. This holds even for the read-write memory abstraction, which is a building block of many modern distributed systems. Indeed, such systems often store data in key-value stores, which are instances of the read-write memory abstraction. Moreover, we prove that, surprisingly, the second problem is undecidable, and again this holds even for the read-write memory abstraction. However, we show that for the read-write memory abstraction, these negative results can be circumvented if the implementations are data independent, i.e., their behaviors do not depend on the data values that are written or read at each moment, which is a realistic assumption.Comment: extended version of POPL 201

Proving sequential consistency by model checking

Sequential consistency is a multiprocessor memory model of both practical and theoretical importance. Unfortunately, the general problem of verifying that a finitestate protocol implements sequential consistency is undecidable, and in practice, validating that a real-world, finitestate protocol implements sequential consistency is very time-consuming and costly. In this work, we show that for memory protocols that occur in practice, a small amount of manual effort can reduce the problem of verifying sequential consistency into a verification task that can be discharged automatically via model checking. Furthermore, we present experimental results on a substantial, directorybased cache coherence protocol, which demonstrate the practicality of our approach.

Causal Consistency Verification in Restful Systems

Replicated systems cannot maintain both availability and (strong) consistency when exposed to network partitions. Strong consistency requires every read to return the last written value, which can lead clients to experience high latency or even timeout errors. Replicated applications usually rely on weak consistency, since clients can perform operations contacting a single replica, leading to decreased latency and increased availability. Causal consistency is a weak consistency model, however, it is the strongest one for highly available systems. Many applications are switching to this particular consistency model, since it ensures users never observe data items before they observe the ones that influenced their creation. Verifying if applications satisfy the consistency they claim to provide is no easy task. In this dissertation, we propose an algorithm to verify causal consistency in RESTful applications. Our approach adopts a black box testing, where multiple concurrent clients execute operations in a service and records the log of interactions. This log of interactions is then processed to verify if the results respect causal consistency. The key challenge is to infer causal dependencies among operations executed in different clients without adding any additional metadata to the data maintained by the service. When considering a particular operation, the algorithm builds a new dependency graph that considers one of the possible justifications the operation might have, but if this justification results in failure further ahead in the processing, it is necessary to build another graph considering another justification of that same operation. The algorithm relies on recursion in order to achieve this backtracking behaviour. If the algorithm is able to build a graph containing every operation present in the log, where the chosen justifications remain valid until the end of the processing, it outputs that the execution corresponding to that log satisfies causal consistency. The evaluation confirms that the algorithm is able to detect violations when feeding either small or large logs representing executions of RESTful applications that do not satisfy causal consistency.Os sistemas replicados não podem manter a disponibilidade e a consistência (forte) quando expostos a partições de rede. A consistência forte exige que cada leitura retorne o último valor escrito, o que pode levar os clientes a experienciar alta latência ou até mesmo erros de tempo limite. As aplicações replicados geralmente usam consistência fraca, pois os clientes podem realizar operações contactando uma única réplica, levando a latências baixas e maior disponibilidade. A consistência causal é um modelo de consistência fraco, mas é o mais forte para sistemas altamente disponíveis. Muitas aplicações usam este modelo, pois garante que os clientes nunca observem dados antes de observar os que influenciaram a sua criação. Verificar se as aplicações satisfazem a consistência que alegam fornecer não é fácil. Nesta dissertação, propomos um algoritmo para verificar a consistência causal em aplicações RESTful. A nossa abordagem adota um teste de caixa negra, onde vários clientes concurrentes executam operações num serviço, onde as interações são documentadas num ficheiro. Este ficheiro é processado para verificar se os resultados respeitam a consistência causal. O principal desafio é inferir as dependências causais entre as operações executadas em diferentes clientes sem adicionar metadados adicionais aos dados mantidos pelo serviço. Ao considerar uma determinada operação, o algoritmo constrói um novo grafo de dependências que considera uma das possíveis justificações que a operação possa ter, mas se esta justificação resultar em erro mais tarde no processamento, é necessário construir outro grafo considerando outra justificação dessa mesma operação. O algoritmo é recursivo de modo a alcançar esse comportamento de retrocesso. Se o algoritmo conseguir construir um grafo que contém todas as operações presentes no ficheiro, onde as justificações escolhidas permanecem válidas até o final do processamento, indica que a execução correspondente a este ficheiro satisfaz a consistência causal. Aavaliação confirma que o algoritmo é capaz de detectar violações ao fornecer ficheiros pequenos ou grandes representando execuções de aplicações RESTful que não satisfazem a consistência causal

Inductive diagrams for causal reasoning

The Lamport diagram is a pervasive and intuitive tool for informal reasoning about causality in a concurrent system. However, traditional axiomatic formalizations of Lamport diagrams can be painful to work with in a mechanized setting like Agda, whereas inductively-defined data would enjoy structural induction and automatic normalization. We propose an alternative, inductive formalization -- the causal separation diagram (CSD) -- that takes inspiration from string diagrams and concurrent separation logic. CSDs enjoy a graphical syntax similar to Lamport diagrams, and can be given compositional semantics in a variety of domains. We demonstrate the utility of CSDs by applying them to logical clocks -- widely-used mechanisms for reifying causal relationships as data -- yielding a generic proof of Lamport's clock condition that is parametric in a choice of clock. We instantiate this proof on Lamport's scalar clock, on Mattern's vector clock, and on the matrix clocks of Raynal et al. and of Wuu and Bernstein, yielding verified implementations of each. Our results and general framework are mechanized in the Agda proof assistant