Designing a Planetary-Scale IMAP Service with Conflict-free Replicated Data Types

Modern geo-replicated software serving millions of users across the globe faces the consequences of the CAP dilemma, i.e., the inevitable conflicts that arise when multiple nodes accept writes on shared state. The underlying problem is commonly known as fault-tolerant multi-leader replica- tion; actively researched in the distributed systems and database communities. As a more recent theoretical framework, Conflict-free Replicated Data Types (CRDTs) propose a solution to this problem by offering a set of always converging primitives. However, modeling non-trivial system state with CRDT primitives is a challenging and error-prone task. In this work, we propose a solution for a geo-replicated online service with fault-tolerant multi-leader replication based on CRDTs. We chose IMAP as use case due to its prevalence and simplicity. Therefore, we modeled an IMAP-CRDT and verified its correctness with the interactive theorem prover Isabelle/HOL. In order to bridge the gap between theory and practice, we implemented an open-source proto- type pluto and an IMAP benchmark for write-intensive workloads. We evaluated our prototype against the standard IMAP server Dovecot on a multi-continent public cloud. The results ex- pose the limitations of Dovecot with respect to response time performance and replication lag. Our prototype was able to leverage its conceptual advantages and outperformed Dovecot. We find that our approach is promising when facing the multitude of potential concurrency bugs in development of systems at planetary scale

Equivalence classes for named function networking

Named Function Networking (NFN) is a generalization of Content-Centric Networking (CCN) and Named Data Networking (NDN). Beyond mere content retrieval, NFN enables to ask for results of computations. Names are not just content identifiers but λ-expressions that allow an arbitrary composition of function calls and data accesses. λ-expressions are pure and deterministic. In other words, they do not have side effects and they always yield the same result. Both properties together are known to as referential transparency. Referentially transparent functions can be evaluated individually no matter where and in what order, e.g. geographically distributed and concurrently. This simplifies the distribution of computations in a network, an attractive feature in times of rising needs for edge computing. However, NFN is affected by a lacking awareness for referentially opaque expressions that are characterized by having changing results or side effects, i.e. expressions that depend on outer conditions or modify outer states. The fundamental motivation of this thesis is to retrofit NFN with a clearer notion of referentially opaque expressions. They are indispensable not only to many common use cases such as e-mail and database applications, but also to network technologies such as software defined networking. We observed that many protocol decisions are based on expression matching, i.e. the search for equivalent expressions. Driven by this observation, this thesis explores possibilities to adapt the determination of equivalences in dependence of crucial expression properties such as their ability for aggregation, concurrent evaluation or permanently cacheable results. This exploration results in a comprehensive set of equivalence classes that is used for explicit attribution of expressions, leading to a system that is aware of the true nature of handled expressions. Moreover, we deliver a solution to support referentially opaque expressions and mutable states in an architecture that bases upon uniquely named and immutable data packets. Altogether, the findings condense to an extended execution model. It summarizes how the attribution of expressions with equivalence classes influences specific protocol decisions in order to support referentially transparent as well as referentially opaque expressions. We believe that our approach captivates due to its generality and extensibility. Equivalence classes depend upon universal properties. Therefore, our approach is not bound to a specific elaboration like NFN. We evaluate the applicability of our approach in a few application scenarios. Overall, the proposed solutions and concepts are an important contribution towards name-based distributed computations in information-centric networks