Achieving Reliability Through Distributed Data Flows and Recursive Delegation

Strong reliability properties, such as state machine replication or virtual synchrony, are hard to implement in a scalable manner. They are typically expressed in terms of global membership views. As we argue, global membership is non-scalable. We propose a way of modeling protocols that does not rely on global membership. Our approach is based on the concept of a distributed data flow, a set of messages distributed in space and time. We model protocols as networks of such flows, constructed through recursive delegation. The resulting system uses multiple small membership services instead of a single global one while still supporting stronger properties. Our work was inspired by the functional approach to modeling distributed systems pioneered by I/O automata. This paper focuses on the basic model. Internal details of our system architecture and a compiler that translates protocols from our data flow language to real executable code will be discussed elsewhere.This work was supported in part by grants from AFOSR, AFRL, NSF, and Intel Corporatio