Detouring and replication for fast and reliable internet-scale stream processing

iFlow is a replication-based system that can achieve both fast and reliable processing of high volume data streams on the Internet scale. iFlow uses a low degree of replication in conjunction with detouring techniques to overcome network congestion and outages. Computation over iFlow can be expressed as a graph of operators. To cope with varying system conditions these operators continually migrate in a manner that improves performance and availability at the same time. In this paper, we first provide an overview of our iFlow sys-tem. Next, we detail how our detouring technique works in the face of network failures to provide high availability for time critical applications. The paper also includes a de-scription of our implementation and preliminary evaluation results demonstrating that iFlow outperforms previous solu-tions with less overhead. Finally, the paper concludes with our plans for enhancing replication and detouring capabili-ties