Vitis: A Gossip-based Hybrid Overlay for Internet-scale Publish/Subscribe

Peer-to-peer overlay networks are attractive solutions for building Internet-scale publish/subscribe systems. However, scalability comes with a cost: a message published on a certain topic often needs to traverse a large number of uninterested (unsubscribed) nodes before reaching all its subscribers. This might sharply increase resource consumption for such relay nodes (in terms of bandwidth transmission cost, CPU, etc) and could ultimately lead to rapid deterioration of the system’s performance once the relay nodes start dropping the messages or choose to permanently abandon the system. In this paper, we introduce Vitis, a gossip-based publish/subscribe system that significantly decreases the number of relay messages, and scales to an unbounded number of nodes and topics. This is achieved by the novel approach of enabling rendezvous routing on unstructured overlays. We construct a hybrid system by injecting structure into an otherwise unstructured network. The resulting structure resembles a navigable small-world network, which spans along clusters of nodes that have similar subscriptions. The properties of such an overlay make it an ideal platform for efficient data dissemination in large-scale systems. We perform extensive simulations and evaluate Vitis by comparing its performance against two base-line publish/subscribe systems: one that is oblivious to node subscriptions, and another that exploits the subscription similarities. Our measurements show that Vitis significantly outperforms the base-line solutions on various subscription and churn scenarios, from both synthetic models and real-world traces

A Hybrid Multicast-Unicast Infrastructure for Efficient Publish-Subscribe in Enterprise Networks

One of the main challenges in building a large scale publish-subscribe infrastructure in an enterprise network, is to provide the subscribers with the required information, while minimizing the consumed host and network resources. Typically, previous approaches utilize either IP multicast or point-to-point unicast for efficient dissemination of the information. In this work, we propose a novel hybrid framework, which is a combination of both multicast and unicast data dissemination. Our hybrid framework allows us to take the advantages of both multicast and unicast, while avoiding their drawbacks. We investigate several algorithms for computing the best mapping of publishers' transmissions into multicast and unicast transport. Using extensive simulations, we show that our hybrid framework reduces consumed host and network resources, outperforming traditional solutions. To insure the subscribers interests closely resemble those of real-world settings, our simulations are based on stock market data and on recorded IBM WebShpere subscriptions

The Raincore API for clusters of networking elements

Clustering technology offers a way to increase overall reliability and performance of Internet information flow by strengthening one link in the chain without adding others. We have implemented this technology in a distributed computing architecture for network elements. The architecture, called Raincore, originated in the Reliable Array of Independent Nodes, or RAIN, research collaboration between the California Institute of Technology and the US National Aeronautics and Space Agency's Jet Propulsion Laboratory. The RAIN project focused on developing high-performance, fault-tolerant, portable clustering technology for spaceborne computing . The technology that emerged from this project became the basis for a spinoff company, Rainfinity, which has the exclusive intellectual property rights to the RAIN technology. The authors describe the Raincore conceptual architecture and distributed services, which are designed to make it easy for developers to port their applications to run on top of a cluster of networking elements. We include two applications: a Web server prototype that was part of the original RAIN research project and a commercial firewall cluster product from Rainfinity

A Framework for Realistic and Systematic Multicast Performance Evaluation

Previous multicast research often makes commonly accepted but unverifed assumptions on network topologies and group member distribution in simulation studies. In this paper, we propose a framework to systematically evaluate multicast performance for different protocols. We identify a series of metrics, and carry out extensive simulation studies on these metrics with different topological models and group member distributions for three case studies. Our simulation results indicate that realistic topology and group membership models are crucial to accurate multicast performance evaluation. These results can provide guidance for multicast researchers to perform realistic simulations, and facilitate the design and development of multicast protocols

Policy-Based Dynamic Proxy Framework: An Application Level Infrastructure For Active Service Creation And Contents Delivery

This thesis focuses on the dynamic proxy framework named the Chek Proxy Framework (CPF). The main objectives are to scale the existing Internet architecture by conserving the backbone bandwidth, reducing server loads, and improving the overall networking system performance, particularly the client receiving rate. These were achieved by deploying application-level proxy services within the network, to accelerate and customise the delivery of contents. The CPF is based on the 3-tier distributed computing architecture with the client and server residing at the ends of the respective networks. A dynamically appointed middle-tier system, the Dynamic Application Proxy Server (DAPS) is created ondemand and resides at the client-side network based on the designed clustering policy. The uniqueness of CPF lies on the use of voluntary client machines, instead of static and dedicated machines to host DAPS services created at runtime

Understanding a large-scale IPTV network via system logs

Recently, there has been a global trend among the telecommunication industry on the rapid deployment of IPTV (Internet Protocol Television) infrastructure and services. While the industry rushes into the IPTV era, the comprehensive understanding of the status and dynamics of IPTV network lags behind. Filling this gap requires in-depth analysis of large amounts of measurement data across the IPTV network. One type of the data of particular interest is device or system log, which has not been systematically studied before. In this dissertation, we will explore the possibility of utilizing system logs to serve a wide range of IPTV network management purposes including health monitoring, troubleshooting and performance evaluation, etc. In particular, we develop a tool to convert raw router syslogs to meaningful network events. In addition, by analyzing set-top box (STB) logs, we propose a series of models to capture both channel popularity and dynamics, and users' activity on the IPTV network.Ph.D.Committee Chair: Jun Xu; Committee Member: Jia Wang; Committee Member: Mostafa H. Ammar; Committee Member: Nick Feamster; Committee Member: Xiaoli M

CLON: overlay networks and gossip protocols for cloud environments

Although epidemic or gossip-based multicast is a robust and scalable approach to reliable data dissemination, its inherent redundancy results in high resource consumption on both links and nodes. This problem is aggravated in settings that have costlier or resource constrained links as happens in Cloud Computing infrastructures composed by several interconnected data centers across the globe. The goal of this work is therefore to improve the efficiency of gossip-based reliable multicast by reducing the load imposed on those constrained links. In detail, the proposed clon protocol combines an overlay that gives preference to local links and a dissemination strategy that takes into account locality. Extensive experimental evaluation using a very large number of simulated nodes shows that this results in a reduction of traffic in constrained links by an order of magnitude, while at the same time preserving the resilience properties that make gossip-based protocols so attractive.HP Labs Innovation Research Award, project DC2MS (IRA/CW118736