Multicast in DKS(N, k, f) Overlay Networks

Recent developments in the area of peer-to-peer computing show that structured overlay networks implementing distributed hash tables scale well and can serve as infrastructures for Internet scale applications. We are developing a family of infrastructures, DKS(N; k; f), for the construction of peer-to-peer applications. An instance of DKS(N; k; f) is an overlay network that implements a distributed hash table and which has a number of desirable properties: low cost of communication, scalability, logarithmic lookup length, fault-tolerance and strong guarantees of locating any data item that was inserted in the system. In this paper, we show how multicast is achieved in DKS(N, k, f) overlay networks. The design presented here is attractive in three main respects. First, members of a multicast group self-organize in an instance of DKS(N, k, f) in a way that allows co-existence of groups of different sizes, degree of fault-tolerance, and maintenance cost, thereby, providing flexibility. Second, each member of a group can multicast, rather than having single source multicast. Third, within a group, dissemination of a multicast message is optimal under normal system operation in the sense that there are no redundant messages despite the presence of outdated routing information

Multiple-Tree Push-based Overlay Streaming

Multiple-Tree Overlay Streaming has attracted a great amount of attention from researchers in the past years. Multiple-tree streaming is a promising alternative to single-tree streaming in terms of node dynamics and load balancing, among others, which in turn addresses the perceived video quality by the streaming user on node dynamics or when heterogeneous nodes join the network. This article presents a comprehensive survey of the different aproaches and techniques used in this research area. In this paper we identify node-disjointness as the property most approaches aim to achieve. We also present an alternative technique which does not try to achieve this but does local optimizations aiming global optimizations. Thus, we identify this property as not being absolute necessary for creating robust and heterogeneous multi-tree overlays. We identify two main design goals: robustness and support for heterogeneity, and classify existing approaches into these categories as their main focus

An Evaluation of Opportunistic Native Multicast

Hybrid multicast opportunistically combines Application Layer Multicast and native multicast protocols. This paper presents an evaluation of Opportunistic Native Multicast which uses the AMT multicast tunnelling protocol. We describe our opportunistic multicasting approach that tries to solve the islands phenomenon by building unicast tunnels to connect these islands while attempting to utilise the native multicast capability of the islands. It is expected that this hybrid approach will improve both efficiency and availability of multicast. We compare our approach to the ALM protocol SCRIBE. The comparison here was done using five metrics: Stress, Stretch, intra-island traffic, interisland traffic and Delivery rate. In all of them, our proposed model has shown improved results over ALM. Moreover, we investigated what effect the number of islands that the receivers are distributed into, has on performance

Discovering Network Neighborhoods Using Peer-to-Peer Lookups

In many distributed applications, end hosts need to know the network locations of other nearby participating hosts in order to enhance overall performance. Potential applications that can benefit from the location information include automatic selection of nearby Web servers, proximity routing in a peer-to-peer system, and loss recovery in reliable multicasting. We focus in this paper on the network neighborhood discovery problem in large-scale distributed systems. In these systems, the number of participating nodes can be very large, and the membership can dynamically change. Our goal is for each node to discover other "nearby" participating nodes in a completely decentralized manner, where each node probes only a small subset of other nodes in the system. This approach will lead to improved overall performance by matching client requests for services with participants in the peer-to-peer service system that are, on average, nearby in the network sense. Recent works in distributed peer-to-peer systems, such as Chord, CAN, Tapestry and Pastry, provide efficient distributed lookup structures. In this paper, we investigate a rendezvous-based scheme for a node to discover other nearby participating nodes using a peer-to-peer lookup system such as Chord. Given a key, the Chord protocol maps the key onto a node. Our idea for network neighborhood discovery is for each host to compute a key that characterizes its network location on the Internet. We call such a key the location key, and the nodes that these location keys are mapped to the Rendezvous Points. To lookup other nearby participating nodes, a node seeking some service queries its corresponding rendezvous point using its location key. We focus on the issue of how to generate the location key in a distributed fashion such that nodes that are close to each other in the actual network will have similar location key values, and therefore be mapped to nearby locations on the Chord ring. In this paper, we examine the performance tradeoffs of such a rendezvous scheme using the Global Network Positioning (GNP) approach to generate the location keys. In GNP, each node measures its network distances to a few landmark nodes to derive its coordinates in a D-dimensional geometric space. We generate a host's Chord location key from its 1-dimensional GNP coordinate, and use coordinates from a higher dimensional space to refine the searching process for the closest node. We evaluate our scheme in the context of the nearest neighbor discovery problem. Using data from the Active Measurement Project of the National Laboratory for Applied Network Research (NLANR), we compare its performance with a random mapping scheme, where location keys are randomly generated. Using our coordinate-based rendezvous scheme, 66% of the nodes found their actual closest network neighbor by pinging only a small number of nodes.Singapore-MIT Alliance (SMA

Design and Evaluation of Distributed Algorithms for Placement of Network Services

Network services play an important role in the Internet today. They serve as data caches for websites, servers for multiplayer games and relay nodes for Voice over IP: VoIP) conversations. While much research has focused on the design of such services, little attention has been focused on their actual placement. This placement can impact the quality of the service, especially if low latency is a requirement. These services can be located on nodes in the network itself, making these nodes supernodes. Typically supernodes are selected in either a proprietary or ad hoc fashion, where a study of this placement is either unavailable or unnecessary. Previous research dealt with the only pieces of the problem, such as finding the location of caches for a static topology, or selecting better routes for relays in VoIP. However, a comprehensive solution is needed for dynamic applications such as multiplayer games or P2P VoIP services. These applications adapt quickly and need solutions based on the immediate demands of the network. In this thesis we develop distributed algorithms to assign nodes the role of a supernode. This research first builds off of prior work by modifying an existing assignment algorithm and implementing it in a distributed system called Supernode Placement in Overlay Topologies: SPOT). New algorithms are developed to assign nodes the supernode role. These algorithms are then evaluated in SPOT to demonstrate improved SN assignment and scalability. Through a series of simulation, emulation, and experimentation insight is gained into the critical issues associated with allocating resources to perform the role of supernodes. Our contributions include distributed algorithms to assign nodes as supernodes, an open source fully functional distributed supernode allocation system, an evaluation of the system in diverse networking environments, and a simulator called SPOTsim which demonstrates the scalability of the system to thousands of nodes. An example of an application deploying such a system is also presented along with the empirical results

Total order broadcast for fault tolerant exascale systems

In the process of designing a new fault tolerant run-time for future exascale systems, we discovered that a total order broadcast would be necessary. That is, nodes of a supercomputer should be able to broadcast messages to other nodes even in the face of failures. All messages should be seen in the same order at all nodes. While this is a well studied problem in distributed systems, few researchers have looked at how to perform total order broadcasts at large scales for data availability. Our experience implementing a published total order broadcast algorithm showed poor scalability at tens of nodes. In this paper we present a novel algorithm for total order broadcast which scales logarithmically in the number of processes and is not delayed by most process failures. While we are motivated by the needs of our run-time we believe this primitive is of general applicability. Total order broadcasts are used often in datacenter environments and as HPC developers begins to address fault tolerance at the application level we believe they will need similar primitives