EGOIST: Overlay Routing Using Selfish Neighbor Selection

A foundational issue underlying many overlay network applications ranging from routing to P2P file sharing is that of connectivity management, i.e., folding new arrivals into an existing overlay, and re-wiring to cope with changing network conditions. Previous work has considered the problem from two perspectives: devising practical heuristics for specific applications designed to work well in real deployments, and providing abstractions for the underlying problem that are analytically tractable, especially via game-theoretic analysis. In this paper, we unify these two thrusts by using insights gleaned from novel, realistic theoretic models in the design of Egoist – a prototype overlay routing system that we implemented, deployed, and evaluated on PlanetLab. Using measurements on PlanetLab and trace-based simulations, we demonstrate that Egoist's neighbor selection primitives significantly outperform existing heuristics on a variety of performance metrics, including delay, available bandwidth, and node utilization. Moreover, we demonstrate that Egoist is competitive with an optimal, but unscalable full-mesh approach, remains highly effective under significant churn, is robust to cheating, and incurs minimal overhead. Finally, we discuss some of the potential benefits Egoist may offer to applications.National Science Foundation (CISE/CSR 0720604, ENG/EFRI 0735974, CISE/CNS 0524477, CNS/NeTS 0520166, CNS/ITR 0205294; CISE/EIA RI 0202067; CAREER 04446522); European Commission (RIDS-011923

Interaction of overlay networks: properties and control.

Jiang Wenjie.Thesis (M.Phil.)--Chinese University of Hong Kong, 2006.Includes bibliographical references (leaves 89-96).Abstracts in English and Chinese.Acknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Background --- p.1Chapter 1.2 --- Challenges --- p.2Chapter 1.3 --- Our Contribution --- p.4Chapter 1.4 --- Structure of the thesis --- p.5Chapter 2 --- Background Study --- p.7Chapter 2.1 --- An Introduction to Overlay Networks --- p.8Chapter 2.1.1 --- What is an Overlay Network? --- p.8Chapter 2.1.2 --- Benefits of Overlay Networks --- p.13Chapter 2.2 --- Taxonomy of Overlay Networks --- p.16Chapter 2.2.1 --- Routing Overlay Networks --- p.16Chapter 2.2.2 --- Content Delivery Networks (CDNs) --- p.25Chapter 2.2.3 --- Security Overlay Networks --- p.28Chapter 3 --- Mathematical Models for Overlay Routing --- p.32Chapter 3.1 --- Formulation of Routing in Overlay Networks --- p.32Chapter 3.2 --- Optimal Overlay Routing Policy --- p.34Chapter 3.3 --- Illustration of Overlay Routing Policy --- p.37Chapter 4 --- Overlay Routing Game --- p.40Chapter 4.1 --- Strategic Nash Routing Game --- p.40Chapter 4.2 --- Stable Property of Overlay Optimal Routing --- p.43Chapter 4.3 --- Routing Game in Other Forms --- p.44Chapter 5 --- Comparison of Routing Strategies: A Spectrum of Efficiency --- p.46Chapter 5.1 --- Global Optimal Routing --- p.47Chapter 5.2 --- Selfish User Routing --- p.49Chapter 5.3 --- Optimal Overlay Routing --- p.51Chapter 5.4 --- Performance Comparison --- p.54Chapter 6 --- Simulations on Routing Game --- p.56Chapter 6.1 --- Fluid Level Simulation --- p.56Chapter 6.2 --- Packet Level Simulation --- p.59Chapter 7 --- Understanding Various Issues & Implications of Overlay Interaction --- p.65Chapter 7.1 --- Sub-optimality of Nash Equilibrium --- p.66Chapter 7.2 --- Slow convergence to Nash equilibrium --- p.67Chapter 7.3 --- Fairness Paradox --- p.68Chapter 8 --- Overlay Pricing --- p.71Chapter 8.1 --- Pricing mechanism to improve end-to-end delay --- p.71Chapter 8.1.1 --- Fluid-level Simulation --- p.74Chapter 8.1.2 --- Packet-level Simulation --- p.77Chapter 8.2 --- Pricing mechanism to improve fairness --- p.77Chapter 9 --- Related Work --- p.83Chapter 10 --- Conclusion --- p.86Chapter 10.1 --- Summary of the Contribution --- p.86Chapter 10.2 --- Future Directions --- p.87Bibliography --- p.89Chapter A --- Proof of Existence of Nash Equilibrium --- p.9

A study of ISP pricing for networks with peer-to-peer users.

Wang, Qian.Thesis (M.Phil.)--Chinese University of Hong Kong, 2009.Includes bibliographical references (p. 71-74).Abstract also in Chinese.Abstract --- p.iAcknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 2 --- A Review of Pricing in Internet Industry --- p.5Chapter 2.1 --- Static Pricing --- p.6Chapter 2.1.1 --- Flat-rate Pricing --- p.6Chapter 2.1.2 --- Usage-based Pricing --- p.7Chapter 2.1.3 --- Paris Metro Pricing --- p.8Chapter 2.2 --- Dynamic Pricing --- p.9Chapter 2.2.1 --- Smart-market Pricing --- p.9Chapter 2.2.2 --- Responsive Pricing --- p.11Chapter 2.2.3 --- Edge Pricing --- p.12Chapter 2.3 --- Comparisons --- p.14Chapter 2.4 --- Concluding Remarks --- p.17Chapter 3 --- Uplink Pricing --- p.18Chapter 3.1 --- Introduction --- p.18Chapter 3.2 --- Model Description --- p.26Chapter 3.3 --- Uplink Pricing in a Competitive Market --- p.36Chapter 3.4 --- The Cooperative Strategy with Uplink Pricing --- p.40Chapter 3.4.1 --- The Cooperative Case --- p.41Chapter 3.4.2 --- The Threat Strategy --- p.45Chapter 3.5 --- Further Discussion --- p.47Chapter 3.5.1 --- Accounting Cost --- p.47Chapter 3.5.2 --- Peer-to-Peer Locality --- p.48Chapter 3.6 --- Related Works --- p.48Chapter 3.7 --- Concluding Remarks --- p.49Chapter 4 --- Viability of Paris Metro Pricing --- p.51Chapter 4.1 --- The Model --- p.52Chapter 4.2 --- Flat-rate Pricing versus Paris Metro Pricing --- p.54Chapter 4.2.1 --- One-channel Flat-rate Pricing --- p.55Chapter 4.2.2 --- Two-Channel Identical Pricing --- p.56Chapter 4.2.3 --- Flat-rate Pricing versus Two-Channel Iden-tical Pricing --- p.57Chapter 4.2.4 --- Flat-rate Pricing versus Paris Metro Pricing --- p.59Chapter 4.3 --- Case Studies --- p.60Chapter 4.4 --- Concluding Remarks --- p.62Chapter 5 --- Conclusion --- p.63A Equation Derivation --- p.65Chapter A. --- l Proof for Lemma 3.3.2 --- p.65Bibliography --- p.7

Design of Overlay Networks for Internet Multicast - Doctoral Dissertation, August 2002

Multicast is an efficient transmission scheme for supporting group communication in networks. Contrasted with unicast, where multiple point-to-point connections must be used to support communications among a group of users, multicast is more efficient because each data packet is replicated in the network – at the branching points leading to distinguished destinations, thus reducing the transmission load on the data sources and traffic load on the network links. To implement multicast, networks need to incorporate new routing and forwarding mechanisms in addition to the existing are not adequately supported in the current networks. The IP multicast are not adequately supported in the current networks. The IP multicast solution has serious scaling and deployment limitations, and cannot be easily extended to provide more enhanced data services. Furthermore, and perhaps most importantly, IP multicast has ignored the economic nature of the problem, lacking incentives for service providers to deploy the service in wide area networks. Overlay multicast holds promise for the realization of large scale Internet multicast services. An overlay network is a virtual topology constructed on top of the Internet infrastructure. The concept of overlay networks enables multicast to be deployed as a service network rather than a network primitive mechanism, allowing deployment over heterogeneous networks without the need of universal network support. This dissertation addresses the network design aspects of overlay networks to provide scalable multicast services in the Internet. The resources and the network cost in the context of overlay networks are different from that in conventional networks, presenting new challenges and new problems to solve. Our design goal are the maximization of network utility and improved service quality. As the overall network design problem is extremely complex, we divide the problem into three components: the efficient management of session traffic (multicast routing), the provisioning of overlay network resources (bandwidth dimensioning) and overlay topology optimization (service placement). The combined solution provides a comprehensive procedure for planning and managing an overlay multicast network. We also consider a complementary form of overlay multicast called application-level multicast (ALMI). ALMI allows end systems to directly create an overlay multicast session among themselves. This gives applications the flexibility to communicate without relying on service provides. The tradeoff is that users do not have direct control on the topology and data paths taken by the session flows and will typically get lower quality of service due to the best effort nature of the Internet environment. ALMI is therefore suitable for sessions of small size or sessions where all members are well connected to the network. Furthermore, the ALMI framework allows us to experiment with application specific components such as data reliability, in order to identify a useful set of communication semantic for enhanced data services

Systems-compatible Incentives

Originally, the Internet was a technological playground, a collaborative endeavor among researchers who shared the common goal of achieving communication. Self-interest used not to be a concern, but the motivations of the Internet's participants have broadened. Today, the Internet consists of millions of commercial entities and nearly 2 billion users, who often have conflicting goals. For example, while Facebook gives users the illusion of access control, users do not have the ability to control how the personal data they upload is shared or sold by Facebook. Even in BitTorrent, where all users seemingly have the same motivation of downloading a file as quickly as possible, users can subvert the protocol to download more quickly without giving their fair share. These examples demonstrate that protocols that are merely technologically proficient are not enough. Successful networked systems must account for potentially competing interests. In this dissertation, I demonstrate how to build systems that give users incentives to follow the systems' protocols. To achieve incentive-compatible systems, I apply mechanisms from game theory and auction theory to protocol design. This approach has been considered in prior literature, but unfortunately has resulted in few real, deployed systems with incentives to cooperate. I identify the primary challenge in applying mechanism design and game theory to large-scale systems: the goals and assumptions of economic mechanisms often do not match those of networked systems. For example, while auction theory may assume a centralized clearing house, there is no analog in a decentralized system seeking to avoid single points of failure or centralized policies. Similarly, game theory often assumes that each player is able to observe everyone else's actions, or at the very least know how many other players there are, but maintaining perfect system-wide information is impossible in most systems. In other words, not all incentive mechanisms are systems-compatible. The main contribution of this dissertation is the design, implementation, and evaluation of various systems-compatible incentive mechanisms and their application to a wide range of deployable systems. These systems include BitTorrent, which is used to distribute a large file to a large number of downloaders, PeerWise, which leverages user cooperation to achieve lower latencies in Internet routing, and Hoodnets, a new system I present that allows users to share their cellular data access to obtain greater bandwidth on their mobile devices. Each of these systems represents a different point in the design space of systems-compatible incentives. Taken together, along with their implementations and evaluations, these systems demonstrate that systems-compatibility is crucial in achieving practical incentives in real systems. I present design principles outlining how to achieve systems-compatible incentives, which may serve an even broader range of systems than considered herein. I conclude this dissertation with what I consider to be the most important open problems in aligning the competing interests of the Internet's participants

Provider and peer selection in the evolving internet ecosystem

The Internet consists of thousands of autonomous networks connected together to provide end-to-end reachability. Networks of different sizes, and with different functions and business objectives, interact and co-exist in the evolving "Internet Ecosystem". The Internet ecosystem is highly dynamic, experiencing growth (birth of new networks), rewiring (changes in the connectivity of existing networks), as well as deaths (of existing networks). The dynamics of the Internet ecosystem are determined both by external "environmental" factors (such as the state of the global economy or the popularity of new Internet applications) and the complex incentives and objectives of each network. These dynamics have major implications on how the future Internet will look like. How does the Internet evolve? What is the Internet heading towards, in terms of topological, performance, and economic organization? How do given optimization strategies affect the profitability of different networks? How do these strategies affect the Internet in terms of topology, economics, and performance? In this thesis, we take some steps towards answering the above questions using a combination of measurement and modeling approaches. We first study the evolution of the Autonomous System (AS) topology over the last decade. In particular, we classify ASes and inter-AS links according to their business function, and study separately their evolution over the last 10 years. Next, we focus on enterprise customers and content providers at the edge of the Internet, and propose algorithms for a stub network to choose its upstream providers to maximize its utility (either monetary cost, reliability or performance). Third, we develop a model for interdomain network formation, incorporating the effects of economics, geography, and the provider/peer selections strategies of different types of networks. We use this model to examine the "outcome" of these strategies, in terms of the topology, economics and performance of the resulting internetwork. We also investigate the effect of external factors, such as the nature of the interdomain traffic matrix, customer preferences in provider selection, and pricing/cost structures. Finally, we focus on a recent trend due to the increasing amount of traffic flowing from content providers (who generate content), to access providers (who serve end users). This has led to a tussle between content providers and access providers, who have threatened to prioritize certain types of traffic, or charge content providers directly -- strategies that are viewed as violations of "network neutrality". In our work, we evaluate various pricing and connection strategies that access providers can use to remain profitable without violating network neutrality.Ph.D.Committee Chair: Dovrolis, Constantine; Committee Member: Ammar, Mostafa; Committee Member: Feamster, Nick; Committee Member: Willinger, Walter; Committee Member: Zegura, Elle

Service management for multi-domain Active Networks

The Internet is an example of a multi-agent system. In our context, an agent is synonymous with network operators, Internet service providers (ISPs) and content providers. ISPs mutually interact for connectivity's sake, but the fact remains that two peering agents are inevitably self-interested. Egoistic behaviour manifests itself in two ways. Firstly, the ISPs are able to act in an environment where different ISPs would have different spheres of influence, in the sense that they will have control and management responsibilities over different parts of the environment. On the other hand, contention occurs when an ISP intends to sell resources to another, which gives rise to at least two of its customers sharing (hence contending for) a common transport medium. The multi-agent interaction was analysed by simulating a game theoretic approach and the alignment of dominant strategies adopted by agents with evolving traits were abstracted. In particular, the contention for network resources is arbitrated such that a self-policing environment may emerge from a congested bottleneck. Over the past 5 years, larger ISPs have simply peddled as fast as they could to meet the growing demand for bandwidth by throwing bandwidth at congestion problems. Today, the dire financial positions of Worldcom and Global Crossing illustrate, to a certain degree, the fallacies of over-provisioning network resources. The proposed framework in this thesis enables subscribers of an ISP to monitor and police each other's traffic in order to establish a well-behaved norm in utilising limited resources. This framework can be expanded to other inter-domain bottlenecks within the Internet. One of the main objectives of this thesis is also to investigate the impact on multi-domain service management in the future Internet, where active nodes could potentially be located amongst traditional passive routers. The advent of Active Networking technology necessitates node-level computational resource allocations, in addition to prevailing resource reservation approaches for communication bandwidth. Our motivation is to ensure that a service negotiation protocol takes account of these resources so that the response to a specific service deployment request from the end-user is consistent and predictable. To promote the acceleration of service deployment by means of Active Networking technology, a pricing model is also evaluated for computational resources (e.g., CPU time and memory). Previous work in these areas of research only concentrate on bandwidth (i.e., communication) - related resources. Our pricing approach takes account of both guaranteed and best-effort service by adapting the arbitrage theorem from financial theory. The central tenet for our approach is to synthesise insights from different disciplines to address problems in data networks. The greater parts of research experience have been obtained during direct and indirect participation in the 1ST-10561 project known as FAIN (Future Active IP Networks) and ACTS-AC338 project called MIAMI (Mobile Intelligent Agent for Managing the Information Infrastructure). The Inter-domain Manager (IDM) component was integrated as an integral part of the FAIN policy-based network management systems (PBNM). Its monitoring component (developed during the MIAMI project) learns about routing changes that occur within a domain so that the management system and the managed nodes have the same topological view of the network. This enabled our reservation mechanism to reserve resources along the existing route set up by whichever underlying routing protocol is in place