Optimizing on-demand resource deployment for peer-assisted content delivery (PhD thesis)

Increasingly, content delivery solutions leverage client resources in exchange for service in a peer-to-peer (P2P) fashion. Such peer-assisted service paradigms promise significant infrastructure cost reduction, but suffer from the unpredictability associated with client resources, which is often exhibited as an imbalance between the contribution and consumption of resources by clients. This imbalance hinders the ability to guarantee a minimum service fidelity of these services to the clients. In this thesis, we propose a novel architectural service model that enables the establishment of higher fidelity services through (1) coordinating the content delivery to optimally utilize the available resources, and (2) leasing the least additional cloud resources, available through special nodes (angels) that join the service on-demand, and only if needed, to complement the scarce resources available through clients. While the proposed service model can be deployed in many settings, this thesis focuses on peer-assisted content delivery applications, in which the scarce resource is typically the uplink capacity of clients. We target three applications that require the delivery of fresh as opposed to stale content. The first application is bulk-synchronous transfer, in which the goal of the system is to minimize the maximum distribution time -- the time it takes to deliver the content to all clients in a group. The second application is live streaming, in which the goal of the system is to maintain a given streaming quality. The third application is Tor, the anonymous onion routing network, in which the goal of the system is to boost performance (increase throughput and reduce latency) throughout the network, and especially for bandwidth-intensive applications. For each of the above applications, we develop mathematical models that optimally allocate the already available resources. They also optimally allocate additional on-demand resource to achieve a certain level of service. Our analytical models and efficient constructions depend on some simplifying, yet impractical, assumptions. Thus, inspired by our models and constructions, we develop practical techniques that we incorporate into prototypical peer-assisted angel-enabled cloud services. We evaluate those techniques through simulation and/or implementation. (Major Advisor: Azer Bestavros

Optimizing on-demand resource deployment for peer-assisted content delivery

Increasingly, content delivery solutions leverage client resources in exchange for services in a pee-to-peer (P2P) fashion. Such peer-assisted service paradigm promises significant infrastructure cost reduction, but suffers from the unpredictability associated with client resources, which is often exhibited as an imbalance between the contribution and consumption of resources by clients. This imbalance hinders the ability to guarantee a minimum service fidelity of these services to clients especially for real-time applications where content can not be cached. In this thesis, we propose a novel architectural service model that enables the establishment of higher fidelity services through (1) coordinating the content delivery to efficiently utilize the available resources, and (2) leasing the least additional cloud resources, available through special nodes (angels) that join the service on-demand, and only if needed, to complement the scarce resources available through clients. While the proposed service model can be deployed in many settings, this thesis focuses on peer-assisted content delivery applications, in which the scarce resource is typically the upstream capacity of clients. We target three applications that require the delivery of real-time as opposed to stale content. The first application is bulk-synchronous transfer, in which the goal of the system is to minimize the maximum distribution time - the time it takes to deliver the content to all clients in a group. The second application is live video streaming, in which the goal of the system is to maintain a given streaming quality. The third application is Tor, the anonymous onion routing network, in which the goal of the system is to boost performance (increase throughput and reduce latency) throughout the network, and especially for clients running bandwidth-intensive applications. For each of the above applications, we develop analytical models that efficiently allocate the already available resources. They also efficiently allocate additional on-demand resource to achieve a certain level of service. Our analytical models and efficient constructions depend on some simplifying, yet impractical, assumptions. Thus, inspired by our models and constructions, we develop practical techniques that we incorporate into prototypical peer-assisted angel-enabled cloud services. We evaluate these techniques through simulation and/or implementation

Peer-to-peer television for the IP multimedia subsystem

Peer-to-peer (P2P) video streaming has generated a significant amount of interest in both the research community and the industry, which find it a cost-effective solution to the user scalability problem. However, despite the success of Internet-based applications, the adoption has been limited for commercial services, such as Internet Protocol Television (IPTV). With the advent of the next-generation-networks (NGN) based on the IP Multimedia Subsystem (IMS), advocating for an open and inter-operable architecture, P2P emerges as a possible alternative in situations where the traditional mechanisms are not possible or economically feasible. This work proposes a P2P IPTV architecture for an IMS-based NGN, called P2PTV, which allows one or more service providers to use a common P2P infrastructure for streaming the TV channels to their subscribers. Instead of using servers, we rely on the uploading capabilities of the user equipments, like set-top boxes, located at the customers’ premise. We comply with the existing IMS and IPTV standards from the 3rd Generation Partnership Project (3GPP) and the Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN) bodies, where a centralized P2PTV application server (AS) manages the customer access to the service and the peer participation. Because watching TV is a complex and demanding user activity, we face two significant challenges. The first is to accommodate the mandatory IMS signaling, which reserves in the network the necessary QoS resources during every channel change, establishing a multimedia session between communicating peers. The second is represented by the streaming interruptions, or churn, when the uploading peer turns off or changes its current TV channel. To tackle these problems, we propose two enhancements. A fast signaling method, which uses inactive uploading sessions with reserved but unused QoS, to improve the tuning delay for new channel users. At every moment, the AS uses a feedback based algorithm to compute the number of necessary sessions that accommodates well the demand, while preventing the over-reservation of resources. We approach with special care mobility situations, where a proactive transfer of the multimedia session context using the IEEE 802.21 standard offers the best alternative to current methods. The second enhancement addresses the peer churn during channel changes. With every TV channel divided into a number of streams, we enable peers to download and upload streams different from their current channel, increasing the stability of their participation. Unlike similar work, we benefit from our estimation of the user demand and propose a decentralized method for a balanced assignment of peer bandwidth. We evaluate the performance of the P2PTV through modeling and large-scale computer simulations. A simpler experimental setting, with pure P2P streaming, indicates the improvements over the delay and peer churn. In more complex scenarios, especially with resource-poor peers having a limited upload capacity, we envision P2P as a complementary solution to traditional approaches like IP multicast. Reserving P2P for unpopular TV channels exploits the peer capacity and prevents the necessity of a large number of sparsely used multicast trees. Future work may refine the AS algorithms, address different experimental scenarios, and extend the lessons learned to non-IMS networks. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------La transmisión de vídeo con tecnologías peer-to-peer (P2P) ha generado un gran interés, tanto en la industria como en la comunidad científica, quienes han encontrado en dicha unión la solución para afrontar los problemas de escalabilidad de la transmisión de vídeo, reduciendo al mismo tiempo sus costes. A pesar del éxito de estos mecanismos en Internet, la transmisión de vídeo mediante técnicas P2P no se ha utilizado en servicios comerciales como puede ser el de televisión por IP (IPTV). Con la aparición de propuestas de redes de próxima generación basadas en el IP Multimedia Subsystem (IMS), que permite una arquitectura abierta e interoperable, los mecanismos basados en P2P emergen como posibles alternativas en situaciones donde los mecanismos tradicionales de transmisión de vídeo no se pueden desplegar o no son económicamente viables. Esta tesis propone una arquitectura de servicio de televisión peer-to-peer para una red de siguiente generación basada en IMS, que abreviaremos como P2PTV, que permite a uno o más proveedores de servicio utilizar una infraestructura P2P común para la transmisión de canales de TV a sus suscriptores. En vez de utilizar varios servidores, proponemos utilizar la capacidad de envío de los equipos de usuario, como los set-top boxes, localizados en el lado del cliente. En esta tesis extendemos los trabajos de estandarización sobre IMS IPTV de los organismos 3rd Generation Partnership Project (3GPP) y del Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN), donde un servidor de aplicación (AS) central de P2PTV administra el acceso de los clientes al servicio y permite compartir los recursos de los equipos. Debido a que el acceso a los canales de TV por parte de los usuarios es una actividad compleja, nos enfrentamos a dos retos importantes. El primero es administrar la señalización de IMS, con la cual se reservan los recursos de QoS necesarios durante cada cambio de canal, estableciendo una sesión multimedia entre los diferentes elementos de la comunicación. El segundo está representado por las interrupciones de la reproducción de video, causado por los equipos que sirven dicho vídeo cuando estos se desconectan del sistema o cuando cambian de canal. Para afrontar estos retos, proponemos dos mejoras al sistema. La primera mejora introduce el método de señalización rápida, en la cual se utilizan sesiones multimedia inactivas pero con recursos reservados para acelerar las conexiones entre usuarios. En cada momento, el AS utiliza la información extraída del algoritmo propuesto, que calcula el número de sesiones necesarias para administrar la demanda de conexiones, pero sin realizar una sobre-estimación, manteniendo bajo el uso de los recursos. Hemos abordado con especial cuidado la movilidad de los usuarios, donde se ha propuesto una transferencia de sesión pro-activa utilizando el estándar IEEE 802.21, el cual brinda una mejor alternativa que los métodos propuestos hasta la fecha. La segunda mejora se enfoca en las desconexiones de usuarios durante cambios de canal. Dividiendo los canales de TV en varios segmentos, permitimos a los equipos descargar o enviar diferentes partes de cualquier canal, aumentando la estabilidad de su participación. A diferencia de otros trabajos, nuestra propuesta se beneficia de la estimación de la demanda futura de los usuarios, proponiendo un método descentralizado para una asignación balanceada del ancho de banda de los equipos. Hemos evaluado el rendimiento del sistema P2PTV a través de modelado y de simulaciones de ordenador en sistemas IPTV de gran escala. Una configuración simple, con envío P2P puro, indica mejoras en el retardo y número de desconexiones de usuarios. En escenarios más complejos, especialmente con equipos con pocos recursos en la subida, sugerimos el uso de P2P como una solución complementaria a las soluciones tradicionales de multicast IP. Reservando el uso de P2P para los canales de TV poco populares, se permite explotar los recursos de los equipos y se previene la necesidad de un alto número de árboles multicast dispersos. Como trabajo futuro, se propone refinar los algoritmos del AS, abordar diferentes escenarios experimentales y también extender las lecciones aprendidas en esta tesis a otros sistemas no basados en IMS

Incentive Mechanisms in Peer-to-Peer Networks — A Systematic Literature Review

Centralized networks inevitably exhibit single points of failure that malicious actors regularly target. Decentralized networks are more resilient if numerous participants contribute to the network’s functionality. Most decentralized networks employ incentive mechanisms to coordinate the participation and cooperation of peers and thereby ensure the functionality and security of the network. This article systematically reviews incentive mechanisms for decentralized networks and networked systems by covering 165 prior literature reviews and 178 primary research papers published between 1993 and October 2022. Of the considered sources, we analyze 11 literature reviews and 105 primary research papers in detail by categorizing and comparing the distinctive properties of the presented incentive mechanisms. The reviewed incentive mechanisms establish fairness and reward participation and cooperative behavior. We review work that substitutes central authority through independent and subjective mechanisms run in isolation at each participating peer and work that applies multiparty computation. We use monetary, reputation, and service rewards as categories to differentiate the implementations and evaluate each incentive mechanism’s data management, attack resistance, and contribution model. Further, we highlight research gaps and deficiencies in reproducibility and comparability. Finally, we summarize our assessments and provide recommendations to apply incentive mechanisms to decentralized networks that share computational resources

Mathematical analysis of scheduling policies in peer-to-peer video streaming networks

Las redes de pares son comunidades virtuales autogestionadas, desarrolladas en la capa de aplicación sobre la infraestructura de Internet, donde los usuarios (denominados pares) comparten recursos (ancho de banda, memoria, procesamiento) para alcanzar un fin común. La distribución de video representa la aplicación más desafiante, dadas las limitaciones de ancho de banda. Existen básicamente tres servicios de video. El más simple es la descarga, donde un conjunto de servidores posee el contenido original, y los usuarios deben descargar completamente este contenido previo a su reproducción. Un segundo servicio se denomina video bajo demanda, donde los pares se unen a una red virtual siempre que inicien una solicitud de un contenido de video, e inician una descarga progresiva en línea. El último servicio es video en vivo, donde el contenido de video es generado, distribuido y visualizado simultáneamente. En esta tesis se estudian aspectos de diseño para la distribución de video en vivo y bajo demanda. Se presenta un análisis matemático de estabilidad y capacidad de arquitecturas de distribución bajo demanda híbridas, asistidas por pares. Los pares inician descargas concurrentes de múltiples contenidos, y se desconectan cuando lo desean. Se predice la evolución esperada del sistema asumiendo proceso Poisson de arribos y egresos exponenciales, mediante un modelo determinístico de fluidos. Un sub-modelo de descargas secuenciales (no simultáneas) es globalmente y estructuralmente estable, independientemente de los parámetros de la red. Mediante la Ley de Little se determina el tiempo medio de residencia de usuarios en un sistema bajo demanda secuencial estacionario. Se demuestra teóricamente que la filosofía híbrida de cooperación entre pares siempre desempeña mejor que la tecnología pura basada en cliente-servidor

An Emergent Architecture for Scaling Decentralized Communication Systems (DCS)

With recent technological advancements now accelerating the mobile and wireless Internet solution space, a ubiquitous computing Internet is well within the research and industrial community's design reach - a decentralized system design, which is not solely driven by static physical models and sound engineering principals, but more dynamically, perhaps sub-optimally at initial deployment and socially-influenced in its evolution. To complement today's Internet system, this thesis proposes a Decentralized Communication System (DCS) architecture with the following characteristics: flat physical topologies with numerous compute oriented and communication intensive nodes in the network with many of these nodes operating in multiple functional roles; self-organizing virtual structures formed through alternative mobility scenarios and capable of serving ad hoc networking formations; emergent operations and control with limited dependency on centralized control and management administration. Today, decentralized systems are not commercially scalable or viable for broad adoption in the same way we have to come to rely on the Internet or telephony systems. The premise in this thesis is that DCS can reach high levels of resilience, usefulness, scale that the industry has come to experience with traditional centralized systems by exploiting the following properties: (i.) network density and topological diversity; (ii.) self-organization and emergent attributes; (iii.) cooperative and dynamic infrastructure; and (iv.) node role diversity. This thesis delivers key contributions towards advancing the current state of the art in decentralized systems. First, we present the vision and a conceptual framework for DCS. Second, the thesis demonstrates that such a framework and concept architecture is feasible by prototyping a DCS platform that exhibits the above properties or minimally, demonstrates that these properties are feasible through prototyped network services. Third, this work expands on an alternative approach to network clustering using hierarchical virtual clusters (HVC) to facilitate self-organizing network structures. With increasing network complexity, decentralized systems can generally lead to unreliable and irregular service quality, especially given unpredictable node mobility and traffic dynamics. The HVC framework is an architectural strategy to address organizational disorder associated with traditional decentralized systems. The proposed HVC architecture along with the associated promotional methodology organizes distributed control and management services by leveraging alternative organizational models (e.g., peer-to-peer (P2P), centralized or tiered) in hierarchical and virtual fashion. Through simulation and analytical modeling, we demonstrate HVC efficiencies in DCS structural scalability and resilience by comparing static and dynamic HVC node configurations against traditional physical configurations based on P2P, centralized or tiered structures. Next, an emergent management architecture for DCS exploiting HVC for self-organization, introduces emergence as an operational approach to scaling DCS services for state management and policy control. In this thesis, emergence scales in hierarchical fashion using virtual clustering to create multiple tiers of local and global separation for aggregation, distribution and network control. Emergence is an architectural objective, which HVC introduces into the proposed self-management design for scaling and stability purposes. Since HVC expands the clustering model hierarchically and virtually, a clusterhead (CH) node, positioned as a proxy for a specific cluster or grouped DCS nodes, can also operate in a micro-capacity as a peer member of an organized cluster in a higher tier. As the HVC promotional process continues through the hierarchy, each tier of the hierarchy exhibits emergent behavior. With HVC as the self-organizing structural framework, a multi-tiered, emergent architecture enables the decentralized management strategy to improve scaling objectives that traditionally challenge decentralized systems. The HVC organizational concept and the emergence properties align with and the view of the human brain's neocortex layering structure of sensory storage, prediction and intelligence. It is the position in this thesis, that for DCS to scale and maintain broad stability, network control and management must strive towards an emergent or natural approach. While today's models for network control and management have proven to lack scalability and responsiveness based on pure centralized models, it is unlikely that singular organizational models can withstand the operational complexities associated with DCS. In this work, we integrate emergence and learning-based methods in a cooperative computing manner towards realizing DCS self-management. However, unlike many existing work in these areas which break down with increased network complexity and dynamics, the proposed HVC framework is utilized to offset these issues through effective separation, aggregation and asynchronous processing of both distributed state and policy. Using modeling techniques, we demonstrate that such architecture is feasible and can improve the operational robustness of DCS. The modeling emphasis focuses on demonstrating the operational advantages of an HVC-based organizational strategy for emergent management services (i.e., reachability, availability or performance). By integrating the two approaches, the DCS architecture forms a scalable system to address the challenges associated with traditional decentralized systems. The hypothesis is that the emergent management system architecture will improve the operational scaling properties of DCS-based applications and services. Additionally, we demonstrate structural flexibility of HVC as an underlying service infrastructure to build and deploy DCS applications and layered services. The modeling results demonstrate that an HVC-based emergent management and control system operationally outperforms traditional structural organizational models. In summary, this thesis brings together the above contributions towards delivering a scalable, decentralized system for Internet mobile computing and communications

Large-scale sensor-rich video management and delivery

Ph.DDOCTOR OF PHILOSOPH

Actas da 10ª Conferência sobre Redes de Computadores

Universidade do MinhoCCTCCentro AlgoritmiCisco SystemsIEEE Portugal Sectio