Design of Scalable On-Demand Video Streaming Systems Leveraging Video Viewing Patterns

The explosive growth in on-demand access of video across all forms of delivery (Internet, traditional cable, IPTV, wireless) has renewed the interest in scalable delivery methods. Approaches using Content Delivery Networks (CDNs), Peer-to-Peer (P2P) approaches, and their combinations have been proposed as viable options to ease the load on servers and network links. However, there has been little focus on how to take advantage of user viewing patterns to understand their impact on existing mechanisms and to design new solutions that improve the streaming service quality. In this dissertation, we leverage on the observation that users watch only a small portion of videos to understand the limits of existing designs and to optimize two scalable approaches -- the content placement and P2P Video-on-Demand (VoD) streaming. Then, we present our novel scalable system called Joint-Family which enables adaptive bitrate streaming (ABR) in P2P VoD, supporting user viewing patterns. We first provide evidence of such user viewing behavior from data collected from a nationally deployed VoD service. In contrast to using a simplistic popularity-based placement and traditionally proposed caching strategies (such as CDNs), we use a Mixed Integer Programming formulation to model the placement problem and employ an innovative approach that scales well. We have performed detailed simulations using actual traces of user viewing sessions (including stream control operations such as pause, fast-forward, and rewind). Our results show that the use of segment-based placement strategy yields substantial savings in both disk storage requirements at origin servers/VHOs as well as network bandwidth use. For example, compared to a simple caching scheme using full videos, our MIP-based placement using segments can achieve up to 71% reduction in peak link bandwidth usage. Secondly, we note that the policies adopted in existing P2P VoD systems have not taken user viewing behavior -- that users abandon videos -- into account. We show that abandonment can result in increased interruptions and wasted resources. As a result, we reconsider the set of policies to use in the presence of abandonment. Our goal is to balance the conflicting needs of delivering videos without interruptions while minimizing wastage. We find that an Earliest-First chunk selection policy in conjunction with the Earliest-Deadline peer selection policy allows us to achieve high download rates. We take advantage of abandonment by converting peers to "partial seeds"; this increases capacity. We minimize wastage by using a playback lookahead window. We use analysis and simulation experiments using real-world traces to show the effectiveness of our approach. Finally, we propose Joint-Family, a protocol that combines P2P and adaptive bitrate (ABR) streaming for VoD. While P2P for VoD and ABR have been proposed previously, they have not been studied together because they attempt to tackle problems with seemingly orthogonal goals. We motivate our approach through analysis that overcomes a misconception resulting from prior analytical work, and show that the popularity of a P2P swarm and seed staying time has a significant bearing on the achievable per-receiver download rate. Specifically, our analysis shows that popularity affects swarm efficiency when seeds stay "long enough". We also show that ABR in a P2P setting helps viewers achieve higher playback rates and/or fewer interruptions. We develop the Joint-Family protocol based on the observations from our analysis. Peers in Joint-Family simultaneously participate in multiple swarms to exchange chunks of different bitrates. We adopt chunk, bitrate, and peer selection policies that minimize occurrence of interruptions while delivering high quality video and improving the efficiency of the system. Using traces from a large-scale commercial VoD service, we compare Joint-Family with existing approaches for P2P VoD and show that viewers in Joint-Family enjoy higher playback rates with minimal interruption, irrespective of video popularity

Modeling and analysis of P2P streaming.

Zhou, Yipeng.Thesis (M.Phil.)--Chinese University of Hong Kong, 2008.Includes bibliographical references (leaves 64-66).Abstracts in English and Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Background --- p.1Chapter 1.2 --- Contribution --- p.2Chapter 1.3 --- Organization --- p.4Chapter 2 --- Related Work --- p.5Chapter 2.1 --- Work of Streaming --- p.5Chapter 2.2 --- Work of P2P VoD --- p.6Chapter 3 --- Basic Model of Synchronized Case --- p.8Chapter 4 --- Model of Chunk Selection Strategies --- p.13Chapter 4.1 --- Chunk Selection Strategies --- p.13Chapter 4.1.1 --- Greedy Strategy --- p.14Chapter 4.1.2 --- Rarest First Strategy --- p.15Chapter 4.1.3 --- "Buffer Size, Peer Population and Continuity" --- p.16Chapter 4.1.4 --- Mixed Strategy --- p.17Chapter 4.2 --- Some Conclusion and Extension --- p.19Chapter 4.3 --- Metrics --- p.20Chapter 4.3.1 --- Continuity --- p.20Chapter 4.3.2 --- Start-up Latency --- p.20Chapter 5 --- Experiment and Application --- p.22Chapter 5.1 --- Numerical Examples and Analysis --- p.22Chapter 5.2 --- Sensitivity study --- p.30Chapter 5.2.1 --- Discrete Model with Factor --- p.30Chapter 5.2.2 --- Validate Discrete Model with Factor --- p.31Chapter 5.2.3 --- Server Use Pull Strategy --- p.31Chapter 5.2.4 --- Vary Subset Size Touched by Server --- p.32Chapter 5.3 --- Application to Real-world Protocols --- p.32Chapter 6 --- Model of Unsynchronized Case --- p.34Chapter 6.1 --- The model for unsynchronized playback --- p.34Chapter 6.1.1 --- Overlap maximization problem --- p.37Chapter 6.1.2 --- Properties of the synchronized cluster --- p.38Chapter 6.2 --- Analysis of playback continuity --- p.40Chapter 6.2.1 --- Peers with different buffer sizes --- p.41Chapter 6.2.2 --- Analysis of two clusters with a lag --- p.44Chapter 7 --- Performance Evaluation of Unsynchronized System --- p.48Chapter 7.1 --- Performance Evaluation --- p.48Chapter 8 --- conclusion --- p.54Chapter 8.1 --- Conclusion --- p.54Chapter A --- Equation Derivation --- p.56Bibliography --- p.6

A credit-based approach to scalable video transmission over a peer-to-peer social network

PhDThe objective of the research work presented in this thesis is to study scalable video transmission over peer-to-peer networks. In particular, we analyse how a credit-based approach and exploitation of social networking features can play a significant role in the design of such systems. Peer-to-peer systems are nowadays a valid alternative to the traditional client-server architecture for the distribution of multimedia content, as they transfer the workload from the service provider to the final user, with a subsequent reduction of management costs for the former. On the other hand, scalable video coding helps in dealing with network heterogeneity, since the content can be tailored to the characteristics or resources of the peers. First of all, we present a study that evaluates subjective video quality perceived by the final user under different transmission scenarios. We also propose a video chunk selection algorithm that maximises received video quality under different network conditions. Furthermore, challenges in building reliable peer-to-peer systems for multimedia streaming include optimisation of resource allocation and design mechanisms based on rewards and punishments that provide incentives for users to share their own resources. Our solution relies on a credit-based architecture, where peers do not interact with users that have proven to be malicious in the past. Finally, if peers are allowed to build a social network of trusted users, they can share the local information they have about the network and have a more complete understanding of the type of users they are interacting with. Therefore, in addition to a local credit, a social credit or social reputation is introduced. This thesis concludes with an overview of future developments of this research work

Telecommunications Networks

This book guides readers through the basics of rapidly emerging networks to more advanced concepts and future expectations of Telecommunications Networks. It identifies and examines the most pressing research issues in Telecommunications and it contains chapters written by leading researchers, academics and industry professionals. Telecommunications Networks - Current Status and Future Trends covers surveys of recent publications that investigate key areas of interest such as: IMS, eTOM, 3G/4G, optimization problems, modeling, simulation, quality of service, etc. This book, that is suitable for both PhD and master students, is organized into six sections: New Generation Networks, Quality of Services, Sensor Networks, Telecommunications, Traffic Engineering and Routing

MULTI-TASK NEURAL NETWORK MODEL FOR CUSTOMER SERVICE EXPERIENCE PREDICTION ANALYSIS BASED IN QOS AND QOE DATA, OVER TELECOMMUNICATIONS INTERNET AND TV SERVICES

Telecommunication companies monitor the infrastructure and equipment to guarantee the quality delivery of their services to the customer. Customer experience is of utmost importance in the telecommunication area due to the fierce market competition. There are two main reasons for a client to experience bad quality of service: disruption events and service degradation. Disruptive events map to situations where the client loses complete access to the service, whereas service degradation limits the client’s service and/or can origin multiple failures during the service use. In disruption situations, the client will immediately call and make a complaint. In degradation the client may not complain due to many different reasons, but mainly because the client does not use the service or does not want to go through the process of calling the operator to place a complaint. This dissertation presents a solution that can identify customers with degradation in their services so that the company can proactively call the signaled customers and sched- ule an intervention to correct the problems. Currently is suspected that the intersection between the services is higher than is perceived, so it is the thesis objective to explore and verify services correlation. To achieve the goal we start by characterizing the problem, then explore the real data consumption patterns, and analyze the intersection in experi- ence problems. To cope with TV and Internet services complains target simultaneously the thesis leverages a model with a multi-task architecture whose goal is to learn the implicit information available in the services intersection. Multi-task model solves the limitations existent in current solutions and adds the possibility to have multiple targets that share information between them. Then, the model develops the capacity to learn situations where the symptom and the resolution are in different services. Through the multi-task model capacity to explore the implicit information contained in the services intersection the results are improved 25% against the benchmark models which do not support multi-task. Given the performance increase, the company project strategy was impacted and new uses cases considered.As empresas de telecomunicações monitorizam a infraestrutura e os equipamentos para conseguirem garantir qualidade na entrega dos serviços aos clientes. Num mercado feroz como o de telecomunicações, investir na experiência de utilizador é muito importante. As duas razões principais para explicar má experiencia de serviço são: eventos disruptivos e degradação no serviço. Nos eventos disruptivos o cliente perde totalmente o acesso ao serviço, enquanto que para problemas de degradação o serviço fica limitado e/ou pode falhar durante a utilização do mesmo. Em situações disruptivas, o cliente contacta immediatamente a empresa para apreentar uma queixa. Já nos casos de degradação o cliente pode não se queixar. Entre muitas razões possiveis sobressaiem as seguintes, o cliente não usa o serviço e o cliente prefere não passar pelo processo duro de contactar o operador para se queixar do mau serviço. A dissertação implementa uma solução que identifica clientes com degradação no serviço, para que a empresa possa ter uma atitude proactiva em relação aos clientes sinalizados, e consiga marcar intervenções técnicas para resolver o problema. A tese tem como objectivo validar que a interseção entre os serviços de TV e Internet é maior do que atualmente reconhecido, e que a interseção produz informação que deve ser explorada. No primeiro exercicio caracterizamos do problema, exploramos os dados reais de padrões de consumo, e é analisada a situação de interseção entre os serviços. Desenvolvemos o modelo segundo um arquitetura multi-task que permite a partilha de informação e portanto o modelo aprende a informação implícita que existe na interseção do serviços. O modelo multi-task corrige as limitações existentes nos modelos atuais, e permite a integração de outros problemas que a empresa venha a sentir necessidade de resolver. Os resultados obtidos quando comparados com os modelos de benchmark cuja ar- quitetura não suporta multi-task mostram um aumento de performance acima dos 25%. Devido aos resultados obtidos a estratégia para o projeto foi revista e novas ideias de projetos surgiram

Efficient Passive Clustering and Gateways selection MANETs

Passive clustering does not employ control packets to collect topological information in ad hoc networks. In our proposal, we avoid making frequent changes in cluster architecture due to repeated election and re-election of cluster heads and gateways. Our primary objective has been to make Passive Clustering more practical by employing optimal number of gateways and reduce the number of rebroadcast packets

Semantic search and composition in unstructured peer-to-peer networks

This dissertation focuses on several research questions in the area of semantic search and composition in unstructured peer-to-peer (P2P) networks. Going beyond the state of the art, the proposed semantic-based search strategy S2P2P offers a novel path-suggestion based query routing mechanism, providing a reasonable tradeoff between search performance and network traffic overhead. In addition, the first semantic-based data replication scheme DSDR is proposed. It enables peers to use semantic information to select replica numbers and target peers to address predicted future demands. With DSDR, k-random search can achieve better precision and recall than it can with a near-optimal non-semantic replication strategy. Further, this thesis introduces a functional automatic semantic service composition method, SPSC. Distinctively, it enables peers to jointly compose complex workflows with high cumulative recall but low network traffic overhead, using heuristic-based bidirectional haining and service memorization mechanisms. Its query branching method helps to handle dead-ends in a pruned search space. SPSC is proved to be sound and a lower bound of is completeness is given. Finally, this thesis presents iRep3D for semantic-index based 3D scene selection in P2P search. Its efficient retrieval scales to answer hybrid queries involving conceptual, functional and geometric aspects. iRep3D outperforms previous representative efforts in terms of search precision and efficiency.Diese Dissertation bearbeitet Forschungsfragen zur semantischen Suche und Komposition in unstrukturierten Peer-to-Peer Netzen(P2P). Die semantische Suchstrategie S2P2P verwendet eine neuartige Methode zur Anfrageweiterleitung basierend auf Pfadvorschlägen, welche den Stand der Wissenschaft übertrifft. Sie bietet angemessene Balance zwischen Suchleistung und Kommunikationsbelastung im Netzwerk. Außerdem wird das erste semantische System zur Datenreplikation genannt DSDR vorgestellt, welche semantische Informationen berücksichtigt vorhergesagten zukünftigen Bedarf optimal im P2P zu decken. Hierdurch erzielt k-random-Suche bessere Präzision und Ausbeute als mit nahezu optimaler nicht-semantischer Replikation. SPSC, ein automatisches Verfahren zur funktional korrekten Komposition semantischer Dienste, ermöglicht es Peers, gemeinsam komplexe Ablaufpläne zu komponieren. Mechanismen zur heuristischen bidirektionalen Verkettung und Rückstellung von Diensten ermöglichen hohe Ausbeute bei geringer Belastung des Netzes. Eine Methode zur Anfrageverzweigung vermeidet das Feststecken in Sackgassen im beschnittenen Suchraum. Beweise zur Korrektheit und unteren Schranke der Vollständigkeit von SPSC sind gegeben. iRep3D ist ein neuer semantischer Selektionsmechanismus für 3D-Modelle in P2P. iRep3D beantwortet effizient hybride Anfragen unter Berücksichtigung konzeptioneller, funktionaler und geometrischer Aspekte. Der Ansatz übertrifft vorherige Arbeiten bezüglich Präzision und Effizienz