Content-Aware Multimedia Communications

The demands for fast, economic and reliable dissemination of multimedia information are steadily growing within our society. While people and economy increasingly rely on communication technologies, engineers still struggle with their growing complexity. Complexity in multimedia communication originates from several sources. The most prominent is the unreliability of packet networks like the Internet. Recent advances in scheduling and error control mechanisms for streaming protocols have shown that the quality and robustness of multimedia delivery can be improved significantly when protocols are aware of the content they deliver. However, the proposed mechanisms require close cooperation between transport systems and application layers which increases the overall system complexity. Current approaches also require expensive metrics and focus on special encoding formats only. A general and efficient model is missing so far. This thesis presents efficient and format-independent solutions to support cross-layer coordination in system architectures. In particular, the first contribution of this work is a generic dependency model that enables transport layers to access content-specific properties of media streams, such as dependencies between data units and their importance. The second contribution is the design of a programming model for streaming communication and its implementation as a middleware architecture. The programming model hides the complexity of protocol stacks behind simple programming abstractions, but exposes cross-layer control and monitoring options to application programmers. For example, our interfaces allow programmers to choose appropriate failure semantics at design time while they can refine error protection and visibility of low-level errors at run-time. Based on some examples we show how our middleware simplifies the integration of stream-based communication into large-scale application architectures. An important result of this work is that despite cross-layer cooperation, neither application nor transport protocol designers experience an increase in complexity. Application programmers can even reuse existing streaming protocols which effectively increases system robustness.Der Bedarf unsere Gesellschaft nach kostengünstiger und zuverlässiger Kommunikation wächst stetig. Während wir uns selbst immer mehr von modernen Kommunikationstechnologien abhängig machen, müssen die Ingenieure dieser Technologien sowohl den Bedarf nach schneller Einführung neuer Produkte befriedigen als auch die wachsende Komplexität der Systeme beherrschen. Gerade die Übertragung multimedialer Inhalte wie Video und Audiodaten ist nicht trivial. Einer der prominentesten Gründe dafür ist die Unzuverlässigkeit heutiger Netzwerke, wie z.B.~dem Internet. Paketverluste und schwankende Laufzeiten können die Darstellungsqualität massiv beeinträchtigen. Wie jüngste Entwicklungen im Bereich der Streaming-Protokolle zeigen, sind jedoch Qualität und Robustheit der Übertragung effizient kontrollierbar, wenn Streamingprotokolle Informationen über den Inhalt der transportierten Daten ausnutzen. Existierende Ansätze, die den Inhalt von Multimediadatenströmen beschreiben, sind allerdings meist auf einzelne Kompressionsverfahren spezialisiert und verwenden berechnungsintensive Metriken. Das reduziert ihren praktischen Nutzen deutlich. Außerdem erfordert der Informationsaustausch eine enge Kooperation zwischen Applikationen und Transportschichten. Da allerdings die Schnittstellen aktueller Systemarchitekturen nicht darauf vorbereitet sind, müssen entweder die Schnittstellen erweitert oder alternative Architekturkonzepte geschaffen werden. Die Gefahr beider Varianten ist jedoch, dass sich die Komplexität eines Systems dadurch weiter erhöhen kann. Das zentrale Ziel dieser Dissertation ist es deshalb, schichtenübergreifende Koordination bei gleichzeitiger Reduzierung der Komplexität zu erreichen. Hier leistet die Arbeit zwei Beträge zum aktuellen Stand der Forschung. Erstens definiert sie ein universelles Modell zur Beschreibung von Inhaltsattributen, wie Wichtigkeiten und Abhängigkeitsbeziehungen innerhalb eines Datenstroms. Transportschichten können dieses Wissen zur effizienten Fehlerkontrolle verwenden. Zweitens beschreibt die Arbeit das Noja Programmiermodell für multimediale Middleware. Noja definiert Abstraktionen zur Übertragung und Kontrolle multimedialer Ströme, die die Koordination von Streamingprotokollen mit Applikationen ermöglichen. Zum Beispiel können Programmierer geeignete Fehlersemantiken und Kommunikationstopologien auswählen und den konkreten Fehlerschutz dann zur Laufzeit verfeinern und kontrolliere

On the Stability of Distribution Topologies in Peer-to-Peer Live Streaming Systems

Peer-to-Peer Live-Streaming-Systeme sind ständigen Störungen ausgesetzt.Insbesondere ermöglichen unzuverlässige Teilnehmer Ausfälle und Angriffe, welche überraschend Peers aus dem System entfernen. Die Folgen solcher Vorfälle werden großteils von der Verteilungstopologie bestimmt, d.h. der Kommunikationsstruktur zwischen den Peers.In dieser Arbeit analysieren wir Optimierungsprobleme welche bei der Betrachtung von Stabilitätsbegriffen für solche Verteilungstopologien auftreten. Dabei werden sowohl Angriffe als auch unkoordinierte Ausfälle berücksichtigt.Zunächst untersuchen wir die Berechnungskomplexität und Approximierbarkeit des Problems resourcen-effiziente Angriffe zu bestimmen. Dies demonstriert Beschränkungen in den Planungsmöglichkeiten von Angreifern und zeigt inwieweit die Topologieparameter die Schwierigkeit solcher Angriffsrobleme beeinflussen. Anschließend studieren wir Topologieformationsprobleme. Dabei sind Topologieparameter vorgegeben und es muss eine passende Verteilungstopologie gefunden werden. Ziel ist es Topologien zu erzeugen, welche den durch Angriffe mit beliebigen Parametern erzeugbaren maximalen Schaden minimieren.Wir identifizieren notwendige und hinreichende Eigenschaften solcher Verteilungstopologien. Dies führt zu mathematisch fundierten Zielstellungen für das Topologie-Management von Peer-to-Peer Live-Streaming-Systemen.Wir zeigen zwei große Klassen effizient konstruierbarer Verteilungstopologien, welche den maximal möglichen, durch Angriffe verursachten Paketverlust minimieren. Zusätzlich beweisen wir, dass die Bestimmung dieser Eigenschaft für beliebige Topologien coNP-vollständig ist.Soll die maximale Anzahl von Peers minimiert werden, bei denen ein Angriff zu ungenügender Stream-Qualität führt, ändern sich die Anforderungen an Verteilungstopologien. Wir zeigen, dass dieses Topologieformationsproblem eng mit offenen Problemen aus Design- und Kodierungstheorie verwandt ist.Schließlich analysieren wir Verteilungstopologien die den durch unkoordinierte Ausfälle zu erwartetenden Paketverlust minimieren. Wir zeigen Eigenschaften und Existenzbedingungen. Außerdem bestimmen wir die Berechnungskomplexität des Auffindens solcher Topologien. Unsere Ergebnisse liefern Richtlinien für das Topologie-Management von Peer-to-Peer Live-Streaming-Systemen und zeigen auf, welche Stabilitätsziele effizient erreicht werden können.The stability of peer-to-peer live streaming systems is constantly challenged. Especially, the unreliability and vulnerability of their participants allows for failures and attacks suddenly disabling certain sets of peers. The consequences of such events are largely determined by the distribution topology, i.e., the pattern of communication between the peers.In this thesis, we analyze a broad range of optimization problems concerning the stability of distribution topologies. For this, we discuss notions of stability against both attacks and failures.At first, we investigate the computational complexity and approximability of finding resource-efficient attacks. This allows to point out limitations of an attacker's planning capabilities and demonstrates the influence of the chosen system parameters on the hardness of such attack problems.Then, we turn to study topology formation problems. Here, a set of topology parameters is given and the task consists in finding an eligible distribution topology. In particular, it has to minimize the maximum damage achievable by attacks with arbitrary attack parameters.We identify necessary and sufficient conditions on attack-stable distribution topologies. Thereby, we give mathematically sound guidelines for the topology management of peer-to-peer live streaming systems.We find large classes of efficiently-constructable topologies minimizing the system-wide packet loss under attacks. Additionally, we show that determining this feature for arbitrary topologies is coNP-complete.Considering topologies minimizing the maximum number of peers for which an attack leads to a heavy decrease in perceived streaming quality, the requirements change. Here, we show that the corresponding topology formation problem is closely related to long-standing open problems of Design and Coding Theory.Finally, we study topologies minimizing the expected packet loss due to uncoordinated peer failures. We investigate properties and existence conditions of such topologies. Furthermore, we determine the computational complexity of constructing them.Our results provide guidelines for the topology management of peer-to-peer live streaming systems and mathematically determine which goals can be achieved efficiently

Smart PIN: performance and cost-oriented context-aware personal information network

The next generation of networks will involve interconnection of heterogeneous individual networks such as WPAN, WLAN, WMAN and Cellular network, adopting the IP as common infrastructural protocol and providing virtually always-connected network. Furthermore, there are many devices which enable easy acquisition and storage of information as pictures, movies, emails, etc. Therefore, the information overload and divergent content’s characteristics make it difficult for users to handle their data in manual way. Consequently, there is a need for personalised automatic services which would enable data exchange across heterogeneous network and devices. To support these personalised services, user centric approaches for data delivery across the heterogeneous network are also required. In this context, this thesis proposes Smart PIN - a novel performance and cost-oriented context-aware Personal Information Network. Smart PIN's architecture is detailed including its network, service and management components. Within the service component, two novel schemes for efficient delivery of context and content data are proposed: Multimedia Data Replication Scheme (MDRS) and Quality-oriented Algorithm for Multiple-source Multimedia Delivery (QAMMD). MDRS supports efficient data accessibility among distributed devices using data replication which is based on a utility function and a minimum data set. QAMMD employs a buffer underflow avoidance scheme for streaming, which achieves high multimedia quality without content adaptation to network conditions. Simulation models for MDRS and QAMMD were built which are based on various heterogeneous network scenarios. Additionally a multiple-source streaming based on QAMMS was implemented as a prototype and tested in an emulated network environment. Comparative tests show that MDRS and QAMMD perform significantly better than other approaches

Flexible Application-Layer Multicast in Heterogeneous Networks

This work develops a set of peer-to-peer-based protocols and extensions in order to provide Internet-wide group communication. The focus is put to the question how different access technologies can be integrated in order to face the growing traffic load problem. Thereby, protocols are developed that allow autonomous adaptation to the current network situation on the one hand and the integration of WiFi domains where applicable on the other hand

Liquid stream processing on the web: a JavaScript framework

The Web is rapidly becoming a mature platform to host distributed applications. Pervasive computing application running on the Web are now common in the era of the Web of Things, which has made it increasingly simple to integrate sensors and microcontrollers in our everyday life. Such devices are of great in- terest to Makers with basic Web development skills. With them, Makers are able to build small smart stream processing applications with sensors and actuators without spending a fortune and without knowing much about the technologies they use. Thanks to ongoing Web technology trends enabling real-time peer-to- peer communication between Web-enabled devices, Web browsers and server- side JavaScript runtimes, developers are able to implement pervasive Web ap- plications using a single programming language. These can take advantage of direct and continuous communication channels going beyond what was possible in the early stages of the Web to push data in real-time. Despite these recent advances, building stream processing applications on the Web of Things remains a challenging task. On the one hand, Web-enabled devices of different nature still have to communicate with different protocols. On the other hand, dealing with a dynamic, heterogeneous, and volatile environment like the Web requires developers to face issues like disconnections, unpredictable workload fluctuations, and device overload. To help developers deal with such issues, in this dissertation we present the Web Liquid Streams (WLS) framework, a novel streaming framework for JavaScript. Developers implement streaming operators written in JavaScript and may interactively and dynamically define a streaming topology. The framework takes care of deploying the user-defined operators on the available devices and connecting them using the appropriate data channel, removing the burden of dealing with different deployment environments from the developers. Changes in the semantic of the application and in its execution environment may be ap- plied at runtime without stopping the stream flow. Like a liquid adapts its shape to the one of its container, the Web Liquid Streams framework makes streaming topologies flow across multiple heterogeneous devices, enabling dynamic operator migration without disrupting the data flow. By constantly monitoring the execution of the topology with a hierarchical controller infrastructure, WLS takes care of parallelising the operator execution across multiple devices in case of bottlenecks and of recovering the execution of the streaming topology in case one or more devices disconnect, by restarting lost operators on other available devices

Network coding: from theory to media streaming

Network coding has recently emerged as an alternative to traditional routing algorithms in communication systems. In network coding, the network nodes can combine the packets they receive before forwarding them to the neighbouring nodes. Intensive research efforts have demonstrated that such a processing in the network nodes can provide advantages in terms of throughput or robustness. These potentials, combined with the advent of ad hoc and wireless delivery architectures have triggered the interest of research community about the application of the network coding principles to streaming applications. This paper describes the potentials of network coding in emerging delivery architectures such as overlay or peer-to-peer networks. It overviews the principles of practical network coding algorithms and outlines the challenges posed by multimedia streaming applications. Finally, it provides a survey of the recent work on the application of network coding to media streaming applications, both in wireless or wired communication scenarios. Promising results have been demonstrated where network coding is able to bring benefits in media streaming applications. However, delay and complexity constraints are often posed as the main challenging issues that still prevent the wide-scale deployment of network coding algorithms in multimedia communication

Ontwerp en evaluatie van content distributie netwerken voor multimediale streaming diensten.

Traditionele Internetgebaseerde diensten voor het verspreiden van bestanden, zoals Web browsen en het versturen van e-mails, worden aangeboden via één centrale server. Meer recente netwerkdiensten zoals interactieve digitale televisie of video-op-aanvraag vereisen echter hoge kwaliteitsgaranties (QoS), zoals een lage en constante netwerkvertraging, en verbruiken een aanzienlijke hoeveelheid bandbreedte op het netwerk. Architecturen met één centrale server kunnen deze garanties moeilijk bieden en voldoen daarom niet meer aan de hoge eisen van de volgende generatie multimediatoepassingen. In dit onderzoek worden daarom nieuwe netwerkarchitecturen bestudeerd, die een dergelijke dienstkwaliteit kunnen ondersteunen. Zowel peer-to-peer mechanismes, zoals bij het uitwisselen van muziekbestanden tussen eindgebruikers, als servergebaseerde oplossingen, zoals gedistribueerde caches en content distributie netwerken (CDN's), komen aan bod. Afhankelijk van de bestudeerde dienst en de gebruikte netwerktechnologieën en -architectuur, worden gecentraliseerde algoritmen voor netwerkontwerp voorgesteld. Deze algoritmen optimaliseren de plaatsing van de servers of netwerkcaches en bepalen de nodige capaciteit van de servers en netwerklinks. De dynamische plaatsing van de aangeboden bestanden in de verschillende netwerkelementen wordt aangepast aan de heersende staat van het netwerk en aan de variërende aanvraagpatronen van de eindgebruikers. Serverselectie, herroutering van aanvragen en het verspreiden van de belasting over het hele netwerk komen hierbij ook aan bod

Network coding for transport protocols

With the proliferation of smart devices that require Internet connectivity anytime, anywhere, and the recent technological advances that make it possible, current networked systems will have to provide a various range of services, such as content distribution, in a wide range of settings, including wireless environments. Wireless links may experience temporary losses, however, TCP, the de facto protocol for robust unicast communications, reacts by reducing the congestion window drastically and injecting less traffic in the network. Consequently the wireless links are underutilized and the overall performance of the TCP protocol in wireless environments is poor. As content delivery (i.e. multicasting) services, such as BBC iPlayer, become popular, the network needs to support the reliable transport of the data at high rates, and with specific delay constraints. A typical approach to deliver content in a scalable way is to rely on peer-to-peer technology (used by BitTorrent, Spotify and PPLive), where users share their resources, including bandwidth, storage space, and processing power. Still, these systems suffer from the lack of incentives for resource sharing and cooperation, and this problem is exacerbated in the presence of heterogenous users, where a tit-for-tat scheme is difficult to implement. Due to the issues highlighted above, current network architectures need to be changed in order to accommodate the users¿ demands for reliable and quality communications. In other words, the emergent need for advanced modes of information transport requires revisiting and improving network components at various levels of the network stack. The innovative paradigm of network coding has been shown as a promising technique to change the design of networked systems, by providing a shift from how data flows traditionally move through the network. This shift implies that data flows are no longer kept separate, according to the ¿store-and-forward¿ model, but they are also processed and mixed in the network. By appropriately combining data by means of network coding, it is expected to obtain significant benefits in several areas of network design and architecture. In this thesis, we set out to show the benefits of including network coding into three communication paradigms, namely point-topoint communications (e.g. unicast), point-to-multipoint communications (e.g. multicast), and multipoint-to-multipoint communications (e.g. peer-to-peer networks). For the first direction, we propose a network coding-based multipath scheme and show that TCP unicast sessions are feasible in highly volatile wireless environments. For point-to-multipoint communications, we give an algorithm to optimally achieve all the rate pairs from the rate region in the case of degraded multicast over the combination network. We also propose a system for live streaming that ensures reliability and quality of service to heterogenous users, even if data transmissions occur over lossy wireless links. Finally, for multipoint-to-multipoint communications, we design a system to provide incentives for live streaming in a peer-to-peer setting, where users have subscribed to different levels of quality. Our work shows that network coding enables a reliable transport of data, even in highly volatile environments, or in delay sensitive scenarios such as live streaming, and facilitates the implementation of an efficient incentive system, even in the presence of heterogenous users. Thus, network coding can solve the challenges faced by next generation networks in order to support advanced information transport.Postprint (published version