Content-Aware Multimedia Communications

The demands for fast, economic and reliable dissemination of multimedia information are steadily growing within our society. While people and economy increasingly rely on communication technologies, engineers still struggle with their growing complexity. Complexity in multimedia communication originates from several sources. The most prominent is the unreliability of packet networks like the Internet. Recent advances in scheduling and error control mechanisms for streaming protocols have shown that the quality and robustness of multimedia delivery can be improved significantly when protocols are aware of the content they deliver. However, the proposed mechanisms require close cooperation between transport systems and application layers which increases the overall system complexity. Current approaches also require expensive metrics and focus on special encoding formats only. A general and efficient model is missing so far. This thesis presents efficient and format-independent solutions to support cross-layer coordination in system architectures. In particular, the first contribution of this work is a generic dependency model that enables transport layers to access content-specific properties of media streams, such as dependencies between data units and their importance. The second contribution is the design of a programming model for streaming communication and its implementation as a middleware architecture. The programming model hides the complexity of protocol stacks behind simple programming abstractions, but exposes cross-layer control and monitoring options to application programmers. For example, our interfaces allow programmers to choose appropriate failure semantics at design time while they can refine error protection and visibility of low-level errors at run-time. Based on some examples we show how our middleware simplifies the integration of stream-based communication into large-scale application architectures. An important result of this work is that despite cross-layer cooperation, neither application nor transport protocol designers experience an increase in complexity. Application programmers can even reuse existing streaming protocols which effectively increases system robustness.Der Bedarf unsere Gesellschaft nach kostengünstiger und zuverlässiger Kommunikation wächst stetig. Während wir uns selbst immer mehr von modernen Kommunikationstechnologien abhängig machen, müssen die Ingenieure dieser Technologien sowohl den Bedarf nach schneller Einführung neuer Produkte befriedigen als auch die wachsende Komplexität der Systeme beherrschen. Gerade die Übertragung multimedialer Inhalte wie Video und Audiodaten ist nicht trivial. Einer der prominentesten Gründe dafür ist die Unzuverlässigkeit heutiger Netzwerke, wie z.B.~dem Internet. Paketverluste und schwankende Laufzeiten können die Darstellungsqualität massiv beeinträchtigen. Wie jüngste Entwicklungen im Bereich der Streaming-Protokolle zeigen, sind jedoch Qualität und Robustheit der Übertragung effizient kontrollierbar, wenn Streamingprotokolle Informationen über den Inhalt der transportierten Daten ausnutzen. Existierende Ansätze, die den Inhalt von Multimediadatenströmen beschreiben, sind allerdings meist auf einzelne Kompressionsverfahren spezialisiert und verwenden berechnungsintensive Metriken. Das reduziert ihren praktischen Nutzen deutlich. Außerdem erfordert der Informationsaustausch eine enge Kooperation zwischen Applikationen und Transportschichten. Da allerdings die Schnittstellen aktueller Systemarchitekturen nicht darauf vorbereitet sind, müssen entweder die Schnittstellen erweitert oder alternative Architekturkonzepte geschaffen werden. Die Gefahr beider Varianten ist jedoch, dass sich die Komplexität eines Systems dadurch weiter erhöhen kann. Das zentrale Ziel dieser Dissertation ist es deshalb, schichtenübergreifende Koordination bei gleichzeitiger Reduzierung der Komplexität zu erreichen. Hier leistet die Arbeit zwei Beträge zum aktuellen Stand der Forschung. Erstens definiert sie ein universelles Modell zur Beschreibung von Inhaltsattributen, wie Wichtigkeiten und Abhängigkeitsbeziehungen innerhalb eines Datenstroms. Transportschichten können dieses Wissen zur effizienten Fehlerkontrolle verwenden. Zweitens beschreibt die Arbeit das Noja Programmiermodell für multimediale Middleware. Noja definiert Abstraktionen zur Übertragung und Kontrolle multimedialer Ströme, die die Koordination von Streamingprotokollen mit Applikationen ermöglichen. Zum Beispiel können Programmierer geeignete Fehlersemantiken und Kommunikationstopologien auswählen und den konkreten Fehlerschutz dann zur Laufzeit verfeinern und kontrolliere

Ubiquitous Scalable Graphics: An End-to-End Framework using Wavelets

Advances in ubiquitous displays and wireless communications have fueled the emergence of exciting mobile graphics applications including 3D virtual product catalogs, 3D maps, security monitoring systems and mobile games. Current trends that use cameras to capture geometry, material reflectance and other graphics elements means that very high resolution inputs is accessible to render extremely photorealistic scenes. However, captured graphics content can be many gigabytes in size, and must be simplified before they can be used on small mobile devices, which have limited resources, such as memory, screen size and battery energy. Scaling and converting graphics content to a suitable rendering format involves running several software tools, and selecting the best resolution for target mobile device is often done by trial and error, which all takes time. Wireless errors can also affect transmitted content and aggressive compression is needed for low-bandwidth wireless networks. Most rendering algorithms are currently optimized for visual realism and speed, but are not resource or energy efficient on mobile device. This dissertation focuses on the improvement of rendering performance by reducing the impacts of these problems with UbiWave, an end-to-end Framework to enable real time mobile access to high resolution graphics using wavelets. The framework tackles the issues including simplification, transmission, and resource efficient rendering of graphics content on mobile device based on wavelets by utilizing 1) a Perceptual Error Metric (PoI) for automatically computing the best resolution of graphics content for a given mobile display to eliminate guesswork and save resources, 2) Unequal Error Protection (UEP) to improve the resilience to wireless errors, 3) an Energy-efficient Adaptive Real-time Rendering (EARR) heuristic to balance energy consumption, rendering speed and image quality and 4) an Energy-efficient Streaming Technique. The results facilitate a new class of mobile graphics application which can gracefully adapt the lowest acceptable rendering resolution to the wireless network conditions and the availability of resources and battery energy on mobile device adaptively

Preserving data integrity of encoded medical images: the LAR compression framework

International audienceThrough the development of medical imaging systems and their integration into a complete information system, the need for advanced joint coding and network services becomes predominant. PACS (Picture Archiving and Communication System) aims to acquire, store and compress, retrieve, present and distribute medical images. These systems have to be accessible via the Internet or wireless channels. Thus protection processes against transmission errors have to be added to get a powerful joint source-channel coding tool. Moreover, these sensitive data require confidentiality and privacy for both archiving and transmission purposes, leading to use cryptography and data embedding solutions. This chapter introduces data integrity protection and developed dedicated tools of content protection and secure bitstream transmission for medical encoded image purposes. In particular, the LAR image coding method is defined together with advanced securization services

Historical information aware unequal error protection of scalable HEVC/H.265 streaming over free space optical channels

Free space optical (FSO) systems are capable of supporting high data rates between fixed points in the context of flawless video communications. Layered video coding facilitates the creation of different-resolution subset layers for variablethroughput transmission scenarios. In this paper, we propose Historical information Aware Unequal Error Protection (HAUEP) for the scalable high efficiency video codec (SHVC) used for streaming over FSO channels. Specifically, the objective function (OF) of the current video frame is designed based on historical information of its dependent frames. By optimizing this OF, specific subset layers may be selected in conjunction with carefully selected forward error correction (FEC) coding rates, where the expected video distortion is minimized and the required bitrate is reduced under the constraint of a specific throughput. Our simulation results show that the proposed system outperforms the traditional equal error protection (EEP) scheme by about 4.5 dB of Eb=N0 at a peak signal-to-noise ratio (PSNR) of 33 dB. From a throughput-oriented perspective, HA-UEP is capable of reducing the throughput to about 30% compared to that of the EEP benchmarker, while achieving an Eb=N0 gain of 4.5 dB

QUALITY-DRIVEN CROSS LAYER DESIGN FOR MULTIMEDIA SECURITY OVER RESOURCE CONSTRAINED WIRELESS SENSOR NETWORKS

The strong need for security guarantee, e.g., integrity and authenticity, as well as privacy and confidentiality in wireless multimedia services has driven the development of an emerging research area in low cost Wireless Multimedia Sensor Networks (WMSNs). Unfortunately, those conventional encryption and authentication techniques cannot be applied directly to WMSNs due to inborn challenges such as extremely limited energy, computing and bandwidth resources. This dissertation provides a quality-driven security design and resource allocation framework for WMSNs. The contribution of this dissertation bridges the inter-disciplinary research gap between high layer multimedia signal processing and low layer computer networking. It formulates the generic problem of quality-driven multimedia resource allocation in WMSNs and proposes a cross layer solution. The fundamental methodologies of multimedia selective encryption and stream authentication, and their application to digital image or video compression standards are presented. New multimedia selective encryption and stream authentication schemes are proposed at application layer, which significantly reduces encryption/authentication complexity. In addition, network resource allocation methodologies at low layers are extensively studied. An unequal error protection-based network resource allocation scheme is proposed to achieve the best effort media quality with integrity and energy efficiency guarantee. Performance evaluation results show that this cross layer framework achieves considerable energy-quality-security gain by jointly designing multimedia selective encryption/multimedia stream authentication and communication resource allocation

Energy efficient wireless transmission of MPEG-4 fine granular scalable video

Abstract- Fine granular scalability is a coding tool, recently introduced in the emerging MPEG-4 standard, which enables the creation of very flexible scalable video bitstreams. This paper investigates the transmission of fine granular scalable (FGS) video over wireless links, using power management for unequal error protection of the bitstream. In wireless systems, energy may be a limited resource, and a wise use of it is important for system efficiency. An algorithm is proposed which is able to optimally distribute the total available power for the transmission of the enhancement layer, given a distortion or energy constraint. Experimental results demonstrate the performance advantage of the proposed algorithm over fixed power schemes and heuristic approaches.

Robust and Scalable Transmission of Arbitrary 3D Models over Wireless Networks

We describe transmission of 3D objects represented by texture and mesh over unreliable networks, extending our earlier work for regular mesh structure to arbitrary meshes and considering linear versus cubic interpolation. Our approach to arbitrary meshes considers stripification of the mesh and distributing nearby vertices into different packets, combined with a strategy that does not need texture or mesh packets to be retransmitted. Only the valence (connectivity) packets need to be retransmitted; however, storage of valence information requires only 10% space compared to vertices and even less compared to photorealistic texture. Thus, less than 5% of the packets may need to be retransmitted in the worst case to allow our algorithm to successfully reconstruct an acceptable object under severe packet loss. Even though packet loss during transmission has received limited research attention in the past, this topic is important for improving quality under lossy conditions created by shadowing and interference. Results showing the implementation of the proposed approach using linear, cubic, and Laplacian interpolation are described, and the mesh reconstruction strategy is compared with other methods

A Study on the Usage of Cross-Layer Power Control and Forward Error Correction for Embedded Video Transmission over Wireless Links

Cross-layering is a design paradigm for overcoming the limitations deriving from the ISO/OSI layering principle, thus improving the performance of communications in specific scenarios, such as wireless multimedia communications. However, most available solutions are based on empirical considerations, and do not provide a theoretical background supporting such approaches. The paper aims at providing an analytical framework for the study of single-hop video delivery over a wireless link, enabling cross-layer interactions for performance optimization using power control and FEC and providing a useful tool to determine the potential gain deriving from the employment of such design paradigm. The analysis is performed using rate-distortion information of an embedded video bitstream jointly with a Lagrangian power minimization approach. Simulation results underline that cross-layering can provide relevant improvement in specific environments and that the proposed approach is able to capitalize on the advantage deriving from its deployment

Enhancement of Adaptive Forward Error Correction Mechanism for Video Transmission Over Wireless Local Area Network

Video transmission over the wireless network faces many challenges. The most critical challenge is related to packet loss. To overcome the problem of packet loss, Forward Error Correction is used by adding extra packets known as redundant packet or parity packet. Currently, FEC mechanisms have been adopted together with Automatic Repeat reQuest (ARQ) mechanism to overcome packet losses and avoid network congestion in various wireless network conditions. The number of FEC packets need to be generated effectively because wireless network usually has varying network conditions. In the current Adaptive FEC mechanism, the FEC packets are decided by the average queue length and average packet retransmission times. The Adaptive FEC mechanisms have been proposed to suit the network condition by generating FEC packets adaptively in the wireless network. However, the current Adaptive FEC mechanism has some major drawbacks such as the reduction of recovery performance which injects too many excessive FEC packets into the network. This is not flexible enough to adapt with varying wireless network condition. Therefore, the enhancement of Adaptive FEC mechanism (AFEC) known as Enhanced Adaptive FEC (EnAFEC) has been proposed. The aim is to improve recovery performance on the current Adaptive FEC mechanism by injecting FEC packets dynamically based on varying wireless network conditions. The EnAFEC mechanism is implemented in the simulation environment using Network Simulator 2 (NS-2). Performance evaluations are also carried out. The EnAFEC was tested with the random uniform error model. The results from experiments and performance analyses showed that EnAFEC mechanism outperformed the other Adaptive FEC mechanism in terms of recovery efficiency. Based on the findings, the optimal amount of FEC generated by EnAFEC mechanism can recover high packet loss and produce good video quality