Packet loss visibility across SD, HD, 3D, and UHD video streams

The trend towards video streaming with increased spatial resolutions and dimensions, SD, HD, 3D, and 4kUHD, even for portable devices has important implications for displayed video quality. There is an interplay between packetization, packet loss visibility, choice of codec, and viewing conditions, which implies that prior studies at lower resolutions may not be as relevant. This paper presents two sets of experiments, the one at a Variable BitRate (VBR) and the other at a Constant BitRate (CBR), which highlight different aspects of the interpretation. The latter experiments also compare and contrast encoding with either an H.264 or an High Efficiency Video Coding (HEVC) codec, with all results recorded as objective Mean Opinion Score (MOS). The video quality assessments will be of interest to those considering: the bitrates and expected quality in error-prone environments; or, in fact, whether to use a reliable transport protocol to prevent all errors, at a cost in jitter and latency, rather than tolerate low levels of packet errors

A robust error detection mechanism for H.264/AVC coded video sequences based on support vector machines

Current trends in wireless communications provide fast and location-independent access to multimedia services. Due to its high compression efficiency, H.264/AVC is expected to become the dominant underlying technology in the delivery of future wireless video applications. The error resilient mechanisms adopted by this standard alleviate the problem of spatio-temporal propagation of visual artifacts caused by transmission errors by dropping and concealing all macroblocks (MBs) contained within corrupted segments, including uncorrupted MBs. Concealing these uncorrupted MBs generally causes a reduction in quality of the reconstructed video sequence.peer-reviewe

Video Packet Priority Assignment Based On Spatio-Temporal Perceptual Importance

A novel perceptually motivated two-stage algorithm for assigning priority to video packet data to be transmitted over the internet is proposed. Priority assignment is based on temporal and spatial features that are derived from low-level vision concepts. The motivation for a two-stage design is to be able to handle different application settings. The first stage of the algorithm is computationally very efficient and can be directly used in low-delay applications with limited computational resources. The two-stage method performs exceedingly well across a variety of content and can be used in less restrictive operating settings. The efficacy of the proposed algorithm (both stages) is demonstrated using an intelligent packet drop application where it is compared with cumulative mean squared error (cMSE) based priority assignment and random packet dropping. The proposed prioritization algorithm allows for packet drops that result in significantly lower perceptual annoyance at the receiver relative to the other methods considered

Video CODEC with adaptive frame rate control for intelligent transportation system applications

Video cameras are one of the important types of devices in Intelligent Transportation Systems (ITS). The camera images are practical, widely deployable and beneficial for traffic management and congestion control. The advent of image processing has established several applications based on ITS camera images, including vehicle detection, weather monitoring, smart work zones, etc. Unlike digital video entertainment applications, the camera images in ITS applications require high video image quality but usually not a high video frame rate. Traditional block-based video compression standards, which were developed primarily with the video entertainment industry in mind, are dependent on adaptive rate control algorithms to control the video quality and the video frame rate. Modern rate control algorithms range from simple frame skipping to complicated adaptive algorithms based on optimal rate-distortion theory. In this dissertation, I presented an innovative video frame rate control scheme based on adaptive frame dropping. Video transmission schemes were also discussed and a new strategy to reduce the video traffic on the network was presented. Experimental results in a variety of network scenarios shown that the proposed technique could improve video quality in both the temporal and spatial dimensions, as quantified by standard video metrics (up to 6 percent of PSNR, 5 percent of SSIM, and 10 percent VQM compared to the original video). Another benefit of the proposed technique is that video traffic and network congestion are generally reduced. Both FPGA and embedded Linux implementations are considered for video encoder development

Content-Aware Multimedia Communications

The demands for fast, economic and reliable dissemination of multimedia information are steadily growing within our society. While people and economy increasingly rely on communication technologies, engineers still struggle with their growing complexity. Complexity in multimedia communication originates from several sources. The most prominent is the unreliability of packet networks like the Internet. Recent advances in scheduling and error control mechanisms for streaming protocols have shown that the quality and robustness of multimedia delivery can be improved significantly when protocols are aware of the content they deliver. However, the proposed mechanisms require close cooperation between transport systems and application layers which increases the overall system complexity. Current approaches also require expensive metrics and focus on special encoding formats only. A general and efficient model is missing so far. This thesis presents efficient and format-independent solutions to support cross-layer coordination in system architectures. In particular, the first contribution of this work is a generic dependency model that enables transport layers to access content-specific properties of media streams, such as dependencies between data units and their importance. The second contribution is the design of a programming model for streaming communication and its implementation as a middleware architecture. The programming model hides the complexity of protocol stacks behind simple programming abstractions, but exposes cross-layer control and monitoring options to application programmers. For example, our interfaces allow programmers to choose appropriate failure semantics at design time while they can refine error protection and visibility of low-level errors at run-time. Based on some examples we show how our middleware simplifies the integration of stream-based communication into large-scale application architectures. An important result of this work is that despite cross-layer cooperation, neither application nor transport protocol designers experience an increase in complexity. Application programmers can even reuse existing streaming protocols which effectively increases system robustness.Der Bedarf unsere Gesellschaft nach kostengünstiger und zuverlässiger Kommunikation wächst stetig. Während wir uns selbst immer mehr von modernen Kommunikationstechnologien abhängig machen, müssen die Ingenieure dieser Technologien sowohl den Bedarf nach schneller Einführung neuer Produkte befriedigen als auch die wachsende Komplexität der Systeme beherrschen. Gerade die Übertragung multimedialer Inhalte wie Video und Audiodaten ist nicht trivial. Einer der prominentesten Gründe dafür ist die Unzuverlässigkeit heutiger Netzwerke, wie z.B.~dem Internet. Paketverluste und schwankende Laufzeiten können die Darstellungsqualität massiv beeinträchtigen. Wie jüngste Entwicklungen im Bereich der Streaming-Protokolle zeigen, sind jedoch Qualität und Robustheit der Übertragung effizient kontrollierbar, wenn Streamingprotokolle Informationen über den Inhalt der transportierten Daten ausnutzen. Existierende Ansätze, die den Inhalt von Multimediadatenströmen beschreiben, sind allerdings meist auf einzelne Kompressionsverfahren spezialisiert und verwenden berechnungsintensive Metriken. Das reduziert ihren praktischen Nutzen deutlich. Außerdem erfordert der Informationsaustausch eine enge Kooperation zwischen Applikationen und Transportschichten. Da allerdings die Schnittstellen aktueller Systemarchitekturen nicht darauf vorbereitet sind, müssen entweder die Schnittstellen erweitert oder alternative Architekturkonzepte geschaffen werden. Die Gefahr beider Varianten ist jedoch, dass sich die Komplexität eines Systems dadurch weiter erhöhen kann. Das zentrale Ziel dieser Dissertation ist es deshalb, schichtenübergreifende Koordination bei gleichzeitiger Reduzierung der Komplexität zu erreichen. Hier leistet die Arbeit zwei Beträge zum aktuellen Stand der Forschung. Erstens definiert sie ein universelles Modell zur Beschreibung von Inhaltsattributen, wie Wichtigkeiten und Abhängigkeitsbeziehungen innerhalb eines Datenstroms. Transportschichten können dieses Wissen zur effizienten Fehlerkontrolle verwenden. Zweitens beschreibt die Arbeit das Noja Programmiermodell für multimediale Middleware. Noja definiert Abstraktionen zur Übertragung und Kontrolle multimedialer Ströme, die die Koordination von Streamingprotokollen mit Applikationen ermöglichen. Zum Beispiel können Programmierer geeignete Fehlersemantiken und Kommunikationstopologien auswählen und den konkreten Fehlerschutz dann zur Laufzeit verfeinern und kontrolliere