A New Seamless Bitstream Switching Scheme for H.264 Video Adaptation with Enhanced Coding Performance

[[abstract]]In this paper, we propose a new seamless bitstream switching scheme to improve the coding performance of H.264 SP-frames for rate adaptation. Our method removes one of the two re-quantization blocks in the SP-frame encoders so as to significantly improve coding performance. The seamless switching property of SP-frames is retained by properly restructuring the primary and secondary switching frame codecs. Experimental results show that our proposed scheme achieves close coding performance to that of regular H.264 P-frames and significantly better performance than that of SP-frames. The proposed method also provides the advantage of using a single secondary switching bitstream for both switching-up and switching-down processes[[fileno]]2030144030014[[department]]電機工程學

Video processing for panoramic streaming using HEVC and its scalable extensions

Panoramic streaming is a particular way of video streaming where an arbitrary Region-of-Interest (RoI) is transmitted from a high-spatial resolution video, i.e. a video covering a very “wide-angle” (much larger than the human field-of-view – e.g. 360°). Some transport schemes for panoramic video delivery have been proposed and demonstrated within the past decade, which allow users to navigate interactively within the high-resolution videos. With the recent advances of head mounted displays, consumers may soon have immersive and sufficiently convenient end devices at reach, which could lead to an increasing demand for panoramic video experiences. The solution proposed within this paper is built upon tile-based panoramic streaming, where users receive a set of tiles that match their RoI, and consists in a low-complexity compressed domain video processing technique for using H.265/HEVC and its scalable extensions (H.265/SHVC and H.265/MV-HEVC). The proposed technique generates a single video bitstream out of the selected tiles so that a single hardware decoder can be used. It overcomes the scalability issue of previous solutions not using tiles and the battery consumption issue inherent of tile-based panorama streaming, where multiple parallel software decoders are used. In addition, the described technique is capable of reducing peak streaming bitrate during changes of the RoI, which is crucial for allowing a truly immersive and low latency video experience. Besides, it makes it possible to use Open GOP structures without incurring any playback interruption at switching events, which provides a better compression efficiency compared to closed GOP structures

Seamless Multimedia Delivery Within a Heterogeneous Wireless Networks Environment: Are We There Yet?

The increasing popularity of live video streaming from mobile devices, such as Facebook Live, Instagram Stories, Snapchat, etc. pressurizes the network operators to increase the capacity of their networks. However, a simple increase in system capacity will not be enough without considering the provisioning of quality of experience (QoE) as the basis for network control, customer loyalty, and retention rate and thus increase in network operators revenue. As QoE is gaining strong momentum especially with increasing users' quality expectations, the focus is now on proposing innovative solutions to enable QoE when delivering video content over heterogeneous wireless networks. In this context, this paper presents an overview of multimedia delivery solutions, identifies the problems and provides a comprehensive classification of related state-of-the-art approaches following three key directions: 1) adaptation; 2) energy efficiency; and 3) multipath content delivery. Discussions, challenges, and open issues on the seamless multimedia provisioning faced by the current and next generation of wireless networks are also provided

Seamless multimedia delivery within a heterogeneous wireless networks environment: are we there yet?

The increasing popularity of live video streaming from mobile devices such as Facebook Live, Instagram Stories, Snapchat, etc. pressurises the network operators to increase the capacity of their networks. However, a simple increase in system capacity will not be enough without considering the provisioning of Quality of Experience (QoE) as the basis for network control, customer loyalty and retention rate and thus increase in network operators revenue. As QoE is gaining strong momentum especially with increasing users’ quality expectations, the focus is now on proposing innovative solutions to enable QoE when delivering video content over heterogeneous wireless networks. In this context, this paper presents an overview of multimedia delivery solutions, identifies the problems and provides a comprehensive classification of related state-of-the-art approaches following three key directions: adaptation, energy efficiency and multipath content delivery. Discussions, challenges and open issues on the seamless multimedia provisioning faced by the current and next generation of wireless networks are also provided

Description-driven Adaptation of Media Resources

The current multimedia landscape is characterized by a significant diversity in terms of available media formats, network technologies, and device properties. This heterogeneity has resulted in a number of new challenges, such as providing universal access to multimedia content. A solution for this diversity is the use of scalable bit streams, as well as the deployment of a complementary system that is capable of adapting scalable bit streams to the constraints imposed by a particular usage environment (e.g., the limited screen resolution of a mobile device). This dissertation investigates the use of an XML-driven (Extensible Markup Language) framework for the format-independent adaptation of scalable bit streams. Using this approach, the structure of a bit stream is first translated into an XML description. In a next step, the resulting XML description is transformed to reflect a desired adaptation of the bit stream. Finally, the transformed XML description is used to create an adapted bit stream that is suited for playback in the targeted usage environment. The main contribution of this dissertation is BFlavor, a new tool for exposing the syntax of binary media resources as an XML description. Its development was inspired by two other technologies, i.e. MPEG-21 BSDL (Bitstream Syntax Description Language) and XFlavor (Formal Language for Audio-Visual Object Representation, extended with XML features). Although created from a different point of view, both languages offer solutions for translating the syntax of a media resource into an XML representation for further processing. BFlavor (BSDL+XFlavor) harmonizes the two technologies by combining their strengths and eliminating their weaknesses. The expressive power and performance of a BFlavor-based content adaptation chain, compared to tool chains entirely based on either BSDL or XFlavor, were investigated by several experiments. One series of experiments targeted the exploitation of multi-layered temporal scalability in H.264/AVC, paying particular attention to the use of sub-sequences and hierarchical coding patterns, as well as to the use of metadata messages to communicate the bit stream structure to the adaptation logic. BFlavor was the only tool to offer an elegant and practical solution for XML-driven adaptation of H.264/AVC bit streams in the temporal domain

Design of interface selection protocols for multi-homed wireless networks

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel University on 10 December 2010.The IEEE 802.11/802.16 standards conformant wireless communication stations have multi-homing transmission capability. To achieve greater communication efficiency, multi-homing capable stations use handover mechanism to select appropriate transmission channel according to variations in the channel quality. This thesis presents three internal-linked handover schemes, (1) Interface Selection Protocol (ISP), belonging to Wireless Local Area Network (WLAN)- Worldwide Interoperability for Microwave Access (WiMAX) environment (2) Fast Channel Scanning (FCS) and (3) Traffic Manager (TM), (2) and (3) belonging to WiMAX Environment. The proposed schemes in this thesis use a novel mechanism of providing a reliable communication route. This solution is based on a cross-layer communication framework, where the interface selection module uses various network related parameters from Medium Access Control (MAC) sub-layer/Physical Layer (PHY) across the protocol suite for decision making at the Network layer. The proposed solutions are highly responsive when compared with existing multi-homed schemes; responsiveness is one of the key factors in the design of such protocols. Selected route under these schemes is based on the most up to date link-layer information. Therefore, such a route is not only reliable in terms of route optimization but it also fulfils the application demands in terms of throughput and delay. Design of ISP protocol use probing frames during the route discovery process. The 802.11 mandates the use of different rates for data transmission frames. The ISP-metric can be incorporated into various routing aspects and its applicability is determined by the possibility of provision of MAC dependent parameters that are used to determine the best path metric values. In many cases, higher device density, interference and mobility cause variable medium access delays. It causes creation of ‘unreachable zones’, where destination is marked as unreachable. However, by use of the best path metric, the destination has been made reachable, anytime and anywhere, because of the intelligent use of the probing frames and interface selection algorithm implemented. The IEEE 802.16e introduces several MAC level queues for different access categories, maintaining service requirement within these queues; which imply that frames from a higher priority queue, i.e. video frames, are serviced more frequently than those belonging to lower priority queues. Such an enhancement at the MAC sub-layer introduces uneven queuing delays. Conventional routing protocols are unaware of such MAC specific constraints and as a result, these factors are not considered which result in channel performance degradation. To meet such challenges, the thesis presents FCS and TM schemes for WiMAX. For FCS, Its solution is to improve the mobile WiMAX handover and address the scanning latency. Since minimum scanning time is the most important issue in the handover process. This handover scheme aims to utilize the channel efficiently and apply such a procedure to reduce the time it takes to scan the neighboring access stations. TM uses MAC and physical layer (PHY) specific information in the interface metric and maintains a separate path to destination by applying an alternative interface operation. Simulation tests and comparisons with existing multi-homed protocols and handover schemes demonstrate the effectiveness of incorporating the medium dependent parameters. Moreover, show that suggested schemes, have shown better performance in terms of end-to-end delay and throughput, with efficiency up to 40% in specific test scenarios

Development and implementation of an adaptive HTTP streaming fFramework for H.264

Ingeniero (a) ElectrónicoPregrad

Content-Aware Multimedia Communications

The demands for fast, economic and reliable dissemination of multimedia information are steadily growing within our society. While people and economy increasingly rely on communication technologies, engineers still struggle with their growing complexity. Complexity in multimedia communication originates from several sources. The most prominent is the unreliability of packet networks like the Internet. Recent advances in scheduling and error control mechanisms for streaming protocols have shown that the quality and robustness of multimedia delivery can be improved significantly when protocols are aware of the content they deliver. However, the proposed mechanisms require close cooperation between transport systems and application layers which increases the overall system complexity. Current approaches also require expensive metrics and focus on special encoding formats only. A general and efficient model is missing so far. This thesis presents efficient and format-independent solutions to support cross-layer coordination in system architectures. In particular, the first contribution of this work is a generic dependency model that enables transport layers to access content-specific properties of media streams, such as dependencies between data units and their importance. The second contribution is the design of a programming model for streaming communication and its implementation as a middleware architecture. The programming model hides the complexity of protocol stacks behind simple programming abstractions, but exposes cross-layer control and monitoring options to application programmers. For example, our interfaces allow programmers to choose appropriate failure semantics at design time while they can refine error protection and visibility of low-level errors at run-time. Based on some examples we show how our middleware simplifies the integration of stream-based communication into large-scale application architectures. An important result of this work is that despite cross-layer cooperation, neither application nor transport protocol designers experience an increase in complexity. Application programmers can even reuse existing streaming protocols which effectively increases system robustness.Der Bedarf unsere Gesellschaft nach kostengünstiger und zuverlässiger Kommunikation wächst stetig. Während wir uns selbst immer mehr von modernen Kommunikationstechnologien abhängig machen, müssen die Ingenieure dieser Technologien sowohl den Bedarf nach schneller Einführung neuer Produkte befriedigen als auch die wachsende Komplexität der Systeme beherrschen. Gerade die Übertragung multimedialer Inhalte wie Video und Audiodaten ist nicht trivial. Einer der prominentesten Gründe dafür ist die Unzuverlässigkeit heutiger Netzwerke, wie z.B.~dem Internet. Paketverluste und schwankende Laufzeiten können die Darstellungsqualität massiv beeinträchtigen. Wie jüngste Entwicklungen im Bereich der Streaming-Protokolle zeigen, sind jedoch Qualität und Robustheit der Übertragung effizient kontrollierbar, wenn Streamingprotokolle Informationen über den Inhalt der transportierten Daten ausnutzen. Existierende Ansätze, die den Inhalt von Multimediadatenströmen beschreiben, sind allerdings meist auf einzelne Kompressionsverfahren spezialisiert und verwenden berechnungsintensive Metriken. Das reduziert ihren praktischen Nutzen deutlich. Außerdem erfordert der Informationsaustausch eine enge Kooperation zwischen Applikationen und Transportschichten. Da allerdings die Schnittstellen aktueller Systemarchitekturen nicht darauf vorbereitet sind, müssen entweder die Schnittstellen erweitert oder alternative Architekturkonzepte geschaffen werden. Die Gefahr beider Varianten ist jedoch, dass sich die Komplexität eines Systems dadurch weiter erhöhen kann. Das zentrale Ziel dieser Dissertation ist es deshalb, schichtenübergreifende Koordination bei gleichzeitiger Reduzierung der Komplexität zu erreichen. Hier leistet die Arbeit zwei Beträge zum aktuellen Stand der Forschung. Erstens definiert sie ein universelles Modell zur Beschreibung von Inhaltsattributen, wie Wichtigkeiten und Abhängigkeitsbeziehungen innerhalb eines Datenstroms. Transportschichten können dieses Wissen zur effizienten Fehlerkontrolle verwenden. Zweitens beschreibt die Arbeit das Noja Programmiermodell für multimediale Middleware. Noja definiert Abstraktionen zur Übertragung und Kontrolle multimedialer Ströme, die die Koordination von Streamingprotokollen mit Applikationen ermöglichen. Zum Beispiel können Programmierer geeignete Fehlersemantiken und Kommunikationstopologien auswählen und den konkreten Fehlerschutz dann zur Laufzeit verfeinern und kontrolliere

3D multiple description coding for error resilience over wireless networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Mobile communications has gained a growing interest from both customers and service providers alike in the last 1-2 decades. Visual information is used in many application domains such as remote health care, video –on demand, broadcasting, video surveillance etc. In order to enhance the visual effects of digital video content, the depth perception needs to be provided with the actual visual content. 3D video has earned a significant interest from the research community in recent years, due to the tremendous impact it leaves on viewers and its enhancement of the user’s quality of experience (QoE). In the near future, 3D video is likely to be used in most video applications, as it offers a greater sense of immersion and perceptual experience. When 3D video is compressed and transmitted over error prone channels, the associated packet loss leads to visual quality degradation. When a picture is lost or corrupted so severely that the concealment result is not acceptable, the receiver typically pauses video playback and waits for the next INTRA picture to resume decoding. Error propagation caused by employing predictive coding may degrade the video quality severely. There are several ways used to mitigate the effects of such transmission errors. One widely used technique in International Video Coding Standards is error resilience. The motivation behind this research work is that, existing schemes for 2D colour video compression such as MPEG, JPEG and H.263 cannot be applied to 3D video content. 3D video signals contain depth as well as colour information and are bandwidth demanding, as they require the transmission of multiple high-bandwidth 3D video streams. On the other hand, the capacity of wireless channels is limited and wireless links are prone to various types of errors caused by noise, interference, fading, handoff, error burst and network congestion. Given the maximum bit rate budget to represent the 3D scene, optimal bit-rate allocation between texture and depth information rendering distortion/losses should be minimised. To mitigate the effect of these errors on the perceptual 3D video quality, error resilience video coding needs to be investigated further to offer better quality of experience (QoE) to end users. This research work aims at enhancing the error resilience capability of compressed 3D video, when transmitted over mobile channels, using Multiple Description Coding (MDC) in order to improve better user’s quality of experience (QoE). Furthermore, this thesis examines the sensitivity of the human visual system (HVS) when employed to view 3D video scenes. The approach used in this study is to use subjective testing in order to rate people’s perception of 3D video under error free and error prone conditions through the use of a carefully designed bespoke questionnaire.Petroleum Technology Development Fund (PTDF

Design of interface selection protocols for multi-homed wireless networks

The IEEE 802.11/802.16 standards conformant wireless communication stations have multi-homing transmission capability. To achieve greater communication efficiency, multi-homing capable stations use handover mechanism to select appropriate transmission channel according to variations in the channel quality. This thesis presents three internal-linked handover schemes, (1) Interface Selection Protocol (ISP), belonging to Wireless Local Area Network (WLAN)- Worldwide Interoperability for Microwave Access (WiMAX) environment (2) Fast Channel Scanning (FCS) and (3) Traffic Manager (TM), (2) and (3) belonging to WiMAX Environment. The proposed schemes in this thesis use a novel mechanism of providing a reliable communication route. This solution is based on a cross-layer communication framework, where the interface selection module uses various network related parameters from Medium Access Control (MAC) sub-layer/Physical Layer (PHY) across the protocol suite for decision making at the Network layer. The proposed solutions are highly responsive when compared with existing multi-homed schemes; responsiveness is one of the key factors in the design of such protocols. Selected route under these schemes is based on the most up to date link-layer information. Therefore, such a route is not only reliable in terms of route optimization but it also fulfils the application demands in terms of throughput and delay. Design of ISP protocol use probing frames during the route discovery process. The 802.11 mandates the use of different rates for data transmission frames. The ISP-metric can be incorporated into various routing aspects and its applicability is determined by the possibility of provision of MAC dependent parameters that are used to determine the best path metric values. In many cases, higher device density, interference and mobility cause variable medium access delays. It causes creation of ‘unreachable zones’, where destination is marked as unreachable. However, by use of the best path metric, the destination has been made reachable, anytime and anywhere, because of the intelligent use of the probing frames and interface selection algorithm implemented. The IEEE 802.16e introduces several MAC level queues for different access categories, maintaining service requirement within these queues; which imply that frames from a higher priority queue, i.e. video frames, are serviced more frequently than those belonging to lower priority queues. Such an enhancement at the MAC sub-layer introduces uneven queuing delays. Conventional routing protocols are unaware of such MAC specific constraints and as a result, these factors are not considered which result in channel performance degradation. To meet such challenges, the thesis presents FCS and TM schemes for WiMAX. For FCS, Its solution is to improve the mobile WiMAX handover and address the scanning latency. Since minimum scanning time is the most important issue in the handover process. This handover scheme aims to utilize the channel efficiently and apply such a procedure to reduce the time it takes to scan the neighboring access stations. TM uses MAC and physical layer (PHY) specific information in the interface metric and maintains a separate path to destination by applying an alternative interface operation. Simulation tests and comparisons with existing multi-homed protocols and handover schemes demonstrate the effectiveness of incorporating the medium dependent parameters. Moreover, show that suggested schemes, have shown better performance in terms of end-to-end delay and throughput, with efficiency up to 40% in specific test scenarios.EThOS - Electronic Theses Online ServiceGBUnited Kingdo