End-to-end 3D video communication over heterogeneous networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Three-dimensional technology, more commonly referred to as 3D technology, has revolutionised many fields including entertainment, medicine, and communications to name a few. In addition to 3D films, games, and sports channels, 3D perception has made tele-medicine a reality. By the year 2015, 30% of the all HD panels at home will be 3D enabled, predicted by consumer electronics manufacturers. Stereoscopic cameras, a comparatively mature technology compared to other 3D systems, are now being used by ordinary citizens to produce 3D content and share at a click of a button just like they do with the 2D counterparts via sites like YouTube. But technical challenges still exist, including with autostereoscopic multiview displays. 3D content requires many complex considerations--including how to represent it, and deciphering what is the best compression format--when considering transmission or storage, because of its increased amount of data. Any decision must be taken in the light of the available bandwidth or storage capacity, quality and user expectations. Free viewpoint navigation also remains partly unsolved. The most pressing issue getting in the way of widespread uptake of consumer 3D systems is the ability to deliver 3D content to heterogeneous consumer displays over the heterogeneous networks. Optimising 3D video communication solutions must consider the entire pipeline, starting with optimisation at the video source to the end display and transmission optimisation. Multi-view offers the most compelling solution for 3D videos with motion parallax and freedom from wearing headgear for 3D video perception. Optimising multi-view video for delivery and display could increase the demand for true 3D in the consumer market. This thesis focuses on an end-to-end quality optimisation in 3D video communication/transmission, offering solutions for optimisation at the compression, transmission, and decoder levels.Brunel University - Isambard Research Scholarshi

3D multiple description coding for error resilience over wireless networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Mobile communications has gained a growing interest from both customers and service providers alike in the last 1-2 decades. Visual information is used in many application domains such as remote health care, video –on demand, broadcasting, video surveillance etc. In order to enhance the visual effects of digital video content, the depth perception needs to be provided with the actual visual content. 3D video has earned a significant interest from the research community in recent years, due to the tremendous impact it leaves on viewers and its enhancement of the user’s quality of experience (QoE). In the near future, 3D video is likely to be used in most video applications, as it offers a greater sense of immersion and perceptual experience. When 3D video is compressed and transmitted over error prone channels, the associated packet loss leads to visual quality degradation. When a picture is lost or corrupted so severely that the concealment result is not acceptable, the receiver typically pauses video playback and waits for the next INTRA picture to resume decoding. Error propagation caused by employing predictive coding may degrade the video quality severely. There are several ways used to mitigate the effects of such transmission errors. One widely used technique in International Video Coding Standards is error resilience. The motivation behind this research work is that, existing schemes for 2D colour video compression such as MPEG, JPEG and H.263 cannot be applied to 3D video content. 3D video signals contain depth as well as colour information and are bandwidth demanding, as they require the transmission of multiple high-bandwidth 3D video streams. On the other hand, the capacity of wireless channels is limited and wireless links are prone to various types of errors caused by noise, interference, fading, handoff, error burst and network congestion. Given the maximum bit rate budget to represent the 3D scene, optimal bit-rate allocation between texture and depth information rendering distortion/losses should be minimised. To mitigate the effect of these errors on the perceptual 3D video quality, error resilience video coding needs to be investigated further to offer better quality of experience (QoE) to end users. This research work aims at enhancing the error resilience capability of compressed 3D video, when transmitted over mobile channels, using Multiple Description Coding (MDC) in order to improve better user’s quality of experience (QoE). Furthermore, this thesis examines the sensitivity of the human visual system (HVS) when employed to view 3D video scenes. The approach used in this study is to use subjective testing in order to rate people’s perception of 3D video under error free and error prone conditions through the use of a carefully designed bespoke questionnaire.Petroleum Technology Development Fund (PTDF

Adaptive subframe allocation for next generation multimedia delivery over hybrid LTE unicast broadcast

The continued global roll-out of long term evolution (LTE) networks is providing mobile users with perpetually increasing ubiquitous access to a rich selection of high quality multimedia. Interactive viewing experiences including 3-D or free-viewpoint video require the synchronous delivery of multiple video streams. This paper presents a novel hybrid unicast broadcast synchronisation (HUBS) framework to synchronously deliver multi-stream content. Previous techniques on hybrid LTE implementations include staggered modulation and coding scheme grouping, adaptive modulation coding or implementing error recover techniques; the work presented here instead focuses on dynamic allocation of resources between unicast and broadcast, improving stream synchronisation as well as overall cell resource usage. Furthermore, the HUBS framework has been developed to work within the limitations imposed by the LTE specification. Performance evaluation of the framework is performed through the simulation of probable future scenarios, where a popular live event is broadcast with stereo 3-D or multi-angle companion views interactively offered to capable users. The proposed framework forms a ``HUBS group'' that monitors the radio bearer queues to establish a time lead or lag between broadcast and unicast streams. Since unicast and broadcast share the same radio resources, the number of subframes allocated to the broadcast transmission are then dynamically increased or decreased to minimise the average lead/lag time offset between the streams. Dynamic allocation showed improvements for all services across the cell, whilst keeping streams synchronised despite increased user loading

Three-dimensional media for mobile devices

Cataloged from PDF version of article.This paper aims at providing an overview of the core technologies enabling the delivery of 3-D Media to next-generation mobile devices. To succeed in the design of the corresponding system, a profound knowledge about the human visual system and the visual cues that form the perception of depth, combined with understanding of the user requirements for designing user experience for mobile 3-D media, are required. These aspects are addressed first and related with the critical parts of the generic system within a novel user-centered research framework. Next-generation mobile devices are characterized through their portable 3-D displays, as those are considered critical for enabling a genuine 3-D experience on mobiles. Quality of 3-D content is emphasized as the most important factor for the adoption of the new technology. Quality is characterized through the most typical, 3-D-specific visual artifacts on portable 3-D displays and through subjective tests addressing the acceptance and satisfaction of different 3-D video representation, coding, and transmission methods. An emphasis is put on 3-D video broadcast over digital video broadcasting-handheld (DVB-H) in order to illustrate the importance of the joint source-channel optimization of 3-D video for its efficient compression and robust transmission over error-prone channels. The comparative results obtained identify the best coding and transmission approaches and enlighten the interaction between video quality and depth perception along with the influence of the context of media use. Finally, the paper speculates on the role and place of 3-D multimedia mobile devices in the future internet continuum involving the users in cocreation and refining of rich 3-D media content

3D multiple description coding for error resilience over wireless networks

Mobile communications has gained a growing interest from both customers and service providers alike in the last 1-2 decades. Visual information is used in many application domains such as remote health care, video –on demand, broadcasting, video surveillance etc. In order to enhance the visual effects of digital video content, the depth perception needs to be provided with the actual visual content. 3D video has earned a significant interest from the research community in recent years, due to the tremendous impact it leaves on viewers and its enhancement of the user’s quality of experience (QoE). In the near future, 3D video is likely to be used in most video applications, as it offers a greater sense of immersion and perceptual experience. When 3D video is compressed and transmitted over error prone channels, the associated packet loss leads to visual quality degradation. When a picture is lost or corrupted so severely that the concealment result is not acceptable, the receiver typically pauses video playback and waits for the next INTRA picture to resume decoding. Error propagation caused by employing predictive coding may degrade the video quality severely. There are several ways used to mitigate the effects of such transmission errors. One widely used technique in International Video Coding Standards is error resilience. The motivation behind this research work is that, existing schemes for 2D colour video compression such as MPEG, JPEG and H.263 cannot be applied to 3D video content. 3D video signals contain depth as well as colour information and are bandwidth demanding, as they require the transmission of multiple high-bandwidth 3D video streams. On the other hand, the capacity of wireless channels is limited and wireless links are prone to various types of errors caused by noise, interference, fading, handoff, error burst and network congestion. Given the maximum bit rate budget to represent the 3D scene, optimal bit-rate allocation between texture and depth information rendering distortion/losses should be minimised. To mitigate the effect of these errors on the perceptual 3D video quality, error resilience video coding needs to be investigated further to offer better quality of experience (QoE) to end users. This research work aims at enhancing the error resilience capability of compressed 3D video, when transmitted over mobile channels, using Multiple Description Coding (MDC) in order to improve better user’s quality of experience (QoE). Furthermore, this thesis examines the sensitivity of the human visual system (HVS) when employed to view 3D video scenes. The approach used in this study is to use subjective testing in order to rate people’s perception of 3D video under error free and error prone conditions through the use of a carefully designed bespoke questionnaire.EThOS - Electronic Theses Online ServicePetroleum Technology Development Fund (PTDF)GBUnited Kingdo

Feasibility study: Electronic media laboratory according to Industry standard

The goal of this Master's thesis was to find out the Metropolia media technology laboratory’s correlation to industrial electronic media production standards and environments. The laboratory environment for education and training should be following not only media technology curriculum, but also the requirements set for learning audio-visual technology, including the processes, technologies and practices involved in professional electronic production and publishing in Finland. Since the role of the laboratory is to perform as an effective learning environment, there are some understandable restrictions in technology and development, in financial and in management section that prevent it from reaching the industrial level. These limitations and differences between this environment and more sophisticated industrial model were studied in this thesis. Feasibility study, which is suitable for analyzing the technology, financial and administrative development all together, was selected as the research method. The study was intended to determine the achievable laboratory model of functional validation and also to demonstrate the proof of concept. Audiovisual production in the laboratory and the subsequent electronic digital publishing was created as an embedded model. The model takes into account the ongoing changes in the television environment and the traditional radio frequency transmission, which is now competing with other alternatives such as IP network distribution. Video editing as post production was analyzed in its current state and compared to a complete network-based-editing NAS and SAN model, which can also be implemented in cloud technologies for decentralized co-operation. This way the model was following the industry trends and future prospects. Research material was collected and acquired from written sources, industry fairs and conferences, resellers and agents as well as production companies and broadcasters. Practical implementation of electronic media publishing was simulated in the laboratory environment so that all the possible reception technologies were covered. Publishing was customized for compatibility with various devices. Noteworthy aspects in this publishing process were the post-processing, packetizing and distribution related quality management aspects their definition and management issues were also discussed in this thesis. The end result was a feasibility analysis and model, which defines the electronic publishing technology teaching bottlenecks regarding the equivalence of the technologies in laboratory environment.YAMK-insinöörityön tavoitteena oli selvittää Metropolian mediatekniikan laboratorion vastaavuus teollisessa elektronisen median tuotannossa noudatettaviin standardeihin. Laboratorion koulutusympäristönä tulee noudattaa paitsi mediatekniikan opetussuunnitelmassa audio-visuaaliselle tekniikalle oppimiselle asetettuja vaatimuksia, myös niitä prosesseja, käytäntöjä ja teknologioita joita ammattimainen elektroninen tuotanto ja julkaiseminen Suomessa toteuttaa. Oppimisympäristönä toimivan laboratorion kehittämiselle ja varustamiselle on ymmärrettävästi teknisiä, taloudellisia ja hallinnallisia rajoituksia, jotka estävät sen saavuttamasta teollista tasoa. Näiden rajojen ja kehittyneemmän teollisen mallin eroja tutkittiin. Tutkimusmenetelmäksi valittiin saavutettavuustutkimus (eng. feasebility study), joka menetelmänä soveltuu teknologian, taloudellisen tai hallinnollisen kehittämiseen analysointiin. Tutkimuksella pyrittiin määrittelemään saavutettavissa olevan laboratoriomallin toiminnallinen validointi ja toteutettavuuden osoitus (engl.proof of concept). Laboratorion audiovisuaalista tuotantoa ja sitä seuraavaa elektronista sekä digitaalista julkaisemista varten luotiin sulautettu malli. Mallissa huomioitiin televisioympäristössä meneillään oleva muutos, jossa perinteinen radiotaajuuslähettäminen on saanut vaihtoehdoksi IP- verkoilla suoritettavan jakelun. Videon editointi jälkituotantona analysointiin nykytilassa ja vertailtiin sitä täydellisen verkkoeditoinnin malleihin, jotka voivat myös toteuttaa pilviteknologioita hajautetussa tuotannollisessa yhteistyössä. Näin medialaboratorion mallinnuksessa huomioitiin toimialan ajan trendit ja tulevaisuuden näkymät. Tämä tutkimusaineisto hankittiin kirjallisten lähteiden lisäksi, alan messuilta ja konferensseista, maahantuojilta ja edustajilta sekä tuotanto- ja yleisradioyhtiöiltä. Elektronisen median julkaisemista simuloitiin laboratorioympäristössä niin, että kaikki mahdolliset vastaanottotekniikat oli otettu huomioon. Julkaiseminen räätälöitiin myös soveltuvaksi eri päätelaitteille. Julkaisemisessa huomionarvoisiksi asioiksi nousivat audiovisuaalisen sisällön jälkiprosessointiin, paketointiin ja lähettämiseen liittyvät tekniset laadunhallinnan aspektit, esimerkiksi eri videonpakkausalgoritmien soveltuvuus videosisällön jakelussa, julkaisemisen automatisoinnin mahdollistaminen ja eri julkaisualustojen tutkiminen ja määrittely Lopputuloksena saatiin aikaan soveltuvuusanalyysi ja mallinnus, jossa määriteltiin elektronisen julkaisemisen teknisen opetuksen pullonkaulat teolliseen vastaavuuteen nähden. Suurimmaksi kompastuskiveksi voidaan mainita jakelujärjestelmän operoinnin hankaluus sekä verkotetun editointijärjestelmän määrittelyn haasteellisuus

Multimedia delivery in the future internet

The term “Networked Media” implies that all kinds of media including text, image, 3D graphics, audio and video are produced, distributed, shared, managed and consumed on-line through various networks, like the Internet, Fiber, WiFi, WiMAX, GPRS, 3G and so on, in a convergent manner [1]. This white paper is the contribution of the Media Delivery Platform (MDP) cluster and aims to cover the Networked challenges of the Networked Media in the transition to the Future of the Internet. Internet has evolved and changed the way we work and live. End users of the Internet have been confronted with a bewildering range of media, services and applications and of technological innovations concerning media formats, wireless networks, terminal types and capabilities. And there is little evidence that the pace of this innovation is slowing. Today, over one billion of users access the Internet on regular basis, more than 100 million users have downloaded at least one (multi)media file and over 47 millions of them do so regularly, searching in more than 160 Exabytes1 of content. In the near future these numbers are expected to exponentially rise. It is expected that the Internet content will be increased by at least a factor of 6, rising to more than 990 Exabytes before 2012, fuelled mainly by the users themselves. Moreover, it is envisaged that in a near- to mid-term future, the Internet will provide the means to share and distribute (new) multimedia content and services with superior quality and striking flexibility, in a trusted and personalized way, improving citizens’ quality of life, working conditions, edutainment and safety. In this evolving environment, new transport protocols, new multimedia encoding schemes, cross-layer inthe network adaptation, machine-to-machine communication (including RFIDs), rich 3D content as well as community networks and the use of peer-to-peer (P2P) overlays are expected to generate new models of interaction and cooperation, and be able to support enhanced perceived quality-of-experience (PQoE) and innovative applications “on the move”, like virtual collaboration environments, personalised services/ media, virtual sport groups, on-line gaming, edutainment. In this context, the interaction with content combined with interactive/multimedia search capabilities across distributed repositories, opportunistic P2P networks and the dynamic adaptation to the characteristics of diverse mobile terminals are expected to contribute towards such a vision. Based on work that has taken place in a number of EC co-funded projects, in Framework Program 6 (FP6) and Framework Program 7 (FP7), a group of experts and technology visionaries have voluntarily contributed in this white paper aiming to describe the status, the state-of-the art, the challenges and the way ahead in the area of Content Aware media delivery platforms

Study of encapsulation and transport of 3DTV by satellite

The project was developed in EADS ASTRIUM Toulouse in the framework of the MUSCADE project with the latest technologies in 3DTV. Currently most of the research in satellite broadcasting field is focused in 3DTV transmission as the following of HDTV. MUSCADE is a European project funded by the 7th Framework Program whose objective is to demonstrate a complete multiview 3DTV live chain over wireline, wireless and satellite networks. This project aims to set up a satellite testbed to validate the 3D content format defined by MUSCADE in an emulated satellite environment. The document’s first chapter describes the environment where the internship has taken place and a brief overview of the EADS Company. After, a short description of the whole MUSCADE project can be found in section 5. This allows the reader to achieve a global vision of all the technological concepts involved in the project even if this internship is focused in satellite transmission. Section 6 describes the internship development. By means of conclusion, the new skills achieved, the knowledge applied and a professional and personal balance could be found at the end of this report.Ingeniería de TelecomunicaciónTelekomunikazio Ingeniaritz