Understanding Timelines within MPEG Standards

(c) 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Nowadays, media content can be delivered via diverse broadband and broadcast technologies. Although these different technologies have somehow become rivals, their coordinated usage and convergence, by leveraging of their strengths and complementary characteristics, can bring many benefits to both operators and customers. For example, broadcast TV content can be augmented by on-demand broadband media content to provide enriched and personalized services, such as multi-view TV, audio language selection, and inclusion of real-time web feeds. A piece of evidence is the recent Hybrid Broadcast Broadband TV (HbbTV) standard, which aims at harmonizing the delivery and consumption of (hybrid) broadcast and broadband TV content. A key challenge in these emerging scenarios is the synchronization between the involved media streams, which can be originated by the same or different sources, and delivered via the same or different technologies. To enable synchronized (hybrid) media delivery services, some mechanisms providing timelines at the source side are necessary to accurately time align the involved media streams at the receiver-side. This paper provides a comprehensive review of how clock references (timing) and timestamps (time) are conveyed and interpreted when using the most widespread delivery technologies, such as DVB, RTP/RTCP and MPEG standards (e.g., MPEG-2, MPEG-4, MPEG-DASH, and MMT). It is particularly focused on the format, resolution, frequency, and the position within the bitstream of the fields conveying timing information, as well as on the involved components and packetization aspects. Finally, it provides a survey of proofs of concepts making use of these synchronization related mechanisms. This complete and thorough source of information can be very useful for scholars and practitioners interested in media services with synchronization demands.This work has been funded, partially, by the "Fondo Europeo de Desarrollo Regional" (FEDER) and the Spanish Ministry of Economy and Competitiveness, under its R&D&i Support Program in project with ref TEC2013-45492-R.Yuste, LB.; Boronat Segui, F.; Montagut Climent, MA.; Melvin, H. (2015). Understanding Timelines within MPEG Standards. Communications Surveys and Tutorials, IEEE Communications Society. 18(1):368-400. https://doi.org/10.1109/COMST.2015.2488483S36840018

HbbTV-compliant Platform for Hybrid Media Delivery and Synchronization on Single- and Multi-Device Scenarios

[EN] The combination of broadcast and broadband (hybrid) technologies for delivering TV related media contents can bring fascinating opportunities. It is motivated by the large amount and diversity of media contents, together with the ubiquity and multiple connectivity capabilities of modern consumption devices. This paper presents an end-to-end platform for the preparation, delivery, and synchronized consumption of related hybrid (broadcast/broadband) media contents on a single device and/or on multiple close-by devices (i.e., a multi-device scenario). It is compatible with the latest version of the Hybrid Broadcast Broadband TV (HbbTV) standard (version 2.0.1). Additionally, it provides adaptive and efficient solutions for key issues not specified in that standard, but that are necessary to successfully deploy hybrid and multidevice media services. Moreover, apart from MPEG-DASH and HTML5, which are the broadband technologies adopted by HbbTV, the platform also provides support for using HTTP Live Streaming and Real-time Transport Protocol and its companion RTP Control Protocol broadband technologies. The presented platform can provide support for many hybrid media services. In this paper, in order to evaluate it, the use case of multi-device and multi-view TV service has been selected. The results of both objective and subjective assessments have been very satisfactory, in terms of performance (stability, smooth playout, delays, and sync accuracy), usability of the platform, usefulness of its functionalities, and the awaken interest in these kinds of platforms.This work was supported in part by the "Fondo Europeo de Desarrollo Regional" and in part by the Spanish Ministry of Economy and Competitiveness through R&D&I Support Program under Grant TEC2013-45492-R.Boronat, F.; Marfil-Reguero, D.; Montagud, M.; Pastor Castillo, FJ. (2017). HbbTV-compliant Platform for Hybrid Media Delivery and Synchronization on Single- and Multi-Device Scenarios. IEEE Transactions on Broadcasting. 1-26. https://doi.org/10.1109/TBC.2017.2781124S12

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Quality-driven resource utilization methods for video streaming in wireless communication networks

This research is focused on the optimisation of resource utilisation in wireless mobile networks with the consideration of the users’ experienced quality of video streaming services. The study specifically considers the new generation of mobile communication networks, i.e. 4G-LTE, as the main research context. The background study provides an overview of the main properties of the relevant technologies investigated. These include video streaming protocols and networks, video service quality assessment methods, the infrastructure and related functionalities of LTE, and resource allocation algorithms in mobile communication systems. A mathematical model based on an objective and no-reference quality assessment metric for video streaming, namely Pause Intensity, is developed in this work for the evaluation of the continuity of streaming services. The analytical model is verified by extensive simulation and subjective testing on the joint impairment effects of the pause duration and pause frequency. Various types of the video contents and different levels of the impairments have been used in the process of validation tests. It has been shown that Pause Intensity is closely correlated with the subjective quality measurement in terms of the Mean Opinion Score and this correlation property is content independent. Based on the Pause Intensity metric, an optimised resource allocation approach is proposed for the given user requirements, communication system specifications and network performances. This approach concerns both system efficiency and fairness when establishing appropriate resource allocation algorithms, together with the consideration of the correlation between the required and allocated data rates per user. Pause Intensity plays a key role here, representing the required level of Quality of Experience (QoE) to ensure the best balance between system efficiency and fairness. The 3GPP Long Term Evolution (LTE) system is used as the main application environment where the proposed research framework is examined and the results are compared with existing scheduling methods on the achievable fairness, efficiency and correlation. Adaptive video streaming technologies are also investigated and combined with our initiatives on determining the distribution of QoE performance across the network. The resulting scheduling process is controlled through the prioritization of users by considering their perceived quality for the services received. Meanwhile, a trade-off between fairness and efficiency is maintained through an online adjustment of the scheduler’s parameters. Furthermore, Pause Intensity is applied to act as a regulator to realise the rate adaptation function during the end user’s playback of the adaptive streaming service. The adaptive rates under various channel conditions and the shape of the QoE distribution amongst the users for different scheduling policies have been demonstrated in the context of LTE. Finally, the work for interworking between mobile communication system at the macro-cell level and the different deployments of WiFi technologies throughout the macro-cell is presented. A QoEdriven approach is proposed to analyse the offloading mechanism of the user’s data (e.g. video traffic) while the new rate distribution algorithm reshapes the network capacity across the macrocell. The scheduling policy derived is used to regulate the performance of the resource allocation across the fair-efficient spectrum. The associated offloading mechanism can properly control the number of the users within the coverages of the macro-cell base station and each of the WiFi access points involved. The performance of the non-seamless and user-controlled mobile traffic offloading (through the mobile WiFi devices) has been evaluated and compared with that of the standard operator-controlled WiFi hotspots

Omnidirectional view and multi-modal streaming in 3D tele-immersion system

3D Tele-immersion (3DTI) technology allows full-body, multi-modal content delivery among geographically dispersed users. In 3DTI, user’s 3D model will be captured by multiple RGB-D (color plus depth) cameras surround- ing user’s body. In addition, various sensors (e.g., motion sensors, medical sensors, wearable gaming consoles, etc.) specified by the application will be included to deliver a multi-modal experience. In a traditional 2D live video streaming system, the interactivity of end users, choosing a specified viewpoint, has been crippled by the fact that they can only choose to see the physical scene captured by a physical camera, but not between two physical cameras. However, 3DTI system makes it possible rendering a 3D space where the viewers can view physical scene from arbitrary viewpoint. In this thesis, we present systematic solutions of omnidirectional view in 3D tele-immersion system in a real-time manner and in an on-demand streaming manner, called FreeViewer and OmniViewer, respectively. we provide a complete multi-modal 3D video streaming/rendering solution, which achieves the feature of omnidirectional view in monoscopic 3D systems

Analyse intelligente de la qualité d'expérience (QoE) dans les réseaux de diffusion de contenu web et mutimédia

Today user experience is becoming a reliable indicator for service providers and telecommunication operators to convey overall end to end system functioning. Moreover, to compete for a prominent market share, different network operators and service providers should retain and increase the customers’ subscription. To fulfil these requirements they require an efficient Quality of Experience (QoE) monitoring and estimation. However, QoE is a subjective metric and its evaluation is expensive and time consuming since it requires human participation. Therefore, there is a need for an objective tool that can measure the QoE objectively with reasonable accuracy in real-Time. As a first contribution, we analyzed the impact of network conditions on Video on Demand (VoD) services. We also proposed an objective QoE estimation tool that uses fuzzy expert system to estimate QoE from network layer QoS parameters. As a second contribution, we analyzed the impact of MAC layer QoS parameters on VoD services over IEEE 802.11n wireless networks. We also proposed an objective QoE estimation tool that uses random neural network to estimate QoE from the MAC layer perspective. As our third contribution, we analyzed the effect of different adaption scenarios on QoE of adaptive bit rate streaming. We also developed a web based subjective test platform that can be easily integrated in a crowdsourcing platform for performing subjective tests. As our fourth contribution, we analyzed the impact of different web QoS parameters on web service QoE. We also proposed a novel machine learning algorithm i.e. fuzzy rough hybrid expert system for estimating web service QoE objectivelyDe nos jours, l’expérience de l'utilisateur appelé en anglais « User Experience » est devenue l’un des indicateurs les plus pertinents pour les fournisseurs de services ainsi que pour les opérateurs de télécommunication pour analyser le fonctionnement de bout en bout de leurs systèmes (du terminal client, en passant par le réseaux jusqu’à l’infrastructure des services etc.). De plus, afin d’entretenir leur part de marché et rester compétitif, les différents opérateurs de télécommunication et les fournisseurs de services doivent constamment conserver et accroître le nombre de souscription des clients. Pour répondre à ces exigences, ils doivent disposer de solutions efficaces de monitoring et d’estimation de la qualité d'expérience (QoE) afin d’évaluer la satisfaction de leur clients. Cependant, la QoE est une mesure qui reste subjective et son évaluation est coûteuse et fastidieuse car elle nécessite une forte participation humaine (appelé panel de d’évaluation). Par conséquent, la conception d’un outil qui peut mesurer objectivement cette qualité d'expérience avec une précision raisonnable et en temps réel est devenue un besoin primordial qui constitue un challenge intéressant à résoudre. Comme une première contribution, nous avons analysé l'impact du comportement d’un réseau sur la qualité des services de vidéo à la demande (VOD). Nous avons également proposé un outil d'estimation objective de la QoE qui utilise le système expert basé sur la logique floue pour évaluer la QoE à partir des paramètres de qualité de service de la couche réseau. Dans une deuxième contribution, nous avons analysé l'impact des paramètres QoS de couche MAC sur les services de VoD dans le cadre des réseaux sans fil IEEE 802.11n. Nous avons également proposé un outil d'estimation objective de la QoE qui utilise le réseau aléatoire de neurones pour estimer la QoE dans la perspective de la couche MAC. Pour notre troisième contribution, nous avons analysé l'effet de différents scénarios d'adaptation sur la QoE dans le cadre du streaming adaptatif au débit. Nous avons également développé une plate-Forme Web de test subjectif qui peut être facilement intégré dans une plate-Forme de crowd-Sourcing pour effectuer des tests subjectifs. Finalement, pour notre quatrième contribution, nous avons analysé l'impact des différents paramètres de qualité de service Web sur leur QoE. Nous avons également proposé un algorithme d'apprentissage automatique i.e. un système expert hybride rugueux basé sur la logique floue pour estimer objectivement la QoE des Web service

Distribuição de vídeo para grupos de utilizadores em redes móveis heterogéneas19

The evolutions veri ed in mobile devices capabilities (storage capacity, screen resolution, processor, etc.) over the last years led to a signi cant change in mobile user behavior, with the consumption and creation of multimedia content becoming more common, in particular video tra c. Consequently, mobile operator networks, despite being the target of architectural evolutions and improvements over several parameters (such as capacity, transmission and reception performance, amongst others), also increasingly become more frequently challenged by performance aspects associated to the nature of video tra c, whether by the demanding requirements associated to that service, or by its volume increase in such networks. This Thesis proposes modi cations to the mobile architecture towards a more e cient video broadcasting, de ning and developing mechanisms applicable to the network, or to the mobile terminal. Particularly, heterogeneous networks multicast IP mobility supported scenarios are focused, emphasizing their application over di erent access technologies. The suggested changes are applicable to mobile or static user scenarios, whether it performs the role of receiver or source of the video tra c. Similarly, the de ned mechanisms propose solutions targeting operators with di erent video broadcasting goals, or whose networks have di erent characteristics. The pursued methodology combined an experimental evaluation executed over physical testbeds, with the mathematical evaluation using network simulation, allowing the veri cation of its impact on the optimization of video reception in mobile terminalsA evolução veri cada nas características dos dispositivos moveis (capacidade de armazenamento, resolução do ecrã, processador, etc.) durante os últimos anos levou a uma alteração signi cativa nos comportamentos dos utilizadores, sendo agora comum o consumo e produção de conteúdos multimédia envolvendo terminais móveis, em particular o tráfego vídeo. Consequentemente, as redes de operador móvel, embora tendo também sido alvo constante de evoluções arquitecturais e melhorias em vários parâmetros (tais como capacidade, ritmo de transmissão/recepção, entre outros), vêemse cada vez mais frequentemente desa adas por aspectos de desempenho associados à natureza do tráfego de vídeo, seja pela exigência de requisitos associados a esse serviço, quer pelo aumento do volume do mesmo nesse tipo de redes. Esta Tese propôe alterações à arquitetura móvel para a disseminação de vídeo mais e ciente, de nindo e desenvolvendo mecanismos aplicáveis à rede, ou ao utilizador móvel. Em particular, são focados cenários suportados por IP multicast em redes móveis heterogéneas, isto é, com ênfase na aplicação destes mecanismos sobre diferentes tecnologias de acesso. As alterações sugeridas aplicam-se a cenários de utilizador estático ou móvel, sendo este a fonte ou receptor do tráfego vídeo. Da mesma forma, são propostas soluções tendo em vista operadores com diferentes objectivos de disseminação de vídeo, ou cujas redes têm diferentes características. A metodologia utilizada combinou a avaliação experimental em testbeds físicas com a avaliação matemática em simulações de redes, e permitiu veri car o impacto sobre a optimização da recepção de vídeo em terminais móveisPrograma Doutoral em Telecomunicaçõe

Architecture à qualité de service pour systèmes satellites DVB-S/RCS dans un contexte NGN

L'objet de cette thèse est de doter les réseaux satellites d'accès géostationnaires d'une architecture de QoS compatible avec l'infrastructure NGN (Next Generation Network) qui entend réaliser la convergence des réseaux et services de communication existants et à venir. Dans une première étape, nous établissons un modèle de QoS NGN s'inspirant principalement de l'évolution des architectures de QoS dans les réseaux IP. Dans une deuxième étape, nous démontrons que les réseaux satellites ne disposent pas, en l'état actuel, d'une architecture de QoS suffisamment mature pour assurer un accès large bande aux futurs services IP multimédias. Afin d'y remédier, nous spécifions une architecture de QoS NGN unifiée assurant une étroite collaboration entre les mécanismes de QoS déployés à différents niveaux de communication (Application, Session, Réseau et MAC). Elle assure à la fois une différenciation de la QoS adaptée aux besoins des différentes classes d'applications au niveau IP tout en optimisant l'utilisation des ressources satellites via des mécanismes de bande passante à la demande au niveau MAC. Deux solutions applicatives assurant la corrélation dynamique entre les applications et les services réseaux différenciés sont également spécifiées et implémentées. Enfin un émulateur satellite de niveau réseau a été implémenté et nous a permis d'évaluer les garanties de QoS offertes par notre architecture et de valider leur conformité avec les besoins d'applications multimédias réelles. ABSTRACT : This thesis deals with the integration of geostationary access satellite networks in the NGN (Next Generation Network) infrastructure, a global QoS-enabled communication network based on IP. The first part depicts this multiservice ubiquitous infrastructure anchored on a clean separation between a transport/network domain and a service/application domain. Then a unified NGN control and management architecture, mainly inspired by the past and current IP QoS architectures, is proposed. In a second part, starting from this model and through a critical analysis of current DVB-S/RCS satellite systems, we demonstrate that satellite QoS architectures are not mature enough to support broadband access to future multimedia services and to integrate into the emerging NGN infrastructure. In order to cope with these issues, a NGNcompliant QoS architecture for DVB-S/RCS systems is defined. It relies on a cross-layer approach based on a tight interaction between QoS mechanisms available at different communication layers (Application, Session, Network and MAC). It aims at maximizing satellite capacity utilization at different granularity levels through a relevant configuration of DiffServ services using session/application signalling information and the use of optimized bandwidth on demand allocation schemes at the MAC layer. Application solutions (QoS Agent and QoS SIP Proxy) were developed while the satellite network with its associated QoS architecture was implemented through a network emulator. This latter enables us to evaluate the QoS guarantees provided by this architecture and to validate its appropriateness with real multimedia applications needs