How to Perform AMP? Cubic Adjustments for Improving the QoE

[EN] Adaptive Media Playout (AMP) consists of smoothly and dynamically adjusting the media playout rate to recover from undesired (e.g., buffer overflow/underflow or out-of-sync) situations. The existing AMP solutions are mainly characterized by two main aspects. The first one is their goal (e.g., keeping the buffers¿ occupancy into safe ranges or enabling media synchronization). The second one is the criteria that determine the need for triggering the playout adjustments (e.g., buffer fullness or asynchrony levels). This paper instead focuses on a third key aspect, which has not been sufficiently investigated yet: the specific adjustment strategy to be performed. In particular, we propose a novel AMP strategy, called Cubic AMP, which is based on employing a cubic interpolation method to adjust a deviated playout point to a given reference. On the one hand, mathematical analysis and graphical examples show that our proposal provides superior performance than other existing linear and quadratic AMP strategies in terms of the smoothness of the playout curve, while significantly outperforming the quadratic AMP strategy regarding the duration of the adjustment period and without increasing the computational complexity. It has also been proved and discussed that higher-order polynomial interpolation methods are less convenient than cubic ones. On the other hand, the results of subjective tests confirm that our proposal provides better Quality of Experience (QoE) than the other existing AMP strategies.This work has been funded, partially, by the “Fondo Europeo de Desarrollo Regional (FEDER)” and the Spanish Ministry of Economy and Competitiveness, under its R&D&I Support Program, in project with Ref. TEC2013-45492-R.Montagud, M.; Boronat, F.; Roig, B.; Sapena Piera, A. (2017). How to Perform AMP? Cubic Adjustments for Improving the QoE. Computer Communications. 103:61-73. https://doi.org/10.1016/j.comcom.2017.01.017S617310

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

QoE-aware inter-stream synchronization in open N-screens cloud

The growing popularity and increasing performance of mobile devices is transforming the way in which media can be consumed, from single device playback to orchestrated multi-stream experiences across multiple devices. One of the biggest challenges in realizing such immersive media experience is the dynamic management of synchronicity between associated media streams. This is further complicated by the faceted aspects of user perception and heterogeneity of user devices and networks. This paper introduces a QoE-aware open inter-stream media synchronization framework (IMSync). IMSync employs efficient monitoring and control mechanisms, as well as a bespoke QoE impact model derived from subjective user experiments. Given a current lag, IMSync's aim is to use the impact model to determine a good catch-up strategy that minimizes detrimental impact on QoE. The impact model balances the accumulative impact of re-synchronization processes and the degree of non-synchronicity to ensure the QoE. Experimental results verify the run-time performance of the framework as a foundation for immersive media experience in open N-Screens cloud

Closing the gap: human factors in cross-device media synchronization

The continuing growth in the mobile phone arena, particularly in terms of device capabilities and ownership is having a transformational impact on media consumption. It is now possible to consider orchestrated multi-stream experiences delivered across many devices, rather than the playback of content from a single device. However, there are significant challenges in realising such a vision, particularly around the management of synchronicity between associated media streams. This is compounded by the heterogeneous nature of user devices, the networks upon which they operate, and the perceptions of users. This paper describes IMSync, an open inter-stream synchronisation framework that is QoE-aware. IMSync adopts efficient monitoring and control mechanisms, alongside a QoE perception model that has been derived from a series of subjective user experiments. Based on an observation of lag, IMSync is able to use this model of impact to determine an appropriate strategy to catch-up with playback whilst minimising the potential detrimental impacts on a users QoE. The impact model adopts a balanced approach: trading off the potential impact on QoE of initiating a re-synchronisation process compared with retaining the current levels of non-synchronicity, in order to maintain high levels of QoE. A series of experiments demonstrate the potential of the framework as a basis for enabling new, immersive media experiences

A Semantic-Based Middleware for Multimedia Collaborative Applications

The Internet growth and the performance increase of desktop computers have enabled large-scale distributed multimedia applications. They are expected to grow in demand and services and their traffic volume will dominate. Real-time delivery, scalability, heterogeneity are some requirements of these applications that have motivated a revision of the traditional Internet services, the operating systems structures, and the software systems for supporting application development. This work proposes a Java-based lightweight middleware for the development of large-scale multimedia applications. The middleware offers four services for multimedia applications. First, it provides two scalable lightweight protocols for floor control. One follows a centralized model that easily integrates with centralized resources such as a shared too], and the other is a distributed protocol targeted to distributed resources such as audio. Scalability is achieved by periodically multicasting a heartbeat that conveys state information used by clients to request the resource via temporary TCP connections. Second, it supports intra- and inter-stream synchronization algorithms and policies. We introduce the concept of virtual observer, which perceives the session as being in the same room with a sender. We avoid the need for globally synchronized clocks by introducing the concept of user\u27s multimedia presence, which defines a new manner for combining streams coming from multiple sites. It includes a novel algorithm for estimation and removal of clock skew. In addition, it supports event-driven asynchronous message reception, quality of service measures, and traffic rate control. Finally, the middleware provides support for data sharing via a resilient and scalable protocol for transmission of images that can dynamically change in content and size. The effectiveness of the middleware components is shown with the implementation of Odust, a prototypical sharing tool application built on top of the middleware

Quality of service optimization of multimedia traffic in mobile networks

Mobile communication systems have continued to evolve beyond the currently deployed Third Generation (3G) systems with the main goal of providing higher capacity. Systems beyond 3G are expected to cater for a wide variety of services such as speech, data, image transmission, video, as well as multimedia services consisting of a combination of these. With the air interface being the bottleneck in mobile networks, recent enhancing technologies such as the High Speed Downlink Packet Access (HSDPA), incorporate major changes to the radio access segment of 3G Universal Mobile Telecommunications System (UMTS). HSDPA introduces new features such as fast link adaptation mechanisms, fast packet scheduling, and physical layer retransmissions in the base stations, necessitating buffering of data at the air interface which presents a bottleneck to end-to-end communication. Hence, in order to provide end-to-end Quality of Service (QoS) guarantees to multimedia services in wireless networks such as HSDPA, efficient buffer management schemes are required at the air interface. The main objective of this thesis is to propose and evaluate solutions that will address the QoS optimization of multimedia traffic at the radio link interface of HSDPA systems. In the thesis, a novel queuing system known as the Time-Space Priority (TSP) scheme is proposed for multimedia traffic QoS control. TSP provides customized preferential treatment to the constituent flows in the multimedia traffic to suit their diverse QoS requirements. With TSP queuing, the real-time component of the multimedia traffic, being delay sensitive and loss tolerant, is given transmission priority; while the non-real-time component, being loss sensitive and delay tolerant, enjoys space priority. Hence, based on the TSP queuing paradigm, new buffer managementalgorithms are designed for joint QoS control of the diverse components in a multimedia session of the same HSDPA user. In the thesis, a TSP based buffer management algorithm known as the Enhanced Time Space Priority (E-TSP) is proposed for HSDPA. E-TSP incorporates flow control mechanisms to mitigate congestion in the air interface buffer of a user with multimedia session comprising real-time and non-real-time flows. Thus, E-TSP is designed to provide efficient network and radio resource utilization to improve end-to-end multimedia traffic performance. In order to allow real-time optimization of the QoS control between the real-time and non-real-time flows of the HSDPA multimedia session, another TSP based buffer management algorithm known as the Dynamic Time Space Priority (D-TSP) is proposed. D-TSP incorporates dynamic priority switching between the real-time and non-real-time flows. D-TSP is designed to allow optimum QoS trade-off between the flows whilst still guaranteeing the stringent real-time component’s QoS requirements. The thesis presents results of extensive performance studies undertaken via analytical modelling and dynamic network-level HSDPA simulations demonstrating the effectiveness of the proposed TSP queuing system and the TSP based buffer management schemes

HbbTV-compliant Platform for Hybrid Media Delivery and Synchronization on Single- and Multi-Device Scenarios

[EN] The combination of broadcast and broadband (hybrid) technologies for delivering TV related media contents can bring fascinating opportunities. It is motivated by the large amount and diversity of media contents, together with the ubiquity and multiple connectivity capabilities of modern consumption devices. This paper presents an end-to-end platform for the preparation, delivery, and synchronized consumption of related hybrid (broadcast/broadband) media contents on a single device and/or on multiple close-by devices (i.e., a multi-device scenario). It is compatible with the latest version of the Hybrid Broadcast Broadband TV (HbbTV) standard (version 2.0.1). Additionally, it provides adaptive and efficient solutions for key issues not specified in that standard, but that are necessary to successfully deploy hybrid and multidevice media services. Moreover, apart from MPEG-DASH and HTML5, which are the broadband technologies adopted by HbbTV, the platform also provides support for using HTTP Live Streaming and Real-time Transport Protocol and its companion RTP Control Protocol broadband technologies. The presented platform can provide support for many hybrid media services. In this paper, in order to evaluate it, the use case of multi-device and multi-view TV service has been selected. The results of both objective and subjective assessments have been very satisfactory, in terms of performance (stability, smooth playout, delays, and sync accuracy), usability of the platform, usefulness of its functionalities, and the awaken interest in these kinds of platforms.This work was supported in part by the "Fondo Europeo de Desarrollo Regional" and in part by the Spanish Ministry of Economy and Competitiveness through R&D&I Support Program under Grant TEC2013-45492-R.Boronat, F.; Marfil-Reguero, D.; Montagud, M.; Pastor Castillo, FJ. (2017). HbbTV-compliant Platform for Hybrid Media Delivery and Synchronization on Single- and Multi-Device Scenarios. IEEE Transactions on Broadcasting. 1-26. https://doi.org/10.1109/TBC.2017.2781124S12

Multimedia delivery in the future internet

The term “Networked Media” implies that all kinds of media including text, image, 3D graphics, audio and video are produced, distributed, shared, managed and consumed on-line through various networks, like the Internet, Fiber, WiFi, WiMAX, GPRS, 3G and so on, in a convergent manner [1]. This white paper is the contribution of the Media Delivery Platform (MDP) cluster and aims to cover the Networked challenges of the Networked Media in the transition to the Future of the Internet. Internet has evolved and changed the way we work and live. End users of the Internet have been confronted with a bewildering range of media, services and applications and of technological innovations concerning media formats, wireless networks, terminal types and capabilities. And there is little evidence that the pace of this innovation is slowing. Today, over one billion of users access the Internet on regular basis, more than 100 million users have downloaded at least one (multi)media file and over 47 millions of them do so regularly, searching in more than 160 Exabytes1 of content. In the near future these numbers are expected to exponentially rise. It is expected that the Internet content will be increased by at least a factor of 6, rising to more than 990 Exabytes before 2012, fuelled mainly by the users themselves. Moreover, it is envisaged that in a near- to mid-term future, the Internet will provide the means to share and distribute (new) multimedia content and services with superior quality and striking flexibility, in a trusted and personalized way, improving citizens’ quality of life, working conditions, edutainment and safety. In this evolving environment, new transport protocols, new multimedia encoding schemes, cross-layer inthe network adaptation, machine-to-machine communication (including RFIDs), rich 3D content as well as community networks and the use of peer-to-peer (P2P) overlays are expected to generate new models of interaction and cooperation, and be able to support enhanced perceived quality-of-experience (PQoE) and innovative applications “on the move”, like virtual collaboration environments, personalised services/ media, virtual sport groups, on-line gaming, edutainment. In this context, the interaction with content combined with interactive/multimedia search capabilities across distributed repositories, opportunistic P2P networks and the dynamic adaptation to the characteristics of diverse mobile terminals are expected to contribute towards such a vision. Based on work that has taken place in a number of EC co-funded projects, in Framework Program 6 (FP6) and Framework Program 7 (FP7), a group of experts and technology visionaries have voluntarily contributed in this white paper aiming to describe the status, the state-of-the art, the challenges and the way ahead in the area of Content Aware media delivery platforms