MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Timesheets.js: When SMIL Meets HTML5 and CSS3

International audienceIn this paper, we explore different ways to publish multimedia documents on the web. We propose a solution that takes advantage of the new multimedia features of web standards, namely HTML5 and CSS3. To avoid the usual development of complex scripts for handling timing, synchronization and user interaction, we propose to complement HTML5 and CSS3 with SMIL Timesheets. This is made possible by a Timesheets scheduler that runs in the browser. Various applications based on this solution illustrate the paper, ranging from media annotations to web documentaries

Datan synkronointi reaaliaikaisen videon suoratoistoon

Suoratoistopalveluiden räjähdysmäisestä kasvusta on seurannut suoratoisto- ja koodekkiteknologioiden nopea kehitys. Selainympäristö ei ole pysynyt tämän nopean kehityksen mukana. Tarjotakseen suoratoistoa kaikilla yleisesti käytetyillä selaimilla, sovelluskehittäjän on tuettava useampaa teknologiaa. Kasvava suoratoistopalveluiden kysyntä on aiheuttanut useita tarpeita multimediaratkaisuihin. Multimediassa on erityisen tärkeää, että kaikki datalähteet on synkronoitu. Tällaisen synkronoinnin toteuttaminen kaikille suoratoistoteknologioille on kallista ja teknologioihin perustuvien ratkaisujen käyttäminen estää uusien suoratoisto- ja koodausteknologioiden käyttöönoton. Tässä työssä esitetään median synkronointiin toteutus, joka ei ole riippuvainen suoratoisto- tai koodausteknologiasta. Toteutus pohjautuu äänisignaalin avulla dataa siirtäviin järjestelmiin. Suoratoistettavaan videon äänisignaaliin lisätään reaaliaikaisia aikaleimoja sisältävä äänisignaali taajuusavainnuksen avulla. Taajuusavainnuksessa käytetään heksadesimaalimerkkejä ja 16 niitä vastaavia eri taajuutta. Aikaleimoja tunnistetaan HTML5-standardin määrittämän rajapinnan avulla, joka on toteutettu kaikissa yleisimmissä selaimissa. Aikaleimojen tunnistuksessa käytetään nopeaa Fourierin muunnos -algoritmia, jolla signaali saadaan muutettua taajuustasoon. Taajuuksien tunnistamisessa käytetään hyvin häiriötä sietävää tapaa. Tämä takaa, ettei äänen koodausprosessilla ole merkitystä järjestelmän toimiseen, kunhan äänenlaatu säilytetään riittävänä. Tunnistettujen aikaleimojen avulla JavaScript-komponentti voi luoda aikajanan, joka mahdollistaa useamman datalähteen synkronoinnin tarkasti ja tehokkaasti. Toteutus evaluoitiin toimivaksi ja sen käyttämät parametrit optimoitiin käyttäen tarjolla olevia avoimia työkaluja

IDMS solution for hybrid broadcast broadband delivery within the context of HbbTV standard

"© 2019 IEEE. Personal use of this material is permitted. Permissíon from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertisíng or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works."[EN] Inter-destination media synchronization (IDMS) is a key requirement to enable successful networked shared media experiences between remote users. This paper presents an adaptive, accurate and standard-compliant IDMS solution for hybrid broadcast and broadband delivery. Apart from providing multi- and cross-technology support, the presented IDMS solution is able to accomplish synchronization when different formats/versions of the same, or even related, contents are being played out in a shared session. It is also able to independently manage the playout processes of different groups of users. The IDMS solution has been integrated within an end-to-end platform, which is compatible with the hybrid broadcast broadband TV standard. It has been applied to digital video broadcasting-terrestrial technology and tested for a Social TV scenario, by also including an ad-hoc chat tool as an interaction channel. The results of the conducted (objective and subjective) evaluations prove the statisfactory behavior and performance of the IDMS solution and platform as well as in terms of the perceived quality of experience.This work was supported by Generalitat Valenciana, Investigacion competitiva proyectos, through the Research and Development Program "Grants for research groups to be consolidated, AICO/2017," under Grant AICO/2017/059.Marfil-Reguero, D.; Boronat, F.; Montagud, M.; Sapena Piera, A. (2019). IDMS solution for hybrid broadcast broadband delivery within the context of HbbTV standard. IEEE Transactions on Broadcasting. 65(4):645-663. https://doi.org/10.1109/TBC.2018.2878285S64566365

Hybrid Broadcast/Broadband TV Services and Media Synchronization. Demands, Preferences and Expectations of Spanish Consumers

© 2018 IEEE. Personal use of this material is permitted. Permissíon from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertisíng or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.[EN] The combination of broadcast and broadband ( hybrid) technologies for delivering TV related media contents is already a reality. It has been motivated by the large amount and diversity of media contents, together with the ubiquity and multiple connectivity capabilities of modern consumption devices. The use of connected TVs and companion devices (e.g., tablets, smartphones etc.) is gaining momentum. It enables personalized and enriched TV media experiences, by also exploiting social communication opportunities. Likewise, the media consumption paradigm is worldwide evolving from passive and isolated consumer experiences toward interactive and group shared experiences between remote consumers. Nevertheless, despite the specification of standards, such as hybrid broadcast broadband TV, and the efforts from operators and content providers in the last years, the adoption of hybrid TV media services in Europe is still not as high as expected. This paper presents the concept and some examples of hybrid TV media services, emphasizing the importance of including a combination of media synchronization solutions, known as hybrid sync, to guarantee a satisfactory level of quality of experience. Additionally, it includes the summary and discussion of the results of a research study focused on more than 1000 Spanish users' habits, preferences, and expectations regarding four representative hybrid TV media services. Many valuable insights and conclusions have been derived. For instance, the current low market adoption of hybrid TV media services, despite their advantages and the high interest of consumers, and that key technological challenges still need to be overcome. The obtained results and impressions foresee the impact (or potential) of such services in the upcoming TV related media consumption landscape. Therefore, devising proper standard-compliant technological solutions (paying special attention to hybrid sync) and equipment should be continued producing appropriate contents, deploying proper hybrid TV media services and applications. As well, it should be convenient to undertake marketing and commercial efforts to boost their deployment. The contributions of this paper can be very valuable to the interested agents to be aware of the remaining challenges, envisage the opportunities, and drive their efforts to maximize the market adoption of such services.This work was supported in part by the Fondo Europeo de Desarrollo Regional, in part by the Spanish Ministry of Economy and Competitiveness under Its R&D&I Support Program under Project TEC2013-45492-R.Boronat, F.; Montagud, M.; Marfil-Reguero, D.; Luzón, C. (2018). Hybrid Broadcast/Broadband TV Services and Media Synchronization. Demands, Preferences and Expectations of Spanish Consumers. IEEE Transactions on Broadcasting. 64(1):52-69. https://doi.org/10.1109/TBC.2017.2737819S526964

HbbTV-compliant Platform for Hybrid Media Delivery and Synchronization on Single- and Multi-Device Scenarios

[EN] The combination of broadcast and broadband (hybrid) technologies for delivering TV related media contents can bring fascinating opportunities. It is motivated by the large amount and diversity of media contents, together with the ubiquity and multiple connectivity capabilities of modern consumption devices. This paper presents an end-to-end platform for the preparation, delivery, and synchronized consumption of related hybrid (broadcast/broadband) media contents on a single device and/or on multiple close-by devices (i.e., a multi-device scenario). It is compatible with the latest version of the Hybrid Broadcast Broadband TV (HbbTV) standard (version 2.0.1). Additionally, it provides adaptive and efficient solutions for key issues not specified in that standard, but that are necessary to successfully deploy hybrid and multidevice media services. Moreover, apart from MPEG-DASH and HTML5, which are the broadband technologies adopted by HbbTV, the platform also provides support for using HTTP Live Streaming and Real-time Transport Protocol and its companion RTP Control Protocol broadband technologies. The presented platform can provide support for many hybrid media services. In this paper, in order to evaluate it, the use case of multi-device and multi-view TV service has been selected. The results of both objective and subjective assessments have been very satisfactory, in terms of performance (stability, smooth playout, delays, and sync accuracy), usability of the platform, usefulness of its functionalities, and the awaken interest in these kinds of platforms.This work was supported in part by the "Fondo Europeo de Desarrollo Regional" and in part by the Spanish Ministry of Economy and Competitiveness through R&D&I Support Program under Grant TEC2013-45492-R.Boronat, F.; Marfil-Reguero, D.; Montagud, M.; Pastor Castillo, FJ. (2017). HbbTV-compliant Platform for Hybrid Media Delivery and Synchronization on Single- and Multi-Device Scenarios. IEEE Transactions on Broadcasting. 1-26. https://doi.org/10.1109/TBC.2017.2781124S12

PAMTEL-RT: Web-based Multimedia Platform for Tele-Assistance of Pediatric Health Emergencies in Real Time in Training Centers

[EN] In this paper, PAMTEL-RT, a web-based platform to provide remote health professional support and guidance during the initial assistance of emergency situations is presented. In particular, this platform focuses on the pediatric population and has been specifically designed to be used by non-healthcare professionals in training centers, such as educational or sport centers. It allows pediatricians in hospitals to use Information and Communication Technologies to assist or guide, in real-time, training personnel in charge of children when any emergency situation may arise. A direct and secure real-time connection between the medical and training centers is established, allowing sharing multiple media, such as audio, video, images, documents, text and data. Since the implementation relies on standard web-based technologies, cross-network, cross-platform and cross-device support are ensured. At first, an opinion and acceptance survey was conducted to collect requirements from some potential users to consider their opinions when designing and developing the platform. In line with the obtained results, the architecture of the platform has been designed, and a prototype has been developed and preliminarily evaluated through 12 drills/simulations in 7 education and sports centers. In the evaluation, 24 training personnel were involved, who considered that its usability was excellent and that it would have applicability in their centers. Furthermore, they considered that the included functionalities in the platform are appropriate and believed that the use of PAMTEL-RT could be highly effective in the early management of extreme emergency situations, and, in some cases, in avoiding tragic consequences.This work has been partially funded by the Universitat Politecnica de Valencia (UPV) and the Foundation for the Promotion of Health and Biomedical Research of Valencia Region (FISABIO), under their R&D&I Support Program with references UPV-FISABIO-2017-002-940. Authors also want to thank educational and sport institutions involved in the drills: Gregori Maians and Esclavas del Sagrado Corazón de Jesús Schools in Gandia city; Batoi and Pare Victoria Schools in Alcoi city; and U.E. Gandia Football Club, and C.A. Safor Delikia and C.C. El Garbí athletics clubs, in Gandia cityBoronat, F.; Escrivá, P.; Salvador-Llàcer, P.; Pareja, F.; Pastor, J. (2021). PAMTEL-RT: Web-based Multimedia Platform for Tele-Assistance of Pediatric Health Emergencies in Real Time in Training Centers. Multimedia Systems. 27(2):205-227. https://doi.org/10.1007/s00530-020-00730-0205227272Tomines, A.: Pediatric telehealth: approaches by specialty and implications for general pediatric care. Adv. Pediatr. 66, 55–85 (2019). https://doi.org/10.1016/j.yapd.2019.04.005European Commission. A healthy approach. Technology for personalized, preventative healthcare. ICT Research: the policy perspective. Publications Office of the European Union, 2010. 24 pp. ISBN 978-92-79-16085–1. (2010). https://doi.org/10.2759/33350EXPH (EXpert Panel on effective ways of investing in Health), Assessing the impact of digital transformation of health services, 20 November 2018. https://ec.europa.eu/health/expert_panel/sites/expertpanel/files/docsdir/022_digitaltransformation_en.pdf (2020). Accessed May 2020Olson, C.A., McSwain, S.D., Curfman, A.L., et al.: The current pediatric telehealth landscape. Pediatrics 141(3), e20172334 (2018)Burke, B.L., Jr., Hall, R.W.: Telemedicine: pediatric applications. Pediatrics 136(1), e293-308 (2015). https://doi.org/10.1542/peds.2015-1517McConnochie, K.M., Tan, J., Wood, N.E., et al.: Acute illness utilization patterns before and after telemedicine in childcare for inner-city children: a cohort study. Telemed. J. E Health 13(4), 381–390 (2007)McConnochie, K.M., Wood, N.E., Herendeen, N.E., et al.: Telemedicine in urban and suburban childcare and elementary schools lightens family burdens. Telemed. J. E Health 16(5), 533–542 (2010). https://doi.org/10.1089/tmj.2009.0138Utidjian, L., Abramson, E.: Pediatric telehealth: opportunities and challenges. Pediatr Clin. N. Am. 63(2), 367–378 (2016). https://doi.org/10.1016/j.pcl.2015.11.006Qubty, W., Patniyot, I., Gelfand, A.: Telemedicine in a pediatric headache clinic: a prospective survey. Neurology. 90(19), e1702–e1705 (2018). https://doi.org/10.1212/WNL.0000000000005482Stavas, N., Shea, J., Keddem, S., et al.: Perceptions of caregivers and adolescents of the use of telemedicine for the child sexual abuse examination. Child Abuse Negl. 85, 47–57 (2018)Vyas, S., Murren-Boezem, J., Solo-Josephson, P.: Analysis of a pediatric telemedicine program. Prog. Telemed. J. E Health (2018). https://doi.org/10.1089/tmj.2017.0281Bala, N., Price, S.N., Horan, C.M., et al.: Use of telehealth to enhance care in a family-centered childhood obesity intervention. Clin. Pediatr. 58(7), 789–797 (2019)mHealth Economics 2017. Current Status and Future Trends in Mobile Health: Research 2 Guidance. http://www.uzelf.org/wp-content/uploads/2017/12/R2G-mHealth-Developer-Economics-2017-Status-And-Trends.pdf (2020). Accessed May 2020Cisotto, G., Casarin, E., Tomasin, S.: Requirements and enablers of advanced healthcare services over future cellular systems. IEEE Commun. Mag. 58(3), 76–81 (2020). https://doi.org/10.1109/MCOM.001.1900349Morosi, S., Jayousi, S., Mucchi, L., et al.: Medical Tele-Monitoring and Tele-Assistance for Diabetics Patients by Means of 5G Cellular Networks, in Proceedings of BodyNets 2019. In: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 297. Springer, Cham (2019)Montagud, M., Boronat, F., Belda, J., et al.: Use of Web Components to Develop Interactive, Customizable and Multi-Device Video Consumption Platforms, Applications and Usability of Interactive TV, pp. 26–43. Springer, Berlin (2016)Arntzen, I., Bjorch, N., Daoust, F.: Media Synchronization on the Web. In: Montagud, M., Cesar, P., Boronat, F., Jansen, J. (eds.) MediaSync: Handbook on Multimedia Synchronization. Springer, Cham (2018)Pauwels, D., van der Hooft, J., Petrangeli, S., et al.: A Web-based framework for fast synchronization of live video players, IFIP/IEEE Symposium on Integrated Network and Service Management, Lisbon, 2017, pp. 524–530 (2017)Rainer, B., Petscharnig, S., Timmerer, C.: Merge and forward: a self-organized inter-destination media synchronization scheme for adaptive media streaming over HTTP. In: Montagud, M., Cesar, P., Boronat, F., Jansen, J. (eds.) MediaSync: Handbook on Multimedia Synchronization. Springer, Cham (2018)Montagud, M., Boronat, F., Stokking, H., et al.: Inter-destination multimedia synchronization: schemes, use cases and standardization. Multimedia Syst. J. 18(6), 459–482 (2012). https://doi.org/10.1007/s00530-012-0278-9Montagud, M., Boronat, F., Stokking, H., et al.: Design development and assessment of control schemes for IDMS in a standardized RTCP-based solution. COMNET 70(9), 240–259 (2014). https://doi.org/10.1016/j.comnet.2014.06.004Dynamic Adaptive Streaming Over HTTP (DASH): Part 1: media presentation description and segment formats, ISO/IEC Standard 23009-1: 2012 (2012)Gómez, D., Boronat, F., Montagud, M., et al.: End-to-end DASH platform including a network-based and client-based adaptive quality switching module. In: Proc. Int. Conf. Multimedia Syst., Klagenfurt, Austria, Art. no. 38 (2016)Melvin, H., Shannon, J., Stanton, K.: Time, frequency and phase synchronization for multimedia—basics, issues, developments and opportunities. In: Montagud, M., Cesar, P., Boronat, F., Jansen, J. (eds.) MediaSync: Handbook on Multimedia Synchronization. Springer, Cham (2018)Boronat, F., Lloret, J., García, M.: Multimedia group and inter-stream synchronization techniques: a comparative study. Inf. Syst. 34(1), 108–131 (2009). https://doi.org/10.1016/j.is.2008.05.001Montagud, M., Boronat, F., Pastor, J., et al.: Web-based platform for a customizable and synchronized presentation of subtitles in single- and multi-screen scenarios. Multimed Tools Appl (2020). https://doi.org/10.1007/s11042-020-08955-xSu, Y., Yang, Y., Lu, M., et al.: Smooth control of adaptive media playout for video streaming. IEEE TMM 1(7), 1331–1339 (2009)Rainer, B., Timmerer, C.: Self-Organized inter-destination multimedia synchronization for adaptive media streaming. In: Proceedings of the 22nd ACM international conference on Multimedia (MM '14). ACM, New York, NY, USA, pp. 327–336 (2014)Montagud, M., Boronat, F., Roig, B., et al.: How to perform AMP? Cubic adjustments for improving the QoE. Comput. Commun. 103, 61–73 (2017). https://doi.org/10.1016/j.comcom.2017.01.017Hossfeld, T., Seufert, M., Hirth, M., et al.: Quantification of YouTube QoE via crowdsourcing. IEEE ISM (2011). https://doi.org/10.1109/ISM.2011.87Park, S., Kim, J.: An adaptive media playout for intra-media synchronization of networked-video applications. J. Vis. Commun. Image Represent. 19(2), 106–120 (2008)Brooke, J.: Sus-a quick and dirty usability scale. In the book “Usability evaluation in industry”, issue 194, pages 189–194, Taylor and Francis Ed (1996)Bangor, A., Kortum, P., Miller, J.: Determining what individual sus scores mean: adding an adjective rating scale. J. Usability Stud. 4(3), 114–123 (2009

Impact of Social Media on TV Content Consumption: New Market Strategies, Scenarios and Trends

The mass adoption of Social Media together with the proliferation and widely usage of multi-connected companion devices have tremendously transformed the TV/video consumption paradigm, opening the door to a new range of possibilities. This Special Issue has aimed at analyzing, from different point of views, the impact of Social Media and social interaction tools on the TV/video consumption area. The targeted topics of this Special Issue and a general overview of the accepted articles are provided in this Guest Editorial

Binaural Spatialization for 3D immersive audio communication in a virtual world

Realistic 3D audio can greatly enhance the sense of presence in a virtual environment. We introduce a framework for capturing, transmitting and rendering of 3D audio in presence of other bandwidth savvy streams in a 3D Tele-immersion based virtual environment. This framework presents an efficient implementation for 3D Binaural Spatialization based on the positions of current objects in the scene, including animated avatars and on the fly reconstructed humans. We present a general overview of the framework, how audio is integrated in the system and how it can exploit the positions of the objects and room geometry to render realistic reverberations using head related transfer functions. The network streaming modules used to achieve lip-synchronization, high-quality audio frame reception, and accurate localization for binaural rendering are also presented. We highlight how large computational and networking challenges can be addressed efficiently. This represents a first step in adequate networking support for Binaural 3D Audio, useful for telepresence. The subsystem is successfully integrated with a larger 3D immersive system, with state of art capturing and rendering modules for visual data