Inter-Destination Multimedia Synchronization; Schemes, Use Cases and Standardization

Traditionally, the media consumption model has been a passive and isolated activity. However, the advent of media streaming technologies, interactive social applications, and synchronous communications, as well as the convergence between these three developments, point to an evolution towards dynamic shared media experiences. In this new model, geographically distributed groups of consumers, independently of their location and the nature of their end-devices, can be immersed in a common virtual networked environment in which they can share multimedia services, interact and collaborate in real-time within the context of simultaneous media content consumption. In most of these multimedia services and applications, apart from the well-known intra and inter-stream synchronization techniques that are important inside the consumers playout devices, also the synchronization of the playout processes between several distributed receivers, known as multipoint, group or Inter-destination multimedia synchronization (IDMS), becomes essential. Due to the increasing popularity of social networking, this type of multimedia synchronization has gained in popularity in recent years. Although Social TV is perhaps the most prominent use case in which IDMS is useful, in this paper we present up to 19 use cases for IDMS, each one having its own synchronization requirements. Different approaches used in the (recent) past by researchers to achieve IDMS are described and compared. As further proof of the significance of IDMS nowadays, relevant organizations (such as ETSI TISPAN and IETF AVTCORE Group) efforts on IDMS standardization (in which authors have been and are participating actively), defining architectures and protocols, are summarized.This work has been financed, partially, by Universitat Politecnica de Valencia (UPV), under its R&D Support Program in PAID-05-11-002-331 Project and in PAID-01-10, and by TNO, under its Future Internet Use Research & Innovation Program. The authors also want to thank Kevin Gross for providing some of the use cases included in Sect. 1.2.Montagud, M.; Boronat Segui, F.; Stokking, H.; Van Brandenburg, R. (2012). Inter-Destination Multimedia Synchronization; Schemes, Use Cases and Standardization. Multimedia Systems. 18(6):459-482. https://doi.org/10.1007/s00530-012-0278-9S459482186Kernchen, R., Meissner, S., Moessner, K., Cesar, P., Vaishnavi, I., Boussard, M., Hesselman, C.: Intelligent multimedia presentation in ubiquitous multidevice scenarios. IEEE Multimedia 17(2), 52–63 (2010)Vaishnavi, I., Cesar, P., Bulterman, D., Friedrich, O., Gunkel, S., Geerts, D.: From IPTV to synchronous shared experiences challenges in design: distributed media synchronization. Signal Process Image Commun 26(7), 370–377 (2011)Geerts, D., Vaishnavi, I., Mekuria, R., Van Deventer, O., Cesar, P.: Are we in sync?: synchronization requirements for watching on-line video together, CHI ‘11, New York, USA (2011)Boronat, F., Lloret, J., García, M.: Multimedia group and inter-stream synchronization techniques: a comparative study. Inf. Syst. 34(1), 108–131 (2009)Chen, M.: A low-latency lip-synchronized videoconferencing system. In: SIGCHI Conference on Human Factors in Computing Systems, CHI’03, ACM, pp. 464–471, New York (2003)Ishibashi, Y., Tasaka, S., Ogawa, H.: Media synchronization quality of reactive control schemes. IEICE Trans. Commun. E86-B(10), 3103–3113 (2003)Ademoye, O.A., Ghinea, G.: Synchronization of olfaction-enhanced multimedia. IEEE Trans. Multimedia 11(3), 561–565 (2009)Cesar, P., Bulterman, D.C.A., Jansen, J., Geerts, D., Knoche, H., Seager, W.: Fragment, tag, enrich, and send: enhancing social sharing of video. ACM Trans. Multimedia Comput. Commun. Appl. 5(3), Article 19, 27 pages (2009)Van Deventer, M.O., Stokking, H., Niamut, O.A., Walraven, F.A., Klos, V.B.: Advanced Interactive Television Service Require Synchronization, IWSSIP 2008. Bratislava, June (2008)Premchaiswadi, W., Tungkasthan, A., Jongsawat, N.: Enhancing learning systems by using virtual interactive classrooms and web-based collaborative work. In: Proceedings of the IEEE Education Engineering Conference (EDUCON 2010), pp. 1531–1537. Madrid, Spain (2010)Diot, C., Gautier, L.: A distributed architecture for multiplayer interactive applications on the internet. IEEE Netw 13(4), 6–15 (1999)Mauve, M., Vogel, J., Hilt, V., Effelsberg, W.: Local-lag and timewarp: providing consistency for replicated continuous applications. IEEE Trans. Multimedia 6(1), 45–57 (2004)Hosoya, K., Ishibashi, Y., Sugawara, S., Psannis, K.E.: Group synchronization control considering difference of conversation roles. In: IEEE 13th International Symposium on Consumer Electronics, ISCE ‘09, pp. 948–952 (2009)Roccetti, M., Ferretti, S., Palazzi, C.: The brave new world of multiplayer online games: synchronization issues with smart solution. In: 11th IEEE Symposium on Object Oriented Real-Time Distributed Computing (ISORC), pp. 587–592 (2008)Ott, D.E., Mayer-Patel, K.: An open architecture for transport-level protocol coordination in distributed multimedia applications. ACM Trans. Multimedia Comput. Commun. Appl. 3(3), 17 (2007)Boronat, F., Montagud, M., Guerri, J.C.: Multimedia group synchronization approach for one-way cluster-to-cluster applications. In: IEEE 34th Conference on Local Computer Networks, LCN 2009, pp. 177–184, Zürich (2009)Boronat, F., Montagud, M., Vidal, V.: Smooth control of adaptive media playout to acquire IDMS in cluster-based applications. In: IEEE LCN 2011, pp. 617–625, Bonn (2011)Huang, Z., Wu, W., Nahrstedt, K., Rivas, R., Arefin, A.: SyncCast: synchronized dissemination in multi-site interactive 3D tele-immersion. In: Proceedings of MMSys, USA (2011)Kim, S.-J., Kuester, F., Kim, K.: A global timestamp-based approach for enhanced data consistency and fairness in collaborative virtual environments. ACM/Springer Multimedia Syst. J. 10(3), 220–229 (2005)Schooler, E.: Distributed music: a foray into networked performance. In: International Network Music Festival, Santa Monica, CA (1993)Miyashita, Y., Ishibashi, Y., Fukushima, N., Sugawara, S., Psannis K.E.: QoE assessment of group synchronization in networked chorus with voice and video. In: Proceedings of IEEE TENCON’11, pp. 393–397 (2011)Hesselman, C., Abbadessa, D., Van Der Beek, W., et al.: Sharing enriched multimedia experiences across heterogeneous network infrastructures. IEEE Commun. Mag. 48(6), 54–65 (2010)Montpetit, M., Klym, N., Mirlacher, T.: The future of IPTV—Connected, mobile, personal and social. Multimedia Tools Appl J 53(3), 519–532 (2011)Cesar, P., Bulterman, D.C.A., Jansen, J.: Leveraging the user impact: an architecture for secondary screens usage in an interactive television environment. ACM/Springer Multimedia Syst. 15(3), 127–142 (2009)Lukosch, S.: Transparent latecomer support for synchronous groupware. In: Proceedings of 9th International Workshop on Groupware (CRIWG), Grenoble, France, pp. 26–41 (2003)Steinmetz, R.: Human perception of jitter and media synchronization. IEEE J. Sel. Areas Commun. 14(1), 61–72 (1996)Stokking, H., Van Deventer, M.O., Niamut, O.A., Walraven, F.A., Mekuria, R.N.: IPTV inter-destination synchronization: a network-based approach, ICIN’2010, Berlin (2010)Mekuria, R.N.: Inter-destination media synchronization for TV broadcasts, Master Thesis, Faculty of Electrical Engineering, Mathematics and Computer Science, Department of Network architecture and Services, Delft University of Technology (2011)Pitt Ian, CS2511: Usability engineering lecture notes, localisation of sound sources. http://web.archive.org/web/20100410235208/http:/www.cs.ucc.ie/~ianp/CS2511/HAP.htmlNielsen, J.: Response times: the three important limits. http://www.useit.com/papers/responsetime.html (1994)ITU-T Rec G. 1010: End-User Multimedia QoS Categories. International Telecommunication Union, Geneva (2001)Biersack, E., Geyer, W.: Synchronized delivery and playout of distributed stored multimedia streams. ACM/Springer Multimedia Syst 7(1), 70–90 (1999)Xie, Y., Liu, C., Lee, M.J., Saadawi, T.N.: Adaptive multimedia synchronization in a teleconference system. ACM/Springer Multimedia Syst. 7(4), 326–337 (1999)Laoutaris, N., Stavrakakis, I.: Intrastream synchronization for continuous media streams: a survey of playout schedulers. IEEE Netw. Mag. 16(3), 30–40 (2002)Ishibashi, Y., Tsuji, A., Tasaka, S.: A group synchronization mechanism for stored media in multicast communications. In: Proceedings of the INFOCOM ‘97, Washington (1997)Ishibashi, Y., Tasaka, S.: A group synchronization mechanism for live media in multicast communications. IEEE GLOBECOM’97, pp. 746–752 (1997)Boronat, F., Guerri, J.C., Lloret, J.: An RTP/RTCP based approach for multimedia group and inter-stream synchronization. Multimedia Tools Appl. J. 40(2), 285–319 (2008)Ishibashi, I., Tasaka, S.: A distributed control scheme for group synchronization in multicast communications. In: Proceedings of International Symposium Communications, Kaohsiung, Taiwan, pp. 317–323 (1999)Lu, Y., Fallica, B., Kuipers, F.A., Kooij, R.E., Van Mieghem, P.: Assessing the quality of experience of SopCast. Int. J. Internet Protoc. Technol 4(1), 11–19 (2009)Shamma, D.A., Bastea-Forte, M., Joubert, N., Liu, Y.: Enhancing online personal connections through synchronized sharing of online video, ACM CHI’08 Extended Abstracts, Florence (2008)Ishibashi, Y., Tasaka, S.: A distributed control scheme for causality and media synchronization in networked multimedia games. In: Proceedings of 11th International Conference on Computer Communications and Networks, pp. 144–149, Miami, USA (2002)Ishibashi, Y., Tomaru, K., Tasaka, S., Inazumi, K.: Group synchronization in networked virtual environments. In: Proceedings of the 38th IEEE International Conference on Communications, pp. 885–890, Alaska, USA (2003)Tasaka, S., Ishibashi, Y., Hayashi, M.: Inter–destination synchronization quality in an integrated wired and wireless network with handover. IEEE GLOBECOM 2, 1560–1565 (2002)Kurokawa, Y., Ishibashi, Y., Asano, T.: Group synchronization control in a remote haptic drawing system. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 572–575, Beijing, China (2007)Hashimoto, T., Ishibashi, Y.: Group Synchronization Control over Haptic Media in a Networked Real-Time Game with Collaborative Work, Netgames’06, Singapore (2006)Nunome, T., Tasaka, S.: Inter-destination synchronization quality in a multicast mobile ad hoc network. In: Proceedings of IEEE 16th International Symposium on Personal, Indoor and Mobile Radio Communications, pp. 1366–1370, Berlin, Germany (2005)Brandenburg, R., van Stokking, H., Van Deventer, M.O., Boronat, F., Montagud, M., Gross, K.: RTCP for inter-destination media synchronization, draft-brandenburg-avtcore-rtcp-for-idms-03.txt. In: IETF Audio/Video Transport Core Maintenance Working Group, Internet Draft, March 9 (2012)ETSI TS 181 016 V3.3.1 (2009-07) Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); Service Layer Requirements to integrate NGN Services and IPTVETSI TS 182 027 V3.5.1 (2011-03) Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); IPTV Architecture; IPTV functions supported by the IMS subsystemETSI TS 183 063 V3.5.2 (2011-03) Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); IMS-based IPTV stage 3 specificationBrandenburg van, R., et al.: RTCP XR Block Type for inter-destination media synchronization, draft-brandenburg-avt-rtcp-for-idms-00.txt. In: IETF Audio/Video Transport Working Group, Internet Draft, Sept 24, 2010Williams, A., et al.: RTP Clock Source Signalling, draft-williams-avtcore-clksrc-00. In: IETF Audio/Video Transport Working Group, Internet Draft, February 28, 201

Wireless triple play system

Dissertação para obtenção do Grau de Mestre em Engenharia Electrotécnica e ComputadoresTriple play is a service that combines three types of services: voice, data and multimedia over a single communication channel for a price that is less than the total price of the individual services. However there is no standard for provisioning the Triple play services, rather they are provisioned individually, since the requirements are quite different for each service. The digital revolution helped to create and deliver a high quality media solutions. One of the most demanding services is the Video on Demand (VoD). This implicates a dedicated streaming channel for each user in order to provide normal media player commands (as pause, fast forward). Most of the multimedia companies that develops personalized products does not always fulfil the users needs and are far from being cheap solutions. The goal of the project was to create a reliable and scalable triple play solution that works via Wireless Local Area Network (WLAN), fully capable of dealing with the existing state of the art multimedia technologies only resorting to open-source tools. This project was design to be a transparent web environment using only web technologies to maximize the potential of the services. HyperText Markup Language (HTML),Cascading Style Sheets (CSS) and JavaScript were the used technologies for the development of the applications. Both a administration and user interfaces were developed to fully manage all video contents and properly view it in a rich and appealing application, providing the proof of concept. The developed prototype was tested in a WLAN with up to four clients and the Quality of Service (QoS) and Quality of Experience (QoE) was measured for several combinations of active services. In the end it is possible to acknowledge that the developed prototype was capable of dealing with all the problems of WLAN technologies and successfully delivery all the proposed services with high QoE

High-Speed Communications Over Polymer Optical Fibers for In-Building Cabling and Home Networking

This paper focuses on high-speed cabling using polymer optical fibers (POF) in home networking. In particular, we report about the results obtained in the POF-ALL European Project, which is relevant to the Sixth Framework Program, and after two years of the European Project POF-PLUS, which is relevant to the Seventh Framework Program, focusing on their research activities about the use of poly-metyl-metha-acrilate step-index optical fibers for home applications. In particular, for that which concerns POF-ALL, we will describe eight-level pulse amplitude modulation (8-PAM) and orthogonal frequency-division multiplexing (OFDM) approaches for 100-Mb/s transmission over a target distance of 300 m, while for that which concerns POF-PLUS, we will describe a fully digital and a mixed analog-digital solution, both based on intensity modulation direct detection, for transmitting 1 Gb/s over a target distance of 50 m. The ultimate experimental results from the POF-ALL project will be given, while for POF-PLUS, which is still ongoing, we will only show our most recent preliminary results

The Communicating Home - Definition, Evaluation and Business Opportunities for TeliaSonera in a 3-5 years perspective

The communicating home concept has been defined by identifying the dominating communication technologies to/from and in the homes, the most important customer behavior and user needs, the dominating products and the dominating actors of the industry. The evaluation of the communicating home industry has been performed with a five forces framework analysis and theories regarding value migration and value structures of industries. It has been concluded that actors of communication technologies, goods and access services will compete fiercely. The barriers of entry will however be high. When it comes to content services the situation is completely different. These actors will, to the largest extent, meet lower competition and low barriers of entry. As a consequence a value migration from communication technologies, goods and access services to content services will occur. TeliaSonera is suggested to develop a content service portal. It should, in its first version, contain IPTV, music on demand, video on demand, data security, online storing and online gaming in the first place. The TeliaSonera group has the right prerequisites to implement and capitalize on a content service portal. In addition to the content service portal some other business opportunities have also been addressed in the business model

Tecnología IPTV para el servicio de Internet inalámbrico proveedor (WISP)

This article describes the implementation of the IPTV technology through an open source system for the wireless Internet service provider (WISP) in the Cerecita site, located in the province of Guayas, Ecuador. The feasibility of implementing the IPTV technology is given through the use of the Emby Server application, in addition to the use of an operating system such as CentOS 7, which are open source and have the necessary requirements for the creation of this service. The analysis of data tra c generated on the WISP network, which will establish the advantages and disadvantages generated by the transmission of this technology, created to VOD multimedia content server that allows the video signal to be sent to users Finally, a feasibility study was carried out to analyze the advantages and disadvantages that exist in the implementation of this IPTV technology.Este artículo describe la implementación de la tecnología IPTV a través de un sistema de código abierto para el proveedor de servicios de Internet inalámbrico (WISP) en el sitio Cerecita, ubicado en la provincia de Guayas, Ecuador. La factibilidad de implementar la tecnología IPTV se da mediante el uso de la aplicación Emby Server, además del uso de un sistema operativo como CentOS 7, que son de código abierto y tienen los requisitos necesarios para la creación de este servicio. El análisis del trá co de datos generado en la red WISP, que establecerá las ventajas y desventajas generadas por la transmisión de esta tecnología, creado para el servidor de contenido multimedia VOD que permite enviar la señal de video a los usuarios Finalmente, se llevó a cabo un estudio de factibilidad analizar las ventajas y desventajas que existen en la implementación de esta tecnología IPTV

Network and service monitoring in heterogeneous home networks

Home networks are becoming dynamic and technologically heterogeneous. They consist of an increasing number of devices which offer several functionalities and can be used for many different services. In the home, these devices are interconnected using a mixture of networking technologies (for example, Ethernet, Wifi, coaxial cable, or power-line). However, interconnecting these devices is often not easy. The increasing heterogeneity has led to significant device- and service-management complexity. In addition, home networks provide a critical "last meters" access to the public telecom and Internet infrastructure and have a dramatic impact on to the end-to-end reliability and performance of services from these networks. This challenges service providers not only to maintain a satisfactory quality of service level in such heterogeneous home networks, but also to remotely monitor and troubleshoot them. The present thesis work contributes research and several solutions in the field of network and service monitoring in home networks, mainly in three areas: (1) providing automatic device- and service-discovery and configuration, (2) remote management, and (3) providing quality of service (QoS). With regard to the first area, current service discovery technology is designed to relieve the increasing human role in network and service administration. However, the relevant Service Discovery Protocols (SDPs) are lacking crucial features namely: (1) they are not platform- and network-independent, and (2) they do not provide sufficient mechanisms for (device) resource reservation. Consequently, devices implementing different SDPs cannot communicate with each other and share their functionalities and resources in a managed way, especially when they use different network technologies. As a solution to the first problem, we propose a new proxy server architecture that enables IP-based devices and services to be discovered on non-IP based network and vice versa. We implemented the proxy architecture using UPnP respectively Bluetooth SDP as IP- and non-IP-based SDPs. The proxy allows Bluetooth devices and UPnP control points to discover, access, and utilize services located on the other network. Validation experiments with the proxy prototype showed that seamless inter-working can be achieved keeping all proxy functionalities on a single device, thus not requiring modification of currently existing UPnP and Bluetooth end devices. Although the proxy itself taxes the end-to-end performance of the service, it is shown to be still acceptable for an end user. For mitigating resource conflicts in SDPs, we propose a generic resource reservation scheme with properties derived from common SDP operation. Performance studies with a prototype showed that this reservation scheme significantly improves the scalability and sustainability of service access in SDPs, at a minor computational cost. With regard to the second area, it is known that the end-to-end quality of Internet services depends crucially on the performance of the home network. Consequently, service providers require the ability to monitor and configure devices in the home network, behind the home gateway (HG). However, they can only put limited requirements to these off-the-shelf devices, as the consumer electronics market is largely outside their span of control. Therefore they have to make intelligent use of the given device control and management protocols. In this work, we propose an architecture for remote discovery and management of devices in a highly heterogeneous home network. A proof-of-concept is developed for the remote management of UPnP devices in the home with a TR-069/UPnP proxy on the HG. Although this architecture is protocol specific, it can be easily adapted to other web-services based protocols. Service providers are also asking for diagnostic tools with which they can remotely troubleshoot the home networks. One of these tools should be able to gather information about the topology of the home network. Although topology discovery protocols already exist, nothing is known yet about their performance. In this work we propose a set of key performance indicators for home network topology discovery architectures, and how they should be measured. We applied them to the Link-Layer Topology Discovery (LLTD) protocol and the Link-Layer Discovery Protocol (LLDP). Our performance measurement results show that these protocols do not fulfill all the requirements as formulated by the service providers. With regard to the third area, current QoS solutions are mostly based on traffic classification. Because they need to be supported by all devices in the network, they are relatively expensive for home networks. Furthermore, they are not interoperable between different networking technologies. Alternative QoS provision techniques have been proposed in the literature. These techniques require end-user services to pragmatically adapt their properties to the actual condition of the network. For this, the condition of the home network in terms of its available bandwidth, delay, jitter, etc., needs to be known in real time. Appropriate tools for determining the available home network resources do not yet exist. In this work we propose a new method to probe the path capacity and available bandwidth between a server and a client in a home network. The main features of this method are: (a) it does not require adaptation of existing end devices, (b) it does not require pre-knowledge of the link-layer network topology, and (c) it is accurate enough to make reliable QoS predictions for the most relevant home applications. To use these predictions for effective service- or content-adaptation or admission control, one should also know how the state of the home network is expected to change immediately after the current state has been probed. However, not much is known about the stochastic properties of traffic in home networks. Based on a relatively small set of traffic observations in several home networks in the Netherlands, we were able to build a preliminary model for home network traffic dynamics

Deliverable D8.2 First market analysis

This deliverable provides an overview of a first market analysis of the IPTV market. It points out possible customers, competitors and the differences between LinkedTV and their competitive firms

Converged digital TV services: the role of middleware and future directions of interactive television

The subject of the future of the interactive Television medium has become a topic of great interest to the academic and industrial communities particularly since in the recent years there has been a dramatic increase in the pace of innovation of convergence of digital TV systems and services. The purpose of this paper is to provide a brief overview of what we know as digital TV converged services, to present and categorise the digital Television middleware technologies that contributed to it, and to present possible future trends and directions. A new Television era of converged wireless and mobile content delivery, user-authored content, multimodal interaction, intelligent personalisation, smart space awareness, and 3D content sensations is foreseen, creating ambient and immersive experiences