Enhanced adaptive RTCP-based inter-destination multimedia synchronization approach for distributed applications

[EN] Newer social multimedia applications, such as Social TV or networked multi-player games, enable independent groups (or clusters) of users to interact among themselves and share services within the context of simultaneous media content consumption. In such scenarios, concurrently synchronized playout points must be ensured so as not to degrade the user experience on such interaction. We refer to this process as Inter-Destination Multimedia Synchronization (IDMS). This paper presents the design, implementation and evaluation of an evolved version of an RTCP-based IDMS approach, including an Adaptive Media Playout (AMP) scheme that aims to dynamically and smoothly adjust the playout timing of each one of the geographically distributed consumers in a specific cluster if an allowable asynchrony threshold between their playout states is exceeded. For that purpose, we previously had also to develop a full implementation of RTP/RTCP protocols for NS-2, in which we included the IDMS approach as an optional functionality. Simulation results prove the feasibility of such IDMS and AMP proposals, by adopting several dynamic master reference selection policies, to maintain an overall synchronization status (within allowable limits) in each cluster of participants, while minimizing the occurrence of long-term playout discontinuities (such as skips/pauses) which are subjectively more annoying and less tolerable to users than small variations in the media playout rate.This work has been financed, partially, by Universitat Politecnica de Valencia (UPV), under its R&D Support Program in PAID-05-11-002-331 Project and in PAID-01-10. Authors also would like to thank the anonymous reviewers that helped to significantly improve the quality of the paper with their constructive comments.Montagud, M.; Boronat, F. (2012). Enhanced adaptive RTCP-based inter-destination multimedia synchronization approach for distributed applications. Computer Networks. 56(12):2912-2933. https://doi.org/10.1016/j.comnet.2012.05.00329122933561

Implementation and Evaluation of an M/S Scheme for Inter-Destination Multimedia Synchronization (IDMS)

[EN] Nowadays, the media consumption model is changing from passive isolated activities towards dynamic group shared experiences. People separated in space can already interact and collaborate within the context of simultaneous media content delivery (e.g. they can launch a real time conversation during a shared video watching). However, to stimulate the acceptance and usability of those socially aware interactive media services, synchronous communications (i.e. consistent playout states) across separated locations must be guaranteed. Inter-Destination Multimedia Synchronization (IDMS) has become a key research topic to enable geographically dispersed consumers to experience a satisfying feeling of togetherness. Several (distributed and centralized) approaches to perform IDMS have been adopted by the research community up to date. In this paper, a Master/Slave (M/S) Control Scheme for IDMS is implemented and tested, thus analyzing its strengths and weaknesses for distributed networked environments. Simulation results prove the effectiveness and suitability of the proposed M/S Scheme in those scenarios in which scalability, consistency, interactivity and low control traffic overhead features must be provided.This work has been financed, partially, by Universitat Politècnica de Valencia (UPV), under its R&D Support Program in PAID-05-11-002-331 Project and in PAID-01-10.Montagud, M.; Boronat Segui, F. (2011). Implementation and Evaluation of an M/S Scheme for Inter-Destination Multimedia Synchronization (IDMS). Network Protocols and Algorithms. 3(3):80-98. https://doi.org/10.5296/npa.v3i3.1089S80983

Inter-Destination Multimedia Synchronization; Schemes, Use Cases and Standardization

Traditionally, the media consumption model has been a passive and isolated activity. However, the advent of media streaming technologies, interactive social applications, and synchronous communications, as well as the convergence between these three developments, point to an evolution towards dynamic shared media experiences. In this new model, geographically distributed groups of consumers, independently of their location and the nature of their end-devices, can be immersed in a common virtual networked environment in which they can share multimedia services, interact and collaborate in real-time within the context of simultaneous media content consumption. In most of these multimedia services and applications, apart from the well-known intra and inter-stream synchronization techniques that are important inside the consumers playout devices, also the synchronization of the playout processes between several distributed receivers, known as multipoint, group or Inter-destination multimedia synchronization (IDMS), becomes essential. Due to the increasing popularity of social networking, this type of multimedia synchronization has gained in popularity in recent years. Although Social TV is perhaps the most prominent use case in which IDMS is useful, in this paper we present up to 19 use cases for IDMS, each one having its own synchronization requirements. Different approaches used in the (recent) past by researchers to achieve IDMS are described and compared. As further proof of the significance of IDMS nowadays, relevant organizations (such as ETSI TISPAN and IETF AVTCORE Group) efforts on IDMS standardization (in which authors have been and are participating actively), defining architectures and protocols, are summarized.This work has been financed, partially, by Universitat Politecnica de Valencia (UPV), under its R&D Support Program in PAID-05-11-002-331 Project and in PAID-01-10, and by TNO, under its Future Internet Use Research & Innovation Program. The authors also want to thank Kevin Gross for providing some of the use cases included in Sect. 1.2.Montagud, M.; Boronat Segui, F.; Stokking, H.; Van Brandenburg, R. (2012). Inter-Destination Multimedia Synchronization; Schemes, Use Cases and Standardization. Multimedia Systems. 18(6):459-482. https://doi.org/10.1007/s00530-012-0278-9S459482186Kernchen, R., Meissner, S., Moessner, K., Cesar, P., Vaishnavi, I., Boussard, M., Hesselman, C.: Intelligent multimedia presentation in ubiquitous multidevice scenarios. IEEE Multimedia 17(2), 52–63 (2010)Vaishnavi, I., Cesar, P., Bulterman, D., Friedrich, O., Gunkel, S., Geerts, D.: From IPTV to synchronous shared experiences challenges in design: distributed media synchronization. Signal Process Image Commun 26(7), 370–377 (2011)Geerts, D., Vaishnavi, I., Mekuria, R., Van Deventer, O., Cesar, P.: Are we in sync?: synchronization requirements for watching on-line video together, CHI ‘11, New York, USA (2011)Boronat, F., Lloret, J., García, M.: Multimedia group and inter-stream synchronization techniques: a comparative study. Inf. Syst. 34(1), 108–131 (2009)Chen, M.: A low-latency lip-synchronized videoconferencing system. In: SIGCHI Conference on Human Factors in Computing Systems, CHI’03, ACM, pp. 464–471, New York (2003)Ishibashi, Y., Tasaka, S., Ogawa, H.: Media synchronization quality of reactive control schemes. IEICE Trans. Commun. E86-B(10), 3103–3113 (2003)Ademoye, O.A., Ghinea, G.: Synchronization of olfaction-enhanced multimedia. IEEE Trans. Multimedia 11(3), 561–565 (2009)Cesar, P., Bulterman, D.C.A., Jansen, J., Geerts, D., Knoche, H., Seager, W.: Fragment, tag, enrich, and send: enhancing social sharing of video. ACM Trans. Multimedia Comput. Commun. Appl. 5(3), Article 19, 27 pages (2009)Van Deventer, M.O., Stokking, H., Niamut, O.A., Walraven, F.A., Klos, V.B.: Advanced Interactive Television Service Require Synchronization, IWSSIP 2008. Bratislava, June (2008)Premchaiswadi, W., Tungkasthan, A., Jongsawat, N.: Enhancing learning systems by using virtual interactive classrooms and web-based collaborative work. In: Proceedings of the IEEE Education Engineering Conference (EDUCON 2010), pp. 1531–1537. Madrid, Spain (2010)Diot, C., Gautier, L.: A distributed architecture for multiplayer interactive applications on the internet. IEEE Netw 13(4), 6–15 (1999)Mauve, M., Vogel, J., Hilt, V., Effelsberg, W.: Local-lag and timewarp: providing consistency for replicated continuous applications. IEEE Trans. Multimedia 6(1), 45–57 (2004)Hosoya, K., Ishibashi, Y., Sugawara, S., Psannis, K.E.: Group synchronization control considering difference of conversation roles. In: IEEE 13th International Symposium on Consumer Electronics, ISCE ‘09, pp. 948–952 (2009)Roccetti, M., Ferretti, S., Palazzi, C.: The brave new world of multiplayer online games: synchronization issues with smart solution. In: 11th IEEE Symposium on Object Oriented Real-Time Distributed Computing (ISORC), pp. 587–592 (2008)Ott, D.E., Mayer-Patel, K.: An open architecture for transport-level protocol coordination in distributed multimedia applications. ACM Trans. Multimedia Comput. Commun. Appl. 3(3), 17 (2007)Boronat, F., Montagud, M., Guerri, J.C.: Multimedia group synchronization approach for one-way cluster-to-cluster applications. In: IEEE 34th Conference on Local Computer Networks, LCN 2009, pp. 177–184, Zürich (2009)Boronat, F., Montagud, M., Vidal, V.: Smooth control of adaptive media playout to acquire IDMS in cluster-based applications. In: IEEE LCN 2011, pp. 617–625, Bonn (2011)Huang, Z., Wu, W., Nahrstedt, K., Rivas, R., Arefin, A.: SyncCast: synchronized dissemination in multi-site interactive 3D tele-immersion. In: Proceedings of MMSys, USA (2011)Kim, S.-J., Kuester, F., Kim, K.: A global timestamp-based approach for enhanced data consistency and fairness in collaborative virtual environments. ACM/Springer Multimedia Syst. J. 10(3), 220–229 (2005)Schooler, E.: Distributed music: a foray into networked performance. In: International Network Music Festival, Santa Monica, CA (1993)Miyashita, Y., Ishibashi, Y., Fukushima, N., Sugawara, S., Psannis K.E.: QoE assessment of group synchronization in networked chorus with voice and video. In: Proceedings of IEEE TENCON’11, pp. 393–397 (2011)Hesselman, C., Abbadessa, D., Van Der Beek, W., et al.: Sharing enriched multimedia experiences across heterogeneous network infrastructures. IEEE Commun. Mag. 48(6), 54–65 (2010)Montpetit, M., Klym, N., Mirlacher, T.: The future of IPTV—Connected, mobile, personal and social. Multimedia Tools Appl J 53(3), 519–532 (2011)Cesar, P., Bulterman, D.C.A., Jansen, J.: Leveraging the user impact: an architecture for secondary screens usage in an interactive television environment. ACM/Springer Multimedia Syst. 15(3), 127–142 (2009)Lukosch, S.: Transparent latecomer support for synchronous groupware. In: Proceedings of 9th International Workshop on Groupware (CRIWG), Grenoble, France, pp. 26–41 (2003)Steinmetz, R.: Human perception of jitter and media synchronization. IEEE J. Sel. Areas Commun. 14(1), 61–72 (1996)Stokking, H., Van Deventer, M.O., Niamut, O.A., Walraven, F.A., Mekuria, R.N.: IPTV inter-destination synchronization: a network-based approach, ICIN’2010, Berlin (2010)Mekuria, R.N.: Inter-destination media synchronization for TV broadcasts, Master Thesis, Faculty of Electrical Engineering, Mathematics and Computer Science, Department of Network architecture and Services, Delft University of Technology (2011)Pitt Ian, CS2511: Usability engineering lecture notes, localisation of sound sources. http://web.archive.org/web/20100410235208/http:/www.cs.ucc.ie/~ianp/CS2511/HAP.htmlNielsen, J.: Response times: the three important limits. http://www.useit.com/papers/responsetime.html (1994)ITU-T Rec G. 1010: End-User Multimedia QoS Categories. International Telecommunication Union, Geneva (2001)Biersack, E., Geyer, W.: Synchronized delivery and playout of distributed stored multimedia streams. ACM/Springer Multimedia Syst 7(1), 70–90 (1999)Xie, Y., Liu, C., Lee, M.J., Saadawi, T.N.: Adaptive multimedia synchronization in a teleconference system. ACM/Springer Multimedia Syst. 7(4), 326–337 (1999)Laoutaris, N., Stavrakakis, I.: Intrastream synchronization for continuous media streams: a survey of playout schedulers. IEEE Netw. Mag. 16(3), 30–40 (2002)Ishibashi, Y., Tsuji, A., Tasaka, S.: A group synchronization mechanism for stored media in multicast communications. In: Proceedings of the INFOCOM ‘97, Washington (1997)Ishibashi, Y., Tasaka, S.: A group synchronization mechanism for live media in multicast communications. IEEE GLOBECOM’97, pp. 746–752 (1997)Boronat, F., Guerri, J.C., Lloret, J.: An RTP/RTCP based approach for multimedia group and inter-stream synchronization. Multimedia Tools Appl. J. 40(2), 285–319 (2008)Ishibashi, I., Tasaka, S.: A distributed control scheme for group synchronization in multicast communications. In: Proceedings of International Symposium Communications, Kaohsiung, Taiwan, pp. 317–323 (1999)Lu, Y., Fallica, B., Kuipers, F.A., Kooij, R.E., Van Mieghem, P.: Assessing the quality of experience of SopCast. Int. J. Internet Protoc. Technol 4(1), 11–19 (2009)Shamma, D.A., Bastea-Forte, M., Joubert, N., Liu, Y.: Enhancing online personal connections through synchronized sharing of online video, ACM CHI’08 Extended Abstracts, Florence (2008)Ishibashi, Y., Tasaka, S.: A distributed control scheme for causality and media synchronization in networked multimedia games. In: Proceedings of 11th International Conference on Computer Communications and Networks, pp. 144–149, Miami, USA (2002)Ishibashi, Y., Tomaru, K., Tasaka, S., Inazumi, K.: Group synchronization in networked virtual environments. In: Proceedings of the 38th IEEE International Conference on Communications, pp. 885–890, Alaska, USA (2003)Tasaka, S., Ishibashi, Y., Hayashi, M.: Inter–destination synchronization quality in an integrated wired and wireless network with handover. IEEE GLOBECOM 2, 1560–1565 (2002)Kurokawa, Y., Ishibashi, Y., Asano, T.: Group synchronization control in a remote haptic drawing system. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 572–575, Beijing, China (2007)Hashimoto, T., Ishibashi, Y.: Group Synchronization Control over Haptic Media in a Networked Real-Time Game with Collaborative Work, Netgames’06, Singapore (2006)Nunome, T., Tasaka, S.: Inter-destination synchronization quality in a multicast mobile ad hoc network. In: Proceedings of IEEE 16th International Symposium on Personal, Indoor and Mobile Radio Communications, pp. 1366–1370, Berlin, Germany (2005)Brandenburg, R., van Stokking, H., Van Deventer, M.O., Boronat, F., Montagud, M., Gross, K.: RTCP for inter-destination media synchronization, draft-brandenburg-avtcore-rtcp-for-idms-03.txt. In: IETF Audio/Video Transport Core Maintenance Working Group, Internet Draft, March 9 (2012)ETSI TS 181 016 V3.3.1 (2009-07) Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); Service Layer Requirements to integrate NGN Services and IPTVETSI TS 182 027 V3.5.1 (2011-03) Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); IPTV Architecture; IPTV functions supported by the IMS subsystemETSI TS 183 063 V3.5.2 (2011-03) Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); IMS-based IPTV stage 3 specificationBrandenburg van, R., et al.: RTCP XR Block Type for inter-destination media synchronization, draft-brandenburg-avt-rtcp-for-idms-00.txt. In: IETF Audio/Video Transport Working Group, Internet Draft, Sept 24, 2010Williams, A., et al.: RTP Clock Source Signalling, draft-williams-avtcore-clksrc-00. In: IETF Audio/Video Transport Working Group, Internet Draft, February 28, 201

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Especificación, implementación y evaluación de soluciones de sincronización multimedia híbrida (broadcast/broadband) e inter-destinatario para posibilitar nuevas experiencias televisivas enriquecidas, personalizadas, inmersivas y compartidas.

[ES] Actualmente, existe una gran variedad de tecnologías de distribución de contenidos multimedia, de dispositivos de consumo, así como de tipos de contenidos. En cuanto a tecnologías de distribución, dichos contenidos se pueden enviar bien a través de redes broadcast, como las redes DVB, o bien a través de redes broadband, como las redes IP (por ejemplo, a través de Internet). La coordinación y convergencia entre los dos tipos de tecnologías para distribuir contenidos relacionados ofrece un gran potencial a los proveedores de contenidos, así como grandes beneficios a los consumidores (una prueba de ello es el estándar Hybrid Broadcast Broadband TV o HbbTV). Sin embargo, la recepción de contenidos relacionados procedentes de diferentes redes híbridas (broadcast/broadband) conlleva una variabilidad de retardos significativa, por lo que se deben superar una serie de retos importantes, como, por ejemplo, la necesidad de mecanismos que garanticen un consumo sincronizado de dichos contenidos, bien en un mismo dispositivo o bien en distintos dispositivos (locales o remotos). Se han especificado, implementado y evaluado soluciones de sincronización de los procesos de reproducción de contenidos híbridos (prevenientes de redes broadcast y/o broadband) tanto en entornos multidispositivo, entre dispositivos locales (Inter-Device Synchronization o IDES), como en entornos multiusuario, entre destinatarios distribuidos en red (Inter-Destination Media Synchronization o IDMS), incluyendo la sincronización de efectos multisensoriales. Se ha diseñado e implementado una plataforma extremo-a-extremo para la generación, distribución (adaptativa) y consumo de contenidos híbridos relacionados, tanto en un mismo dispositivo como en múltiples dispositivos locales (por ejemplo, escenario multipantalla) o remotos (por ejemplo, TV Social). Se han generado y preparado contenidos multimedia tradicionales (2D) y omnidireccionales (360), enriquecidos con efectos multisensoriales que mejoran el grado de realismo y la inmersividad de los usuarios. La plataforma es compatible con el estándar HbbTV y, además, incluye soluciones eficientes para aspectos clave que no están especificados en dicho estándar, pero que son necesarios para la implementación satisfactoria de servicios de TV híbridos y multidispositivo en la misma, como, por ejemplo: la necesidad de un buffer para el contenido broadcast (existente y opcional hasta la especificación HbbTV 2.0.2); mecanismos de señalización, asociación y descripción de contenido disponible y relacionado con el contenido principal; mecanismos de coordinación e interacción entre los dispositivos de consumo; o soluciones de sincronización para alinear en el tiempo el consumo de contenidos de forma satisfactoria. Los resultados obtenidos en la evaluación realizada muestran que el nivel de sincronización adquirido entre los procesos de reproducción de los diversos tipos de contenido disponibles a través de distintas tecnologías de transmisión alcanza unos valores suficientes para proporcionar al usuario un consumo con un nivel de asincronías imperceptible. Dicha precisión alcanzada ha resultado en una QoE para el consumidor muy satisfactoria, ofreciéndole al mismo un completo ecosistema de consumo de contenidos que incluye tanto contenido multimedia tradicional, como contenido omnidireccional y mulsemedia.[CA] Actualment, hi ha una gran varietat de tecnologies de distribució de contingut multimèdia, dispositius de consum, així com tipus de contingut. Pel que fa a les tecnologies de distribució, aquest contingut es pot lliurar a través de xarxes de difusió, com ara xarxes DVB, o a través de xarxes de banda ampla, com les xarxes IP (per exemple, a través d'Internet). La coordinació i convergència entre els dos tipus de tecnologies per distribuir continguts relacionats ofereix un gran potencial als proveïdors de continguts, així com grans beneficis per als consumidors (una prova d'això és l'estàndard Hybrid Broadcast Broadband TV o HbbTV). Tanmateix, la recepció de continguts relacionats des de diferents xarxes híbrides (emissió/banda ampla) comporta una important variabilitat de retard, per la qual cosa s'han de superar una sèrie de reptes importants, com la necessitat de mecanismes que garanteixin el consum sincronitzat d'aquests continguts, ja siga en el mateix dispositiu. o en diferents dispositius (locals o remots). S'han especificat, implementat i evaluat solucions de sincronització per a processos de reproducció de continguts híbrids (xarxes de broadcast i/o de banda ampla) tant en entorns multidispositiu, entre dispositius locals (Inter-Device Synchronization o IDES), com en entorns IDMS (Inter-Destination Media Synchronization o IDMS), incloent la sincronització d'efectes multisensorials. S'ha dissenyat i implementat una plataforma d'extrem a extrem per a la generació, distribució (adaptativa) i consum de contingut híbrid relacionat, tant en un sol dispositiu com en múltiples locals (p. ex., escenari multipantalla) o remots (p. ex., Social TV). dispositius. S'han generat i preparat contingut multimèdia tradicional (2D) i omnidireccional (360), enriquit amb efectes multisensorials que milloren el grau de realisme i immersivitat dels usuaris. La plataforma és compatible amb l'estàndard HbbTV i, a més, inclou solucions eficients per a aspectes clau que no s'especifiquen en aquest estàndard, però que són necessaris per a la implantació amb èxit de serveis de televisió híbrids i multidispositiu en el mateix, com ara: la necessitat per a un buffer per a continguts de difusió (existent i opcional fins a HbbTV 2. 0.2); mecanismes de senyalització, associació i descripció dels continguts disponibles i relacionats amb el contingut principal; mecanismes de coordinació i interacció entre dispositius de consum; o solucions de sincronització per alinear en el temps el consum de continguts de manera satisfactòria. Els resultats obtinguts en l'avaluació realitzada mostren que el nivell de sincronització adquirit entre els processos de reproducció dels diferents tipus de continguts disponibles a través de diferents tecnologies de transmissió assoleix valors suficients per proporcionar a l'usuari un consum amb un nivell d'asincronies imperceptibles. La precisió aconseguida ha donat com a resultat una QoE molt satisfactòria per al consumidor, oferint-li un ecosistema complet de consum de continguts que inclou tant contingut multimèdia tradicional com contingut omnidireccional i mulsemedia.[EN] Currently, there is a wide variety of multimedia content distribution technologies, consumer devices, as well as types of content. In terms of distribution technologies, such content can be delivered either over broadcast networks, such as DVB networks, or over broadband networks, such as IP networks (e.g. over the Internet). The coordination and convergence between the two types of technologies to distribute related content offers great potential to content providers, as well as great benefits to consumers (a proof of this is the Hybrid Broadcast Broadband TV or HbbTV standard). However, the reception of related content from different hybrid networks (broadcast/broadband) entails significant delay variability, so a number of important challenges must be overcome, such as the need for mechanisms to ensure synchronized consumption of such content, either on the same device or on different devices (local or remote). We have specified, implemented and evaluated synchronization solutions for hybrid content playback processes (broadcast and/or broadband networks) both in multi-device environments, between local devices (Inter-Device Synchronization or IDES), and in multi-user environments, between distributed network recipients (Inter-Destination Media Synchronization or IDMS), including the synchronization of multi-sensory effects. An end-to-end platform has been designed and implemented for the generation, distribution (adaptive) and consumption of related hybrid content, both on a single device and on multiple local (e.g., multiscreen scenario) or remote (e.g., Social TV) devices. Traditional (2D) and omnidirectional (360) multimedia content has been generated and prepared, enriched with multisensory effects that enhance the degree of realism and immersiveness for users. The platform is compatible with the HbbTV standard and, in addition, includes efficient solutions for key aspects that are not specified in that standard, but are necessary for the successful implementation of hybrid and multi-device TV services on it, such as: the need for a buffer for broadcast content (existing and optional up to the HbbTV 2. 0.2); mechanisms for signaling, association and description of content available and related to the main content; coordination and interaction mechanisms between consumption devices; or synchronization solutions to align in time the consumption of content in a satisfactory way. The results obtained in the evaluation carried out show that the level of synchronization acquired between the reproduction processes of the different types of content available through different transmission technologies reaches sufficient values to provide the user with a consumption with an imperceptible level of asynchronies. The accuracy achieved has resulted in a very satisfactory QoE for the consumer, offering him a complete content consumption ecosystem that includes traditional multimedia content as well as omnidirectional and mulsemedia content.Marfil Reguero, D. (2022). Especificación, implementación y evaluación de soluciones de sincronización multimedia híbrida (broadcast/broadband) e inter-destinatario para posibilitar nuevas experiencias televisivas enriquecidas, personalizadas, inmersivas y compartidas [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/18574

Model-driven Personalisation of Human-Computer Interaction across Ubiquitous Computing Applications

Personalisation is essential to Ubiquitous Computing (Ubicomp), which focuses on a human-centred paradigm aiming to provide interaction with adaptive content, services, and interfaces towards each one of its users, according to the context of the applications’ scenarios. However, the provision of that appropriated personalised interaction is a true challenge due to different reasons, such as the user interests, heterogeneous environments and devices, dynamic user behaviour and data capture. This dissertation focuses on a model-driven personalisation solution that has the main goal of facili-tating the implementation of a personalised human-computer interaction across different Ubicomp scenarios and applications. The research reported here investigates how a generic and interoperable model for personalisation can be used, shared and processed by different applications, among diverse devices, and across different scenarios, studying how it can enrich human-computer interaction. The research started by the definition of a consistent user model with the integration of context to end in a pervasive model for the definition of personalisations across different applications. Besides the model proposal, the other key contributions within the solution are the modelling frame-work, which encapsulates the model and integrates the user profiling module, and a cloud-based platform to pervasively support developers in the implementation of personalisation across different applications and scenarios. This platform provides tools to put end users in control of their data and to support developers through web services based operations implemented on top of a personalisa-tion API, which can also be used independently of the platform for testing purposes, for instance. Several Ubicomp applications prototypes were designed and used to evaluate, at different phases, both the solution as a whole and each one of its components. Some were specially created with the goal of evaluating specific research questions of this work. Others were being developed with a pur-pose other than for personalisation evaluation, but they ended up as personalised prototypes to better address their initial goals. The process of applying the personalisation model to the design of the latter should also work as a proof of concept on the developer side. On the one hand, developers have been probed with the implementation of personalised applications using the proposed solution, or a part of it, to assess how it works and can help them. The usage of our solution by developers was also important to assess how the model and the platform respond to the developers’ needs. On the other hand, some prototypes that implement our model-driven per-sonalisation solution have been selected for end user evaluation. Usually, user testing was conducted at two different stages of the development, using: (1) a non-personalised version; (2) the final per-sonalised version. This procedure allowed us to assess if personalisation improved the human-com-puter interaction. The first stage was also important to know who were the end users and gather interaction data to come up with personalisation proposals for each prototype. Globally, the results of both developers and end users tests were very positive. Finally, this dissertation proposes further work, which is already ongoing, related to the study of a methodology to the implementation and evaluation of personalised applications, supported by the development of three mobile health applications for rehabilitation