Perceived synchronization of mulsemedia services

Multimedia synchronization involves a temporal relationship between audio and visual media components. The presentation of "in-sync" data streams is essential to achieve a natural impression, as "out-of-sync" effects are often associated with user quality of experience (QoE) decrease. Recently, multi-sensory media (mulsemedia) has been demonstrated to provide a highly immersive experience for its users. Unlike traditional multimedia, mulsemedia consists of other media types (i.e., haptic, olfaction, taste, etc.) in addition to audio and visual content. Therefore, the goal of achieving high quality mulsemedia transmission is to present no or little synchronization errors between the multiple media components. In order to achieve this ideal synchronization, there is a need for comprehensive knowledge of the synchronization requirements at the user interface. This paper presents the results of a subjective study carried out to explore the temporal boundaries within which haptic and air-flow media objects can be successfully synchronized with video media. Results show that skews between sensorial media and multimedia might still give the effect that the mulsemedia sequence is "in-sync" and provide certain constraints under which synchronization errors might be tolerated. The outcomes of the paper are used to provide recommendations for mulsemedia service providers in order for their services to be associated with acceptable user experience levels, e.g. haptic media could be presented with a delay of up to 1 s behind video content, while air-flow media could be released either 5 s ahead of or 3 s behind video content

Digital recording of performing arts: formats and conversion

B

Temporal verification in secure group communication system design

The paper discusses an experience in using a real-time UML/SysML profile and a formal verification toolkit to check a secure group communication system against temporal requirements. A generic framework is proposed and specialized for hierarchical groups

Review of Media Sync Reference Models: Advances and Open Issues

The advances on multimedia systems have brought new challenges and requirements for media sync. Over the years, many media sync solutions have been devised. Due to this variety, several studies have surveyed the existing solutions and proposed classification schemes or reference models for media sync. This paper claims the relevance of media sync reference models to systematically structure and synthesize this research area. Accordingly, a review of the existing reference models is provided, by examining the involved features, components and layers in each one of them. Likewise, some inconsistencies, open issues and missing components in existing reference models have been identified. Accordingly, this study reflects the need for a new modular and extensible theoretical framework or reference model to efficiently comprehend the overall media sync research area

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Managed Forgetting to Support Information Management and Knowledge Work

Trends like digital transformation even intensify the already overwhelming mass of information knowledge workers face in their daily life. To counter this, we have been investigating knowledge work and information management support measures inspired by human forgetting. In this paper, we give an overview of solutions we have found during the last five years as well as challenges that still need to be tackled. Additionally, we share experiences gained with the prototype of a first forgetful information system used 24/7 in our daily work for the last three years. We also address the untapped potential of more explicated user context as well as features inspired by Memory Inhibition, which is our current focus of research.Comment: 10 pages, 2 figures, preprint, final version to appear in KI - K\"unstliche Intelligenz, Special Issue: Intentional Forgettin

Collective phase description of oscillatory convection

We formulate a theory for the collective phase description of oscillatory convection in Hele-Shaw cells. It enables us to describe the dynamics of the oscillatory convection by a single degree of freedom which we call the collective phase. The theory can be considered as a phase reduction method for limit-cycle solutions in infinite-dimensional dynamical systems, namely, stable time-periodic solutions to partial differential equations, representing the oscillatory convection. We derive the phase sensitivity function, which quantifies the phase response of the oscillatory convection to weak perturbations applied at each spatial point, and analyze the phase synchronization between two weakly coupled Hele-Shaw cells exhibiting oscillatory convection on the basis of the derived phase equations.Comment: 16 pages, 4 figures, to appear in Chao

Enhanced adaptive RTCP-based inter-destination multimedia synchronization approach for distributed applications

[EN] Newer social multimedia applications, such as Social TV or networked multi-player games, enable independent groups (or clusters) of users to interact among themselves and share services within the context of simultaneous media content consumption. In such scenarios, concurrently synchronized playout points must be ensured so as not to degrade the user experience on such interaction. We refer to this process as Inter-Destination Multimedia Synchronization (IDMS). This paper presents the design, implementation and evaluation of an evolved version of an RTCP-based IDMS approach, including an Adaptive Media Playout (AMP) scheme that aims to dynamically and smoothly adjust the playout timing of each one of the geographically distributed consumers in a specific cluster if an allowable asynchrony threshold between their playout states is exceeded. For that purpose, we previously had also to develop a full implementation of RTP/RTCP protocols for NS-2, in which we included the IDMS approach as an optional functionality. Simulation results prove the feasibility of such IDMS and AMP proposals, by adopting several dynamic master reference selection policies, to maintain an overall synchronization status (within allowable limits) in each cluster of participants, while minimizing the occurrence of long-term playout discontinuities (such as skips/pauses) which are subjectively more annoying and less tolerable to users than small variations in the media playout rate.This work has been financed, partially, by Universitat Politecnica de Valencia (UPV), under its R&D Support Program in PAID-05-11-002-331 Project and in PAID-01-10. Authors also would like to thank the anonymous reviewers that helped to significantly improve the quality of the paper with their constructive comments.Montagud, M.; Boronat, F. (2012). Enhanced adaptive RTCP-based inter-destination multimedia synchronization approach for distributed applications. Computer Networks. 56(12):2912-2933. https://doi.org/10.1016/j.comnet.2012.05.00329122933561

Self-Supervised Vision-Based Detection of the Active Speaker as Support for Socially-Aware Language Acquisition

This paper presents a self-supervised method for visual detection of the active speaker in a multi-person spoken interaction scenario. Active speaker detection is a fundamental prerequisite for any artificial cognitive system attempting to acquire language in social settings. The proposed method is intended to complement the acoustic detection of the active speaker, thus improving the system robustness in noisy conditions. The method can detect an arbitrary number of possibly overlapping active speakers based exclusively on visual information about their face. Furthermore, the method does not rely on external annotations, thus complying with cognitive development. Instead, the method uses information from the auditory modality to support learning in the visual domain. This paper reports an extensive evaluation of the proposed method using a large multi-person face-to-face interaction dataset. The results show good performance in a speaker dependent setting. However, in a speaker independent setting the proposed method yields a significantly lower performance. We believe that the proposed method represents an essential component of any artificial cognitive system or robotic platform engaging in social interactions.Comment: 10 pages, IEEE Transactions on Cognitive and Developmental System