A customizable open-source framework for measuring and equalizing e2e delays in shared video watching

Low-latency and media sync are essential requirements to enable interactive multi-party services, such as Social TV. In this work, we present an open-source and customizable framework that allows measuring end-to-end (e2e) video delays and provides support for different type

A customizable open-source framework for measuring and equalizing e2e delays in shared video watching

[EN] Low-latency and media sync are essential requirements to enable interactive multi-party services, such as Social TV. In this work, we present an open-source and customizable framework that allows measuring end-to-end (e2e) video delays and provides support for different types of media sync, including Inter-Destination Media Sync (IDMS). This framework can be used by researchers to investigate the suitability of different techniques for optimizing the system performance in terms of e2e delays and media sync.This work has been funded, partially, by UPV under its R&D Support Program in PAID-01-10 Project and by CWI under EU/FP7 REVERIE Project (ICT-2011-7-287723).Montagud, M.; Boronat, F.; Cesar, P. (2014). A customizable open-source framework for measuring and equalizing e2e delays in shared video watching. ACM. 95-96. https://doi.org/10.6084/m9.figshare.1032656S959

An Internet-Based Real-Time Audiovisual Link for Dual MEG Recordings

Hyperscanning Most neuroimaging studies of human social cognition have focused on brain activity of single subjects. More recently, "two-person neuroimaging" has been introduced, with simultaneous recordings of brain signals from two subjects involved in social interaction. These simultaneous "hyperscanning" recordings have already been carried out with a spectrum of neuroimaging modalities, such as functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and functional near-infrared spectroscopy (fNIRS). Dual MEG Setup We have recently developed a setup for simultaneous magnetoencephalographic (MEG) recordings of two subjects that communicate in real time over an audio link between two geographically separated MEG laboratories. Here we present an extended version of the setup, where we have added a video connection and replaced the telephone-landline-based link with an Internet connection. Our setup enabled transmission of video and audio streams between the sites with a one-way communication latency of about 130 ms. Our software that allows reproducing the setup is publicly available. Validation We demonstrate that the audiovisual Internet-based link can mediate real-time interaction between two subjects who try to mirror each others' hand movements that they can see via the video link. All the nine pairs were able to synchronize their behavior. In addition to the video, we captured the subjects' movements with accelerometers attached to their index fingers; we determined from these signals that the average synchronization accuracy was 215 ms. In one subject pair we demonstrate inter-subject coherence patterns of the MEG signals that peak over the sensorimotor areas contralateral to the hand used in the task.Peer reviewe

Impacto de Parámetros de QoS en Aspectos de QoE: Análisis desde el Punto de Vista de la Sincronización Multimedia

La sincronización multimedia ha sido un área de investigación clave desde los inicios de los sistemas multimedia. En este artículo se ofrecen una visión general y un análisis sobre el impacto de varios parámetros de QoS en diferentes aspectos de la QoE, desde el punto de vista de la sincronización multimedia. En primer lugar, se presentan los diferentes tipos de sincronización multimedia y su relevancia para garantizar una QoE satisfactoria. En segundo lugar, se muestra que la magnitud de los retardos y su variabilidad en las redes actuales es bastante superior a los umbrales permisibles por los usuarios en diferentes tipos y ejemplos de sincronización multimedia, reflejando así su necesidad. En tercer lugar, se describe el impacto del ancho de banda y la tasa de pérdidas sobre la sincronización multimedia. Por último, se argumenta la influencia del uso de diferentes alternativas para conseguir la sincronización multimedia sobre varios factores de QoS y de QoE

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Turn-Taking in Human Communicative Interaction

The core use of language is in face-to-face conversation. This is characterized by rapid turn-taking. This turn-taking poses a number central puzzles for the psychology of language. Consider, for example, that in large corpora the gap between turns is on the order of 100 to 300 ms, but the latencies involved in language production require minimally between 600ms (for a single word) or 1500 ms (for as simple sentence). This implies that participants in conversation are predicting the ends of the incoming turn and preparing in advance. But how is this done? What aspects of this prediction are done when? What happens when the prediction is wrong? What stops participants coming in too early? If the system is running on prediction, why is there consistently a mode of 100 to 300 ms in response time? The timing puzzle raises further puzzles: it seems that comprehension must run parallel with the preparation for production, but it has been presumed that there are strict cognitive limitations on more than one central process running at a time. How is this bottleneck overcome? Far from being 'easy' as some psychologists have suggested, conversation may be one of the most demanding cognitive tasks in our everyday lives. Further questions naturally arise: how do children learn to master this demanding task, and what is the developmental trajectory in this domain? Research shows that aspects of turn-taking such as its timing are remarkably stable across languages and cultures, but the word order of languages varies enormously. How then does prediction of the incoming turn work when the verb (often the informational nugget in a clause) is at the end? Conversely, how can production work fast enough in languages that have the verb at the beginning, thereby requiring early planning of the whole clause? What happens when one changes modality, as in sign languages -- with the loss of channel constraints is turn-taking much freer? And what about face-to-face communication amongst hearing individuals -- do gestures, gaze, and other body behaviors facilitate turn-taking? One can also ask the phylogenetic question: how did such a system evolve? There seem to be parallels (analogies) in duetting bird species, and in a variety of monkey species, but there is little evidence of anything like this among the great apes. All this constitutes a neglected set of problems at the heart of the psychology of language and of the language sciences. This research topic welcomes contributions from right across the board, for example from psycholinguists, developmental psychologists, students of dialogue and conversation analysis, linguists interested in the use of language, phoneticians, corpus analysts and comparative ethologists or psychologists. We welcome contributions of all sorts, for example original research papers, opinion pieces, and reviews of work in subfields that may not be fully understood in other subfields

User-centric video delay measurements

The complexities and physical constraints associated with video transmission make the introduction of video playout delays unavoidable. Tuning systems to reduce delay requires an ability to effectively and easily gather delay metrics on a potentially wide range of systems. In order to support this process, we report on a system called videoLat. VideoLat provides an innovative approach to understand glass-to-glass video delays. This paper provides a series of requirements for obtaining representative delay information, it illustrates how such measurements can provide insights into complex (and often closed) video processing systems, and it describes how user-centric testing can be supported in a more realistic manner. We also survey the present state of the art in video delay measurement. The main contribution of this work is that it provides a measuring framework that could serve as the basis for obtaining representative comparative measurements across a wide range of video processing environments