Web-based Platform for Subtitles Customization and Synchronization in Multi-Screen Scenarios

[EN] This paper presents a web-based platform that enables the customization and synchronization of subtitles on both single- and multi-screen scenarios. The platform enables the dynamic customization of the subtitles format (font family, size, color...) and position according to the users preferences and/or needs. Likewise, it allows configuring the number of subtitles lines to be presented, being able to restore the video playout position by clicking on a specific one. It also allows the simultaneous selection of various subtitle languages, and applying a delay offset to the presentation of subtitles. All these functionalities can also be available on (personal) companion devices, allowing the presentation of subtitles in a synchronized manner with the ones on the main screen and their individual customization. With all these functionalities, the platform enables personalized and immersive media consumption experiences, contributing to a better language learning, social integration and an improved Quality of Experience (QoE) in both domestic and multi-culture environments.This work has been funded, partially, by the Fondo Europeo de Desarrollo Regional (FEDER) and the Spanish Ministry of Economy and Competitiveness, under its R&D&I Support Program, in project with Ref. TEC2013- 45492-R, and by Fundación Española para la Ciencia y Tecnología (FECYT) , in project with Ref. FCT-15-9579Montagud, M.; Boronat, F.; González, J.; Pastor, J. (2017). Web-based Platform for Subtitles Customization and Synchronization in Multi-Screen Scenarios. ACM Digital Library. 81-82. https://doi.org/10.1145/3084289.3089913S8182M. Armstrong, et al., "Understanding the Diverse Needs of Subtitle Users in a Rapidly Evolving Media Landscape", IBC 2015, Amsterdam, September 2015.A. Brown, et al, "Dynamic Subtitles: The User Experience". ACM TVX '15, pp. 103--112, Brussels (Belgium), June 2015.Y. Hu, et al. "Speaker-Following Video Subtitles", TOMCCAP, 11, 2, Article 32, 17 pages, January 2015.M. Montagud, et al., "Use of Web Components to Develop Interactive, Customizable and Multi-Device Video Consumption Platforms", Applications and Usability of Interactive TV, Springer, V.605, 2016.A. Rodriguez, et al., "Subtitle Synchronization across Multiple Screens and Devices", Sensors, 12(7), 87108731, June 2012

Sub-Sync: automatic synchronization of subtitles in the broadcasting of true live programs in spanish

Individuals With Sensory Impairment (Hearing Or Visual) Encounter Serious Communication Barriers Within Society And The World Around Them. These Barriers Hinder The Communication Process And Make Access To Information An Obstacle They Must Overcome On A Daily Basis. In This Context, One Of The Most Common Complaints Made By The Television (Tv) Users With Sensory Impairment Is The Lack Of Synchronism Between Audio And Subtitles In Some Types Of Programs. In Addition, Synchronization Remains One Of The Most Significant Factors In Audience Perception Of Quality In Live-Originated Tv Subtitles For The Deaf And Hard Of Hearing. This Paper Introduces The Sub-Sync Framework Intended For Use In Automatic Synchronization Of Audio-Visual Contents And Subtitles, Taking Advantage Of Current Well-Known Techniques Used In Symbol Sequences Alignment. In This Particular Case, These Symbol Sequences Are The Subtitles Produced By The Broadcaster Subtitling System And The Word Flow Generated By An Automatic Speech Recognizing The Procedure. The Goal Of Sub-Sync Is To Address The Lack Of Synchronism That Occurs In The Subtitles When Produced During The Broadcast Of Live Tv Programs Or Other Programs That Have Some Improvised Parts. Furthermore, It Also Aims To Resolve The Problematic Interphase Of Synchronized And Unsynchronized Parts Of Mixed Type Programs. In Addition, The Framework Is Able To Synchronize The Subtitles Even When They Do Not Correspond Literally To The Original Audio And/Or The Audio Cannot Be Completely Transcribed By An Automatic Process. Sub-Sync Has Been Successfully Tested In Different Live Broadcasts, Including Mixed Programs, In Which The Synchronized Parts (Recorded, Scripted) Are Interspersed With Desynchronized (Improvised) Ones