Semi-Automation in Video Editing

Semi-automasjon i video redigering Hvordan kan vi bruke kunstig intelligens (KI) og maskin læring til å gjøre videoredigering like enkelt som å redigere tekst? I denne avhandlingen vil jeg adressere problemet med å bruke KI i videoredigering fra et Menneskelig-KI interaksjons perspektiv, med fokus på å bruke KI til å støtte brukerne. Video er et audiovisuelt medium. Redigere videoer krever synkronisering av både det visuelle og det auditive med presise operasjoner helt ned på millisekund nivå. Å gjøre dette like enkelt som å redigere tekst er kanskje ikke mulig i dag. Men hvordan skal vi da støtte brukerne med KI og hva er utfordringene med å gjøre det? Det er fem hovedspørsmål som har drevet forskningen i denne avhandlingen. Hva er dagens "state-of-the-art" i KI støttet videoredigering? Hva er behovene og forventningene av fagfolkene om KI? Hva er påvirkningen KI har på effektiviteten og nøyaktigheten når det blir brukt på teksting? Hva er endringene i brukeropplevelsen når det blir brukt KI støttet teksting? Hvordan kan flere KI metoder bli brukt for å støtte beskjærings- og panoreringsoppgaver? Den første artikkelen av denne avhandlingen ga en syntese og kritisk gjennomgang av eksisterende arbeid med KI-baserte verktøy for videoredigering. Artikkelen ga også noen svar på hvordan og hva KI kan bli brukt til for å støtte brukere ved en undersøkelse utført av 14 fagfolk. Den andre studien presenterte en prototype av KI-støttet videoredigerings verktøy bygget på et eksisterende videoproduksjons program. I tillegg kom det en evaluasjon av både ytelse og brukeropplevelse på en KI-støttet teksting fra 24 nybegynnere. Den tredje studien beskrev et idiom-basert verktøy for å konvertere bredskjermsvideoer lagd for TV til smalere størrelsesforhold for mobil og sosiale medieplattformer. Den tredje studien utforsker også nye metoder for å utøve beskjæring og panorering ved å bruke fem forskjellige KI-modeller. Det ble også presentert en evaluering fra fem brukere. I denne avhandlingen brukte vi en brukeropplevelse og oppgave basert framgangsmåte, for å adressere det semi-automatiske i videoredigering.How can we use artificial intelligence (AI) and machine learning (ML) to make video editing as easy as "editing text''? In this thesis, this problem of using AI to support video editing is explored from the human--AI interaction perspective, with the emphasis on using AI to support users. Video is a dual-track medium with audio and visual tracks. Editing videos requires synchronization of these two tracks and precise operations at milliseconds. Making it as easy as editing text might not be currently possible. Then how should we support the users with AI, and what are the current challenges in doing so? There are five key questions that drove the research in this thesis. What is the start of the art in using AI to support video editing? What are the needs and expectations of video professionals from AI? What are the impacts on efficiency and accuracy of subtitles when AI is used to support subtitling? What are the changes in user experience brought on by AI-assisted subtitling? How can multiple AI methods be used to support cropping and panning task? In this thesis, we employed a user experience focused and task-based approach to address the semi-automation in video editing. The first paper of this thesis provided a synthesis and critical review of the existing work on AI-based tools for videos editing and provided some answers to how should and what more AI can be used in supporting users by a survey of 14 video professional. The second paper presented a prototype of AI-assisted subtitling built on a production grade video editing software. It is the first comparative evaluation of both performance and user experience of AI-assisted subtitling with 24 novice users. The third work described an idiom-based tool for converting wide screen videos made for television to narrower aspect ratios for mobile social media platforms. It explores a new method to perform cropping and panning using five AI models, and an evaluation with 5 users and a review with a professional video editor were presented.Doktorgradsavhandlin

Diversity and public television: analysis of subtitling as an accessibility service

The present article addresses media pluralism from a critical perspective, as a guarantor of attention to social and cultural diversity. The study focuses on deaf individuals, and an analysis is conducted of subtitles as one of the services that guarantees universal access to the contents offered by Spanish Radio and Television (Radiotelevisión Española, RTVE) for social and cultural diversity. Likewise, given the changes in audiovisual consumption in the form of streaming, and the rise of on-demand content, an analysis is not only performed of linear television, but also of live online and on-demand content. For this, a quantitative analysis was performed of a random sample of more than 5000 RTVE programs (2017-2018), to establish the number of subtitled programs for deaf individuals, their schedule, and the types of contents that included this service. Also, a qualitative analysis was performed on a sample of 72 programs distributed among the different channels belonging to this public corporation. Lastly, to evaluate the quality and suitability of the subtitles, an online survey was provided, which was completed by 183 deaf individuals who were users of the RTVE subtitles. Through this combination of methods, we were able to analyze and evaluate the cultural pluralism of RTVE through the subtitles provided for the deaf, associating it with the defense of the fundamental right of freedom of expression and social inclusion, values that must be guaranteed in democratic societies

User perspectives on developing technology-assisted access services in public broadcasting.

The growing demand for accessible media content in the Creative Industries and increased pressure to produce more content at lower costs has led the industry to look for technological support for creating and managing audiovisual content. In order to design truly accessible technology-assisted solutions and services, it is crucial to center the knowledge and experience of the intended users –both the consumers and the professionals involved in producing the content and services. This article explores potential technological solutions for audiovisual media access services addressing intralingual, interlingual and intersemiotic access in public service television broadcasting. We introduce an ongoing research project taking a user-centered approach, and present work carried out on automatic and semi-automatic methods involving intralingual and interlingual subtitling, and description of visual content. Based on themes raised in individual and focus group interviews, we discuss how potential user groups respondto technological solutions at differing levels of maturity. We examine conceptualizations of quality, trust, and accessibility emerging in the users’ comments, and chart the differences and similarities between different stakeholders. The article demonstrates how diverse user perspectives can inform research and development andenhance our understanding of the role of technology in promoting media accessibility.</p