Semi-Automation in Video Editing

Semi-automasjon i video redigering Hvordan kan vi bruke kunstig intelligens (KI) og maskin læring til å gjøre videoredigering like enkelt som å redigere tekst? I denne avhandlingen vil jeg adressere problemet med å bruke KI i videoredigering fra et Menneskelig-KI interaksjons perspektiv, med fokus på å bruke KI til å støtte brukerne. Video er et audiovisuelt medium. Redigere videoer krever synkronisering av både det visuelle og det auditive med presise operasjoner helt ned på millisekund nivå. Å gjøre dette like enkelt som å redigere tekst er kanskje ikke mulig i dag. Men hvordan skal vi da støtte brukerne med KI og hva er utfordringene med å gjøre det? Det er fem hovedspørsmål som har drevet forskningen i denne avhandlingen. Hva er dagens "state-of-the-art" i KI støttet videoredigering? Hva er behovene og forventningene av fagfolkene om KI? Hva er påvirkningen KI har på effektiviteten og nøyaktigheten når det blir brukt på teksting? Hva er endringene i brukeropplevelsen når det blir brukt KI støttet teksting? Hvordan kan flere KI metoder bli brukt for å støtte beskjærings- og panoreringsoppgaver? Den første artikkelen av denne avhandlingen ga en syntese og kritisk gjennomgang av eksisterende arbeid med KI-baserte verktøy for videoredigering. Artikkelen ga også noen svar på hvordan og hva KI kan bli brukt til for å støtte brukere ved en undersøkelse utført av 14 fagfolk. Den andre studien presenterte en prototype av KI-støttet videoredigerings verktøy bygget på et eksisterende videoproduksjons program. I tillegg kom det en evaluasjon av både ytelse og brukeropplevelse på en KI-støttet teksting fra 24 nybegynnere. Den tredje studien beskrev et idiom-basert verktøy for å konvertere bredskjermsvideoer lagd for TV til smalere størrelsesforhold for mobil og sosiale medieplattformer. Den tredje studien utforsker også nye metoder for å utøve beskjæring og panorering ved å bruke fem forskjellige KI-modeller. Det ble også presentert en evaluering fra fem brukere. I denne avhandlingen brukte vi en brukeropplevelse og oppgave basert framgangsmåte, for å adressere det semi-automatiske i videoredigering.How can we use artificial intelligence (AI) and machine learning (ML) to make video editing as easy as "editing text''? In this thesis, this problem of using AI to support video editing is explored from the human--AI interaction perspective, with the emphasis on using AI to support users. Video is a dual-track medium with audio and visual tracks. Editing videos requires synchronization of these two tracks and precise operations at milliseconds. Making it as easy as editing text might not be currently possible. Then how should we support the users with AI, and what are the current challenges in doing so? There are five key questions that drove the research in this thesis. What is the start of the art in using AI to support video editing? What are the needs and expectations of video professionals from AI? What are the impacts on efficiency and accuracy of subtitles when AI is used to support subtitling? What are the changes in user experience brought on by AI-assisted subtitling? How can multiple AI methods be used to support cropping and panning task? In this thesis, we employed a user experience focused and task-based approach to address the semi-automation in video editing. The first paper of this thesis provided a synthesis and critical review of the existing work on AI-based tools for videos editing and provided some answers to how should and what more AI can be used in supporting users by a survey of 14 video professional. The second paper presented a prototype of AI-assisted subtitling built on a production grade video editing software. It is the first comparative evaluation of both performance and user experience of AI-assisted subtitling with 24 novice users. The third work described an idiom-based tool for converting wide screen videos made for television to narrower aspect ratios for mobile social media platforms. It explores a new method to perform cropping and panning using five AI models, and an evaluation with 5 users and a review with a professional video editor were presented.Doktorgradsavhandlin

The role of attention and emotional responses on online retargeting campaigns

Retargeting consists of communicating towards consumers that have already been in contact with a brand - because they visited the website or clicked on an advert, for example. Although nowadays people tend to avoid advertising, retargeting has proven to be a very successful method for bringing back consumers that did not conclude a purchase or simply people that showed previous interest in a brand. Also, it is known that attention and emotions play a big role in how people react to advertising and how they perceive the brands that communicate with them. Bearing this in mind, this study hypothesizes that retargeted advertising gets higher levels of attention than either generic or targeted advertising. In the same way, it is proposed that retargeted advertising induces higher levels of positive emotions than the other types of advertising. In order to study such topic, a two-day experiment was created to simulate a decision-making process. Participants were exposed to products but did not finish a purchase of their choice on day one, only to see it advertised on a blog a few days later, among other types of advertising. This way, it was possible to study participant’s reactions to different types of advertising - retargeted, targeted and generic - on a longitudinal study and how retargeted adverts impact their intention to revisit the website, purchase and recommend. This study shows that retargeted advertising gets higher levels of attention than the other two types of ads. Also, it was possible to understand that retargeted advertising has a positive direct relationship with intention to revisit, and a positive indirect relationship with intention to purchase and intention to recommend, both mediated by intention to revisit.O retargeting consiste em comunicar directamente com consumidores que já tenham estado em contacto com a marca - porque visitaram o website anteriormente ou porque clicaram num anúncio da marca. Apesar de se saber que as pessoas tendem a evitar os anúncios, o retargeting tem provado ser um método muito bem-sucedido para trazer de volta consumidores que não chegaram a finalizar uma compra, ou que simplesmente mostraram interesse na marca anteriormente. É também sabido que a atenção e as emoções têm um papel muito importante na definição da maneira como as pessoas reagem à publicidade e do modo como percepcionam as marcas que comunicam consigo. Tendo isto em consideração, o presente estudo lança a hipótese de que anúncios retargeted recebem níveis mais elevados de atenção que anúncios targeted ou genéricos. Da mesma forma, é proposta a hipótese de que os anúncios retargeted induzem níveis mais positivos de emoções, quando comparados com os restantes tipos. Uma experiência de dois dias foi criada de modo a simular um processo de decisão de compra incompleto. Os participantes não finalizavam a compra de um produto que escolhiam como o seu desejo, de modo a que alguns dias depois esse mesmo produto aparecesse num anúncio num blog, entre os outros tipos de anúncios. Desta forma, foi possível estudar as reações dos participantes aos diferentes tipos de publicidade - retargeted, targeted e genérico - mas também estudar o modo como os anúncios retargeted influenciam a intenção de compra, intenção de revisita e intenção de recomendação. Este estudo permitiu concluir que os anúncios retargeted têm genericamente melhores níveis de atenção que os restantes tipos de anúncio. Também foi possível perceber que a publicidade retargeted tem uma relação direta positiva com a intenção de revisita, e uma relação indirecta positiva com a intenção de compra e de recomendação - ambas mediadas pela intenção de revisita

The use of motion capture in non-realistic animation

The Use of Motion Capture in Non-realistic Animation explores the possibility of creating non-realistic animation through the use of motion capture. In this study we look to the particularities of cartoony/non-realistic animation while trying to as-certain if it is viable to create this type of animation through the process of motion capture. This dissertation will, firstly, expose the historical, theoretical, technical and artistic context. There will be a brief description of important landmarks and general overview of the history of animation. There will also be an explanation of how animators’ will to mimic real life motion, led to the invention of several technologies in order to achieve this goal. Next we will describe the several stages that compose the motion capture process. Lastly there will be a comparison be-tween key-frame animation and motion capture animation techniques and also the analysis of several examples of films where motion capture was used. Finally there will be a description of the production phases of an animated short film called Na-poleon’s Unsung Battle. In this film the majority of its animated content was obtained through the use of motion capture while aiming for a cartoony/non-realistic style of animation. There is still margin for improvement on the final results but there is also proof that it is possible to obtain a non-realistic style of animation while using motion capture technology. The questions that remain are: is it time effective and can the process be optimized for this less than common use

Design Techniques for Lithography-Friendly Nanometer CMOS Integrated Circuits

The Integrated Circuits industry has been a major driver of the outstanding changes and improvements in the modern day technology and life style that we are observing in our day to day life. The continuous scaling of CMOS technology has been one of the major challenges and success stories. However, as the CMOS technology advances deeply into the deep sub-micron technology nodes, the whole industry (both manufacturing and design) is starting to face new challenges. One major challenge is the control of the variation in device parameters. Lithography variations result from the industry incapability to come up with new light sources with a smaller wavelength than ArF source (193 nm wavelength). In this research, we develop better understanding of the photo-lithography variations and their effect on how the design gets patterned. We investigate the state-of-the-art mask correction and design manipulation techniques. We are focusing in our study on the different Optical Proximity Correction (OPC) and design retargeting techniques to assess how we can improve both the functional and parametric yield. Our goal is to achieve a fast and accurate Model Based Re-Targeting (MBRT) technique that can achieve a better functional yield during manufacturing by establishing the techniques to produce more lithography-friendly targets. Moreover, it can be easily integrated into a fab's PDK (due to its relatively high speed) to feedback the exact final printing on wafer to the designers during the early design phase. In this thesis, we focus on two main topics. First is the development of a fast technique that can predict the final mask shape with reasonable accuracy. This is our proposed Model-based Initial Bias (MIB) methodology, in which we develop the full methodology for creating compact models that can predict the perturbation needed to get to an OPC initial condition that is much closer to the final solution. This is very useful in general in the OPC domain, where it can save almost 50% of the OPC runtime. We also use MIB in our proposed Model-Based Retargeting (MBRT) flow to accurately compute lithography hot-spots location and severity. Second, we develop the fast model-based retargeting methodology that is capable of fixing lithography hot spots and improving the functional yield. Moreover, in this methodology we introduce to the first time the concept of distributed retargeting. In distributed MBRT, not only the design portion that is suffering from the hot-spot is moving to get it fixed but also the surrounding designs and design fragments also contribute to the hot-spot fix. Our proposed model-based retargeting methodology also includes the multiple-patterning awareness as well as the electrical-connectivity-awareness (via-awareness). We used Mentor Graphics Calibre Litho-API c-based programing to develop all of the methodologies we explain in this thesis and tested it on 20nm and 10nm nodes