VFX – A New Frontier: The Impact of Innovative Technology on Visual Effects

Although Visual Effects (VFX) are an increasingly important element of the media content demanded by audiences, of media production (filmmaking and storytelling) and of the media industries, VFX remains a relatively under-research area within academic media or film studies. Innovations in technology are instrumental to the continuous developments in VFX technology, enabling the evolution of storytelling techniques and expanding the boundaries of VFX content and the VFX industries. In particular, a new wave of cutting-edge technologies have contributed to a period of extensive technical and organisational changes in the VFX industry. The implementation of these technologies is occurring during a period of growth in demand for VFX content, ever hight standards of quality (in particular the realism of VFX effects) and resulting demand for VFX workers. Supplying this demand for both greater quantity and quality of VFX content has increased the pressure for VFX production to be as efficient as possible. This has brought pressure on production budgets (to produce more and better content from the same or even diminishing resources) and production timeframes (“turnaround times”). One result of all these changes is that VFX workers now confront a multitude of new challenges. This study investigates the new technology which is driving or enabling these changes and in particular focuses on the impact of implementing these technologies on VFX production (the VFX workflow). The study collects evidence to show how these new technologies, combined with the broader changes in the industry, are impacting VFX production and labour. The thesis approaches this research task by use economic and sociological theories of technology, innovation, and production/labour to provide a conceptual framework to use in understanding how these changes are impacting the products produced by the industry and the work experience of VFX professionals. The next step is to fill in the gaps in knowledge resulting from the relatively under-researched nature of VFX production withing academic media and film studies. The thesis provides a detailed account of the emergence and growth of “the VFX industry”, including historical and current product and process innovations. Rather than defining the object of study in relation to content genres or types of business, the study defines the industry in terms of workers using a common set of tools. This section of the thesis explores the economic and cultural causes of changes in the industry and maps out the qualitative changes in the creativity, job satisfaction and job security/precarity of VFX labour. The collection of primary data through interviews with industry professionals provides the unique contribution of this study, setting out how VFX work is changing in different content genres, types of business and production roles, at different hierarchical levels. This study contributes to the field by addressing the need for academic and empirical research in this neglected area of study. The thesis contributes original knowledge on the impact of current technological innovations by providing research based on primary data collected from interviews with the VFX workers impacted by the implementation of the technologies. Potential policy and practical applications of this research include assisting industry professionals in deconstructing the marketing “hype” around these cutting-edge technologies and outlining uncertainties and implications of these technologies, helping them in the complex decision making of evaluating and implementing current innovative technology

Artificial Intelligence in the Creative Industries: A Review

This paper reviews the current state of the art in Artificial Intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically Machine Learning (ML) algorithms, is provided including Convolutional Neural Network (CNNs), Generative Adversarial Networks (GANs), Recurrent Neural Networks (RNNs) and Deep Reinforcement Learning (DRL). We categorise creative applications into five groups related to how AI technologies are used: i) content creation, ii) information analysis, iii) content enhancement and post production workflows, iv) information extraction and enhancement, and v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, machine learning-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of machine learning in domains with fewer constraints, where AI is the `creator', remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human centric -- where it is designed to augment, rather than replace, human creativity

Intelligenza artificiale e industrie culturali storia, tecnologie e potenzialità dell’ia nella produzione cinematografica

Negli ultimi anni stiamo assistendo, in svariati campi, a un sempre più vasto utilizzo di tecnologie che utilizzano quella che viene comunemente chiamata “intelligenza artificiale”. Anche il settore audiovisivo, da sempre recettore di novità e incline a evolversi continuamente, sta già vivendo quei processi che lo porteranno a essere rivoluzionato da questo tipo di tecnologie. In un periodo di frenetico progresso scientifico è difficile riuscire a fissare nel tempo e su carta lo stato attuale dello sviluppo tecnologico, dato che ciò che oggi viene considerato come novità domani potrebbe già essere stato superato. È necessario, quindi, uno strumento che riesca a catalogare, se non tutte, almeno le più importanti rivoluzionarie tecnologie d’intelligenza artificiale che hanno investito il mondo della produzione artistica e delle industrie culturali. Uno studio approfondito è dedicato, in particolare, all’industria cinematografica. Dopo una breve introduzione di carattere storico, vengono descritti i principali tipi di rete neurale artificiale e la loro evoluzione. Sono poi delineate e descritte le principali tecnologie d’IA applicate all’elaborazione, comprensione e generazione automatica o assistita d’immagine e testo. Ancora più nel dettaglio sono osservate alcune soluzioni tecnologiche che interessano le varie fasi del processo di produzione cinematografica, come la fase di scrittura e analisi della sceneggiatura, quella di editing e montaggio video, così come quelle riguardanti l’implementazione di effetti visivi e la composizione musicale. Il testo risulta essere, da un lato, una fotografia sul passato che ha interessato lo sviluppo delle tecnologie d’IA, dall’altro uno strumento che illustra il presente così da aiutarci, se non a predire, almeno a non trovarci completamente impreparati di fronte agli sviluppi futuri che interesseranno sia la produzione audiovisiva che, in senso più ampio, la nostra vita di tutti i giorni

Virtual reality and artificial intelligence applied to consumer experience: a scenario-based approach

Emerging technologies like Virtual Reality (VR) and Artificial Intelligence (AI) have been receiving much attention in the Marketing context as a new way to attract customers and enrich their experiences with brands. This study aimed to research the current state of these technologies in Business and discuss the extent to which experiences using Virtual Reality and Artificial Intelligence may impact a customer experience in three different points of the customer journey: pre-purchase, purchase, and post- purchase stage. An online survey was constructed to measure the effectiveness of VR and AI on a scenario-based approach, along three consumer behaviours in the customer journey – brand associations (pre-purchase stage), purchase intention (purchase stage) and brand loyalty (post-purchase stage). From a sample of 302 respondents, the results suggest that these technologies represent valuable opportunities in Marketing. Overall, the results revealed that the type of experiences addressed in this paper are more prone to instil action (purchase stage) than to build brand associations (pre-purchase stage) or brand loyalty (post-purchase stage). An in-depth analysis of these results manifested interesting outcomes, suggesting several managerial on how to apply these technologies in an integrated marketing communication (IMC) plan.Realidade Virtual (RV) e Inteligência Artificial (IA) têm recebido bastante atenção nos últimos anos pelo facto de serem uma nova forma de atrair os consumidores e enriquecer as suas experiências com as marcas. O principal objetivo desta dissertação é oferecer uma visão do "state of the art" da aplicação destas tecnologias no marketing e investigar até que ponto as experiências com RV e IA podem contribuir para uma melhor experiência do consumidor, nomeadamente em três fases distintas da jornada do consumidor: antes da compra, no momento da compra, e após a compra. Um questionário online foi construído para medir a eficácia de 7 cenários usando RV e AI, em três dos mais representativos comportamentos de uma jornada do consumidor: associações à marca (antes da compra), intenção de comprar (momento da compra) e lealdade à marca (após a compra). Com base numa amostra de 302 participantes, os resultados sugerem que RV e IA contribuem para a criação de significantes oportunidades no marketing. Numa forma geral, os resultados indicam que as experiências que foram selecionadas para este estudo são mais eficientes a persuadir os potenciais consumidores a comprar (momento da compra) do que a contribuir para a criação de associações à marca (antes da compra) ou para construir lealdade à marca (depois da compra). Sendo que, numa análise mais profunda dos resultados, diversas recomendações em como aplicar estas tecnologias numa comunicação integrada de marketing foram retratadas neste estudo

"An Adapt-or-Die Type of Situation": Perception, Adoption, and Use of Text-To-Image-Generation AI by Game Industry Professionals

Text-to-image generation (TTIG) models, a recent addition to creative AI, can generate images based on a text description. These models have begun to rival the work of professional creatives, and sparked discussions on the future of creative work, loss of jobs, and copyright issues, amongst other important implications. To support the sustainable adoption of TTIG, we must provide rich, reliable and transparent insights into how professionals perceive, adopt and use TTIG. Crucially though, the public debate is shallow, narrow and lacking transparency, while academic work has focused on studying the use of TTIG in a general artist population, but not on the perceptions and attitudes of professionals in a specific industry. In this paper, we contribute a qualitative, exploratory interview study on TTIG in the Finnish videogame industry. Through a Template Analysis on semi-structured interviews with 14 game professionals, we reveal 12 overarching themes, structured into 49 sub-themes on professionals' perception, adoption and use of TTIG systems in games industry practice. Experiencing (yet another) change of roles and creative processes, our participants' reflections can inform discussions within the industry, be used by policymakers to inform urgently needed legislation, and support researchers in games, HCI and AI to support the sustainable, professional use of TTIG to benefit people and games as cultural artefacts.Comment: 32 pages (incl. appendix), 3 figures, 4 tables. Coding template (31 pages, 10 tables), study invitations (email, social media) and pre-study survey provided as supplementary (ancillary) material. Accepted with minor revisions at ACM CHI Play 202

Virtual Production: Real-Time Rendering Pipelines for Indie Studios and the Potential in Different Scenarios

[EN] This work aims to identify and propose a functional pipeline for indie live-action films using Virtual Production with photorealistic real-time rendering game engines. The new production landscape is radically changing how movies and shows are made. Those were made in a linear pipeline, and now filmmakers can execute multiple tasks in a parallel mode using real-time renderers with high potential for different types of productions. Four interviews of professionals in the Spanish film and television market were conducted to obtain the whole perspective of the new paradigm. Following those examples, a virtual production set was implemented with an Antilatency tracking system, Unreal Engine (version 5.3), and Aximmetry (version 2023.3.2) as the leading software applications. Results are commented on, presenting how all the work is currently closely connected between pre-production, shooting, and post-production and analyzing its potential in different fields.Spanish Government (Grant PID2020-117421RB-C21 funded by MCIN/AEI/10.13039/501100011033) and Generalitat Valenciana (Grant INVEST/2022/324).Silva Jasaui, D.; Martí Testón, A.; Muñoz García, A.; Moriniello, F.; Solanes, JE.; Gracia Calandin, LI. (2024). Virtual Production: Real-Time Rendering Pipelines for Indie Studios and the Potential in Different Scenarios. Applied Sciences. 14(6). https://doi.org/10.3390/app1406253014

Large-scale Text-to-Image Generation Models for Visual Artists' Creative Works

Large-scale Text-to-image Generation Models (LTGMs) (e.g., DALL-E), self-supervised deep learning models trained on a huge dataset, have demonstrated the capacity for generating high-quality open-domain images from multi-modal input. Although they can even produce anthropomorphized versions of objects and animals, combine irrelevant concepts in reasonable ways, and give variation to any user-provided images, we witnessed such rapid technological advancement left many visual artists disoriented in leveraging LTGMs more actively in their creative works. Our goal in this work is to understand how visual artists would adopt LTGMs to support their creative works. To this end, we conducted an interview study as well as a systematic literature review of 72 system/application papers for a thorough examination. A total of 28 visual artists covering 35 distinct visual art domains acknowledged LTGMs' versatile roles with high usability to support creative works in automating the creation process (i.e., automation), expanding their ideas (i.e., exploration), and facilitating or arbitrating in communication (i.e., mediation). We conclude by providing four design guidelines that future researchers can refer to in making intelligent user interfaces using LTGMs.Comment: 15 pages, 3 figure

To Affinity and Beyond: Interactive Digital Humans as a Human Computer Interface

The field of human computer interaction is increasingly exploring the use of more natural, human-like user interfaces to build intelligent agents to aid in everyday life. This is coupled with a move to people using ever more realistic avatars to represent themselves in their digital lives. As the ability to produce emotionally engaging digital human representations is only just now becoming technically possible, there is little research into how to approach such tasks. This is due to both technical complexity and operational implementation cost. This is now changing as we are at a nexus point with new approaches, faster graphics processing and enabling new technologies in machine learning and computer vision becoming available. I articulate the issues required for such digital humans to be considered successfully located on the other side of the phenomenon known as the Uncanny Valley. My results show that a complex mix of perceived and contextual aspects affect the sense making on digital humans and highlights previously undocumented effects of interactivity on the affinity. Users are willing to accept digital humans as a new form of user interface and they react to them emotionally in previously unanticipated ways. My research shows that it is possible to build an effective interactive digital human that crosses the Uncanny Valley. I directly explore what is required to build a visually realistic digital human as a primary research question and I explore if such a realistic face provides sufficient benefit to justify the challenges involved in building it. I conducted a Delphi study to inform the research approaches and then produced a complex digital human character based on these insights. This interactive and realistic digital human avatar represents a major technical undertaking involving multiple teams around the world. Finally, I explored a framework for examining the ethical implications and signpost future research areas

Procedurally generated AI compound media for expanding audial creations, broadening immersion and perception experience

Recently, the world has been gaining vastly increasing access to more and more advanced artificial intelligence tools. This phenomenon does not bypass the world of sound and visual art, and both of these worlds can benefit in ways yet unexplored, drawing them closer to one another. Recent breakthroughs open possibilities to utilize AI driven tools for creating generative art and using it as a compound of other multimedia. The aim of this paper is to present an original concept of using AI to create a visual compound material to existing audio source. This is a way of broadening accessibility thus appealing to different human senses using source media, expanding its initial form. This research utilizes a novel method of enhancing fundamental material consisting of text audio or text source (script) and sound layer (audio play) by adding an extra layer of multimedia experience – a visual one, generated procedurally. A set of images generated by AI tools, creating a story-telling animation as a new way to immerse into the experience of sound perception and focus on the initial audial material. The main idea of the paper consists of creating a pipeline, form of a blueprint for the process of procedural image generation based on the source context (audial or textual) transformed into text prompts and providing toolsto automate it by programming a set of code instructions. This process allows creation of coherent and cohesive (to a certain extent) visual cues accompanying audial experience levering it to multimodal piece of art. Using nowadays technologies, creators can enhance audial forms procedurally, providing them with visual context. The paper refers to current possibilities, use cases, limitations and biases giving presented tools and solutions