Free software and free culture in Higher Education: pipelines and workflows for creative purposes

OpenLab ESEV is a project of the School of Education of the Polytechnic Institute of Viseu (ESEV), Portugal, that aims to promote, foster and support the use of Free/Libre Software and Open Source Software, Open Educational Resources, Free Culture, Free file formats and more flexible copyright licenses for creative and educational purposes in the ESEV's domains of activity (education, arts, media). Most of the OpenLab ESEV activities are related to the teacher education and arts and multimedia programs, with a special focus on the later. In this paper, the project and some activities are presented, starting with its origins and its conceptual framework. The presented overview is intended as background for the examination of the use of Free/Libre Software and Free Culture in educational settings, specially at the higher education level, and for creative purposes. The activities developed with students and professionals generated pipelines and workflows implemented for different creative purposes, software packages used for different tasks, choices for file formats and copyright licenses. Finished and ongoing multimedia and arts projects will be presented as real case scenarios

VFX – A New Frontier: The Impact of Innovative Technology on Visual Effects

Although Visual Effects (VFX) are an increasingly important element of the media content demanded by audiences, of media production (filmmaking and storytelling) and of the media industries, VFX remains a relatively under-research area within academic media or film studies. Innovations in technology are instrumental to the continuous developments in VFX technology, enabling the evolution of storytelling techniques and expanding the boundaries of VFX content and the VFX industries. In particular, a new wave of cutting-edge technologies have contributed to a period of extensive technical and organisational changes in the VFX industry. The implementation of these technologies is occurring during a period of growth in demand for VFX content, ever hight standards of quality (in particular the realism of VFX effects) and resulting demand for VFX workers. Supplying this demand for both greater quantity and quality of VFX content has increased the pressure for VFX production to be as efficient as possible. This has brought pressure on production budgets (to produce more and better content from the same or even diminishing resources) and production timeframes (“turnaround times”). One result of all these changes is that VFX workers now confront a multitude of new challenges. This study investigates the new technology which is driving or enabling these changes and in particular focuses on the impact of implementing these technologies on VFX production (the VFX workflow). The study collects evidence to show how these new technologies, combined with the broader changes in the industry, are impacting VFX production and labour. The thesis approaches this research task by use economic and sociological theories of technology, innovation, and production/labour to provide a conceptual framework to use in understanding how these changes are impacting the products produced by the industry and the work experience of VFX professionals. The next step is to fill in the gaps in knowledge resulting from the relatively under-researched nature of VFX production withing academic media and film studies. The thesis provides a detailed account of the emergence and growth of “the VFX industry”, including historical and current product and process innovations. Rather than defining the object of study in relation to content genres or types of business, the study defines the industry in terms of workers using a common set of tools. This section of the thesis explores the economic and cultural causes of changes in the industry and maps out the qualitative changes in the creativity, job satisfaction and job security/precarity of VFX labour. The collection of primary data through interviews with industry professionals provides the unique contribution of this study, setting out how VFX work is changing in different content genres, types of business and production roles, at different hierarchical levels. This study contributes to the field by addressing the need for academic and empirical research in this neglected area of study. The thesis contributes original knowledge on the impact of current technological innovations by providing research based on primary data collected from interviews with the VFX workers impacted by the implementation of the technologies. Potential policy and practical applications of this research include assisting industry professionals in deconstructing the marketing “hype” around these cutting-edge technologies and outlining uncertainties and implications of these technologies, helping them in the complex decision making of evaluating and implementing current innovative technology

The Role of Python in Visual Effects Pipeline : Case: Talvi Tools

The purpose of this thesis was to study the concept of visual effects pipelines and analyze how programming language Python can fit into the post-production phase of film making. In the extremely competitive environment of visual effects industry, companies are forced to constantly look out for newer technologies and research more optimal production methodologies. In search of feasible solutions studios often come across Python. The theoretical part of the study outlines a brief history of Python and illustrates the power of this programming language with two exemplified use case applications. In addition to that, various possible Python implementations in the production of computer generated imagery are extensively reviewed. To give a more complete picture of how this programming language aids post-production, the adoption of Python by Industrial Light & Magic is examined. As it became clear that Python would prevail in the production of computer graphics, software vendors started embedding Python support in their products. This claim is further supported by the analysis of Python integration within major contect-creation applications. The outcome of this study is the add-on for Blender developed in Python. The purpose of the add-on is to facilitate and accelerate the export of character and camera animation data, which is specifically useful for projects that require a substantial amount of computer animation to be moved from Blender to another software for later use

An Investigation of How Lighting and Rendering Technology Affects Filmmaking Relative to Arnold’s Transition to a GPU-Based Path-Tracer

Computer Graphic (CGI) technology enables artists to explore a broad spectrum of approaches and styles, from photorealistic to abstract, expanding the boundaries of traditional aesthetic choices. Recent years have witnessed of 3D-CGI production shift towards greater physical fidelity driven by technological developments as well as consumer demand for realistic visuals; this trend can be found across various creative fields like film, video games, and virtual reality experiences with high-quality textures, lighting, rendering, and physics simulations providing enhanced levels of immersion for users. Arnold is one of the famous rendering engines assisting artists to be more creative while producing photorealistic images. Moreover, Arnold renders the engine as one of the main path-tracing renderers and contributes significantly to more fantastic photorealistic productions. Also, Arnold renders not only Support CPU render but also support GPU rendering to take full advantage of faster computation times and real-time interactivity, among many other advantages. Because of that, this study investigates how new technology like developed GPUs helps artists and filmmakers better comprehend 3D rendering solutions that impact their workflows. On the other hand, philosophically exploring the relationship between making a creative decision and technology within 3D photorealistic rendering reveals an intricate yet dynamic relationship that informs the creative processes of both independent artists and small studios alike. This interaction serves as a reminder that Art is driven forward by its creator\u27s creative energy rather than simply technological capabilities; artists and studios can continue pushing limits by embracing this complex dialogue between creativity and tech, opening new paths within digital Art\u27s fast-evolving realm

Animating Unpredictable Effects

Uncanny computer-generated animations of splashing waves, billowing smoke clouds, and characters’ flowing hair have become a ubiquitous presence on screens of all types since the 1980s. This Open Access book charts the history of these digital moving images and the software tools that make them. Unpredictable Visual Effects uncovers an institutional and industrial history that saw media industries conducting more private R&D as Cold War federal funding began to wane in the late 1980s. In this context studios and media software companies took concepts used for studying and managing unpredictable systems like markets, weather, and fluids and turned them into tools for animation. Unpredictable Visual Effects theorizes how these animations are part of a paradigm of control evident across society, while at the same time exploring what they can teach us about the relationship between making and knowing

Shadows, touch and digital puppeteering: a media archaeological approach

Aims The practical aim of this research project is to create a multi-touch digital puppetry system that simulates shadow theatre environments and translates gestural acts of touch into live and expressive control of virtual shadow figures. The research is focussed on the qualities of movement achievable through the haptics of single and multi-touch control of the digital puppets in the simulation. An associated aim is to create a collaborative environment where multiple performers can control dynamic animation and scenography, and create novel visualisations and narratives. The conceptual aim is to link traditional and new forms of puppetry seeking cultural significance in the ‘remediation’ of old forms that avail themselves of new haptic resources and collaborative interfaces. The thesis evaluates related prior art where traditional worlds of shadow performance meet new media, digital projection and 3D simulation, in order to investigate how changing technical contexts transform the potential of shadows as an expressive medium. Methodology The thesis uses cultural analysis of relevant documentary material to contextualise the practical work by relating the media archaeology of 2D puppetry—shadows, shadowgraphs and silhouettes—to landmark work in real-time computer graphics and performance animation. The survey considers the work of puppeteers, animators, computer graphics specialists and media artists. Through practice and an experimental approach to critical digital creativity, the study provides practical evidence of multiple iterations of controllable physics-based animation delivering expressive puppet motion through touch and multiuser interaction. Video sequences of puppet movement and written observational analysis document the intangible aspects of animation in performance. Through re-animation of archival shadow puppets, the study presents an emerging artistic media archaeological method. The major element of this method has been the restoration of a collection of Turkish Karagöz Shadow puppets from the Institut International de la Marionnette (Charleville, France) into a playable digital form. Results The thesis presents a developing creative and analytical framework for digital shadow puppetry. It proposes a media archaeological method for working creatively with puppet archives that unlock the kinetic and expressive potential of restored figures. The interaction design introduces novel approaches to puppetry control systems—using spring networks—with objects under physics-simulation that demonstrate emergent expressive qualities. The system facilitates a dance of agency¹ between puppeteer and digital instrument. The practical elements have produced several software iterations and a tool-kit for generating elegant, nuanced multi-touch shadow puppetry. The study presents accidental discoveries—serendipitous benefits of open-ended practical exploration. For instance: the extensible nature of the control system means novel input—other than touch—can provide exciting potential for accessible user interaction, e.g. with gaze duration and eye direction. The study also identifies limitations including the rate of software change and obsolescence, the scope of physics-based animation and failures of simulation. Originality/value The work has historical value in that it documents and begins a media archaeology of digital puppetry, an animated phenomenon of increasing academic and commercial interest. The work is of artistic value providing an interactive approach to making digital performance from archival material in the domain of shadow theatre. The work contributes to the electronic heritage of existing puppetry collections. The study establishes a survey of digital puppetry, setting a research agenda for future studies. Work may proceed to digitise, rig and create collaborative and web-mediated touch-based motion control systems for 2D and 3D puppets. The present study thus provides a solid platform to restore past performances and create new work from old, near forgotten-forms. ¹ Following Andrew Pickering, puppetry is ‘a temporally extended back-and-forth dance of human and non-human agency in which activity and passivity on both sides are reciprocally intertwined’ PICKERING, A. 2010. Material Culture and the Dance of Agency. In: BEAUDRY, M. C. & HICKS, D. (eds.) Oxford Handbook of Material Culture Studies. Oxford University Press.

From Production to Education: An Analysis of Pipeline Requirements and Practices

Animation, visual effects, and video game studios have to manage complex and highly iterative productions. The processes, tools, and data flow that carry a production from initial idea to finished state is called a ’pipeline.’ Students in academic programs, even ones focused on educating for digital production, often do not have a well- defined pipeline and spend unnecessary time on technical details rather than creative work. Through interviews with industry professionals, analysis of published works on pipeline and digital production, and study of current academic pipelines, this thesis presents general principles for pipelines as well as suggestions for applying these principles in academic environments. Implementing these suggestions could provide a foundation for a robust academic pipeline that lets students spend more time creating and collaborating and prepares them for employment in the digital production industry

Physics-based modelling, simulation, placement and learning for musculo-skeletal animations.

In character production for Visual Effects, the realism of deformations and flesh dynamics is a vital ingredient of the final rendered moving images shown on screen. This work is a collection of projects completed at the hosting company MPC London focused on the main components needed for the animation of musculo-skeletal systems: primitives modeling, physically accurate simulation, interactive placement. Complementary projects are also presented, including the procedural modeling of wrinkles and a machine learning approach for deformable objects based on Deep Neural Networks. Primitives modeling aims at proposing an approach to generating muscle geometry complete with tendons and fibers from superficial patches sketched on the character skin mesh. The method utilizes the physics of inflatable surfaces and produces meshes ready to be tetrahedralized, that is without compenetrations. A framework for the simulation of muscles, fascia and fat tissues based on the Finite Elements Method (FEM) is presented, together with the theoretical foundations of fiber-based materials with activations and their fitting in the Implicit Euler integration. The FEM solver is then simplified in or- der to achieve interactive rates to show the potential of interactive muscle placement on the skeleton to facilitate the creation of intersection-free primitives using collision detection and resolution. Alongside physics simulation for biological tissues, the thesis explores an approach that extends the Implicit Skinning technique with wrinkles based on convolution surfaces by exploiting the gradients of the combination of bones fields. Finally, this work discusses a possible approach to the learning of physics-based deformable objects based on deep neural networks which makes use of geodesic disks convolutional layers

As cinemáticas e narrativas de jogos digitais: implicações para o design de jogos

Game cinematics integrating filming techniques, cutscenes, and animations are likely to have a pivotal role in the player experience, emotional attachment, and relatedness to the game narrative content. In recent years, the avoidance of cuts to generate a seamless and connected game experience has challenged older patterns on storytelling in games and find new ways of intertwining gameplay with the narrative. However, a one-size-fits-all strategy may not be applied when considering different storytelling purposes, structures, and language used in diversified game genres. The purpose of this research is to examine the contributions of cinematics to affect game story comprehension, especially in young adults. A multi-stage development research method is applied, encompassing the following activities: (1) identification of requirements to develop game cinematics based on the literature review and interviews with 16 scholars and industry professionals in Game UX, and cinematic development; (2) Development of a Game Cinematics experiment for the game Mutation Madness, using first- and third- perspectives; and a (3) Comparative evaluation of game cinematics in terms of experience and narrative comprehension. The cinematics developed are tested with 46 young adults to assess the effect of the Point Of View (POV) camera on visual attention and story comprehension in Mutation Madness, using an eye- tracking experiment. The results suggest that third- perspective in game cinematics constitute omniscient knowledge of the story, evoking the sense of time and focusing on the story agents, while first- perspective visually guides the player to the game events happening in the time of gameplay. This research contributes to the Communication Sciences and Technologies field by presenting a set of best practices developing game cinematics.As cinemáticas de jogos que integram técnicas de filmagem, cutscenes, e animações, tendem a ter um papel central na experiência do jogador, envolvimento emocional, e relação com a narrativa. Nos últimos anos, os padrões mais antigos inerentes ao processo de contar histórias nos jogos têm sido desafiados pela ausência de cortes e interrupções na experiência, de modo a garantir a interconexão e o entrelaçar da jogabilidade com a narrativa. No entanto, a adoção de uma estratégia de dimensão única pode não atender a diferentes propósitos da narrativa, estruturas e linguagem utilizadas em diferentes géneros de jogos. O objetivo desta investigação é compreender o modo como as cinemáticas podem afetar a compreensão de histórias de jogos, especialmente em jovens adultos. O método de investigação de desenvolvimento é aplicado, subdividido nas seguintes etapas: (1) Identificação de requisitos para desenvolver cinemáticas de jogos com base na revisão da literatura e entrevista a 16 académicos e profissionais da indústria em experiência de jogo, e desenvolvimento de cinemáticas; (2) Desenvolvimento de cinemáticas do jogo Mutation Madness, com recurso à primeira e terceira perspetiva; e (3) Avaliação comparativa das cinemáticas de jogo em termos de experiência e compreensão da narrativa. As cinemáticas desenvolvidas são testadas por 46 jovens adultos, recorrendo à metodologia eye-tracking, para avaliar o efeito da perspetiva da câmara [Point Of View (POV)] na atenção visual e na compreensão da narrativa do jogo Mutation Madness. Os resultados sugerem que a cinemática em que é adotada a terceira perspetiva no jogo proposto contribui para um conhecimento omnisciente da história, evocando o sentido do tempo e concentrando-se nos agentes da história, enquanto a primeira perspetiva orienta visualmente o jogador para os eventos do jogo que acontecem no tempo da jogabilidade. Esta investigação contribui para a área das Ciências da Tecnologia da Comunicação ao apresentar um conjunto de melhores práticas para desenvolver as cinemáticas nos jogos.Mestrado em Comunicação Multimédi