Storytelling in the Metaverse: From Desktop to Immersive Virtual Reality Storyboarding

Creatives from the animation and film industries have always been experimenting with innovative tools and methodologies to improve the creation of prototypes of their visual sequences before bringing them to life. In recent years, as realistic real-time rendering techniques have emerged, the increasing popularity of virtual reality (VR) can lead to new approaches and solutions, leveraging the immersive and interactive features provided by 3D immersive experiences. A 3D desktop application and a novel storyboarding pipeline, which can automatically generate a storyboard including camera details and a textual description of the actions performed in three-dimensional environments, have already been investigated in previous work. The aim was to exploit new technologies to improve existing 3D storytelling approaches, thus providing a software solution for expert and novice storyboarders. This research investigates 3D storyboarding in immersive virtual reality (IVR) to move toward a new storyboarding paradigm. IVR systems provide peculiarities such as body-controlled exploration of the 3D scene and a head-dependant camera view that can extend features of traditional storyboarding tools. The proposed system enables users to set up the virtual stage, adding elements to the scene and exploring the environment as they build it. After that, users can select the available characters or the camera, control them in first person, position them in the scene, and perform actions selecting from a list of options, each paired with a corresponding animation. Relying on the concept of state-machine, the system can automatically generate the list of available actions depending on the context. Finally, the descriptions for each storyboard panel are automatically generated based on the history of activities performed. The proposed application maintains all the functionalities of the desktop version and can be effectively used to create storyboards in immersive virtual environments

Multitouch-based collaborative pre-visualisation for computer animation

Computer animated pre-visualisation occurs at an early stage of visualising scenes in low-fidelity. This is a collaborative process, in which directors communicate with animators how shot sequences will occur. Producers also take notes to approximate costs and other stakeholders may give further input. The problem with this approach is that the improvement cycles can take a long time, making the process take exponentially longer with more iterations of improvement. Our aim is to create a system that reduces this time, while keeping every stakeholder of the animation on the same page. We have constructed a multitouch-enabled system for low-fidelity, animated 3D pre-visualisation. This tool runs on a single, large multitouch table and caters for simultaneous input from multiple users, to better support collaboration. Users can navigate the virtual environment, place and manipulate 3d objects in the scene, as well as animate them, all using multitouch. The system was constructed using the user-centred systems design (UCSD) methodology. After several iterations of development, we performed a qualitative evaluation of the _nal system using two groups, one consisting of film makers and the other consisting of software developers, and concluded with interviews to get qualitative feedback about our pre-visualisation tool. Both groups suggested that the system's setup promoted collaboration and communication, which is important early on in the planning phase of film creation. However, both groups agreed that such a tool is only useful for low-fidelity pre-visualisation, as it might become "cumbersome" to perform detailed animations using multitouch input. Furthermore, the system was often too dependent on the viewpoint, which was a single user task, effectively minimizing the amount of work that could actually be done by collaborative users simultaneously. This study highlights the potential of a multitouch, collaborative pre-visualisation tool

VFX – A New Frontier: The Impact of Innovative Technology on Visual Effects

Although Visual Effects (VFX) are an increasingly important element of the media content demanded by audiences, of media production (filmmaking and storytelling) and of the media industries, VFX remains a relatively under-research area within academic media or film studies. Innovations in technology are instrumental to the continuous developments in VFX technology, enabling the evolution of storytelling techniques and expanding the boundaries of VFX content and the VFX industries. In particular, a new wave of cutting-edge technologies have contributed to a period of extensive technical and organisational changes in the VFX industry. The implementation of these technologies is occurring during a period of growth in demand for VFX content, ever hight standards of quality (in particular the realism of VFX effects) and resulting demand for VFX workers. Supplying this demand for both greater quantity and quality of VFX content has increased the pressure for VFX production to be as efficient as possible. This has brought pressure on production budgets (to produce more and better content from the same or even diminishing resources) and production timeframes (“turnaround times”). One result of all these changes is that VFX workers now confront a multitude of new challenges. This study investigates the new technology which is driving or enabling these changes and in particular focuses on the impact of implementing these technologies on VFX production (the VFX workflow). The study collects evidence to show how these new technologies, combined with the broader changes in the industry, are impacting VFX production and labour. The thesis approaches this research task by use economic and sociological theories of technology, innovation, and production/labour to provide a conceptual framework to use in understanding how these changes are impacting the products produced by the industry and the work experience of VFX professionals. The next step is to fill in the gaps in knowledge resulting from the relatively under-researched nature of VFX production withing academic media and film studies. The thesis provides a detailed account of the emergence and growth of “the VFX industry”, including historical and current product and process innovations. Rather than defining the object of study in relation to content genres or types of business, the study defines the industry in terms of workers using a common set of tools. This section of the thesis explores the economic and cultural causes of changes in the industry and maps out the qualitative changes in the creativity, job satisfaction and job security/precarity of VFX labour. The collection of primary data through interviews with industry professionals provides the unique contribution of this study, setting out how VFX work is changing in different content genres, types of business and production roles, at different hierarchical levels. This study contributes to the field by addressing the need for academic and empirical research in this neglected area of study. The thesis contributes original knowledge on the impact of current technological innovations by providing research based on primary data collected from interviews with the VFX workers impacted by the implementation of the technologies. Potential policy and practical applications of this research include assisting industry professionals in deconstructing the marketing “hype” around these cutting-edge technologies and outlining uncertainties and implications of these technologies, helping them in the complex decision making of evaluating and implementing current innovative technology

Traces d'animation intelligentes pour l'animation en VR

International audienceThe artistic crafting of 3D animations by designers is a complex and iterative process. While classical animation tools have brought significant improvements in creating and manipulating shapes over time, most approaches rely on classical 2D input devices to create 3D contents. With the advent of virtual reality technologies and their ability to dive the users in their 3D worlds and to precisely track devices in 6 dimensions (position and orientation), a number of VR creative tools have emerged such as Quill, AnimVR, Tvori, Tiltbrush or MasterPieceVR. While these tools provide intuitive means to directly design in the 3D space by exploiting both the 6D tracking capacity of the hand devices and the stereoscopic perception by the user, the animation capacities or such tools remain strongly limited, and often reproduce classical 2D manipulators in VR. In this work, we propose the design of smart interactive manipulators which leverage on the specificity of VR to animate poly-articulated animations. We then perform a user study to evaluate the benefits of such manipulators over traditional 2D tools for three groups of users: beginner, intermediate, and professional artists. We build on this user to discuss how smart tools (e.g. using a variety of AI techniques) can be coupled with VR technologies to improve content creation.L'élaboration artistique d'animations 3D par des concepteurs est un processus complexe et itératif. Bien que les outils d'animation classiques aient apporté des améliorations significatives dans la création et la manipulation des formes au fil du temps, la plupart des approches s'appuient sur des dispositifs d'entrée 2D classiques pour créer des contenus 3D. Avec l'avènement des technologies de réalité virtuelle et leur capacité à plonger les utilisateurs dans leurs mondes 3D et à suivre précisément les dispositifs en 6 dimensions (position et orientation), un certain nombre d'outils de création VR ont émergé tels que Quill, AnimVR, Tvori, Tiltbrush ou MasterPieceVR. Alors que ces outils fournissent des moyens intuitifs pour concevoir directement dans l'espace 3D en exploitant à la fois la capacité de suivi 6D des dispositifs manuels et la perception stéréoscopique de l'utilisateur, les capacités d'animation de ces outils restent fortement limitées, et reproduisent souvent des manipulateurs 2D classiques dans la RV. Dans ce travail, nous proposons la conception de manipulateurs interactifs intelligents qui tirent parti de la spécificité de la RV pour animer des animations poly-articulées. Nous réalisons ensuite une étude d'utilisateurs pour évaluer les avantages de tels manipulateurs par rapport aux outils 2D traditionnels pour trois groupes d'utilisateurs : les artistes débutants, intermédiaires et professionnels. Nous nous appuyons sur cet utilisateur pour discuter de la façon dont des outils intelligents (par exemple, utilisant une variété de techniques d'IA) peuvent être couplés aux technologies de RV pour améliorer la création de contenu

A grammar model and curriculum resource for stereoscopic 3-D film production techniques

David Crowe's research was sparked by the lack of a suitable storytelling model for stereoscopic 3-D in cinema. David not only refined a working 3-D curriculum, which is now delivered as a Masters module in Higher Education, but also refined an appropriate 3-D film grammar, for now and the future

Electronic Imaging & the Visual Arts. EVA 2015 Florence

Information Technologies of interest for Culture Heritage are presented: multimedia systems, data-bases, data protection, access to digital content, Virtual Galleries. Particular reference is reserved to digital images (Electronic Imaging & the Visual Arts), regarding Cultural Institutions (Museums, Libraries, Palace – Monuments, Archaeological Sites). The International Conference includes the following Sessions: Strategic Issues; New Technologies & Applications; New 2D-3D Technical Developments & Applications; Virtual Galleries – Museums and Related Initiatives; Access to the Culture Information. Two Workshops regard: International Cooperation; Innovation and Enterprise

VR as a Content Creation Tool for Movie Previsualisation

International audienc

Evoking presence through creative practice on Pepper's ghost displays.

This thesis proposes a theoretic framework for the analysis of presence research in the context of Pepper’s ghost. Pepper’s ghost as a media platform offers new possibilities for performances, real-time communication and media art. The thesis gives an overview on the 150 year old history, as well as contemporary art creation on Pepper’s ghost with a specific focus on telepresence. Telepresence, a concept that infused academic debate since 1980, discusses the topic of remote communication, perceived presence transmitted through networked environments. This discourse of telepresence revealed shortcomings in current analytical frameworks. This thesis presents a new model for presence in the context of my research. The standard telepresence model (STM) assumes a direct link between three fundamental components of presence and a measurable impact on the audience. Its three pillars are conceptualised as presence co-factors immersion, interactivity and realism, presented individually in the framework of my practice. My research is firmly rooted in the field of media art and considers the effect of presence in the context of Pepper’s ghost. This Victorian parlour trick serves as an interface, an intermediary for the discussion of live streaming experiences. Three case studies present pillars of the standard model, seeking answers to elemental questions of presence research. The hypothesis assumes a positive relationship between presence and its three co-factors. All case studies were developed as media art pieces in the context of Pepper’s ghost. As exemplifiers, they illustrate the concept of presence in respect of my own creative practice. KIMA, a real-time sound representation experience, proposes a form of telepresence that relies exclusively on immersive sound as a medium. Immersion as co-factor of presence is analysed and explored creatively on the Pepper’s ghost canvas. Transmission, the second case study, investigates the effect of physical interaction on presence experiences. An experiment helps to draw inferences in a mixed method approach. The third case study, Aura, discusses variations of realism as presence co factor in the specific context of Pepper’s ghost. The practical example is accompanied by an in-depth meta-analysis of realism factors, specifically focusing on the intricacies of Pepper’s ghost creative production processes. Together, these three case studies help to shed light on new strategies to improve production methods with possible impact on presence in Pepper’s ghost related virtual environments – and beyond