Depth, shading, and stylization in stereoscopic cinematography

Due to the constantly increasing focus of the entertainment industry on stereoscopic imaging, techniques and tools that enable precise control over the depth impression and help to overcome limitations of the current stereoscopic hardware are gaining in importance. In this dissertation, we address selected problems encountered during stereoscopic content production, with a particular focus on stereoscopic cinema. First, we consider abrupt changes of depth, such as those induced by cuts in films. We derive a model predicting the time the visual system needs to adapt to such changes and propose how to employ this model for film cut optimization. Second, we tackle the issue of discrepancies between the two views of a stereoscopic image due to view-dependent shading of glossy materials. The suggested solution eliminates discomfort caused by non-matching specular highlights while preserving the perception of gloss. Last, we deal with the problem of filmgrainmanagement in stereoscopic productions and propose a new method for film grain application that reconciles visual comfort with the idea of medium-scene separation.Aufgrund der ständig steigenden Beachtung der stereoskopische Abbildung durch die Unterhaltungsindustrie, gewinnen Techniken und Werkzeuge an Bedeutung, die eine präzise Steuerung der Tiefenwahrnehmung ermöglichen und Einschränkungen der gegenwärtigen stereoskopischen Geräte überwinden. In dieser Dissertation adressieren wir ausgewählte Probleme, die während der Erzeugung von stereoskopischen Inhalten auftreten, mit besonderem Schwerpunkt auf der stereoskopischen Kinematographie. Zuerst betrachten wir abrupte Tiefenänderungen, wie sie durch Filmschnitte hervergerufen werden. Wir leiten ein Modell her, das die Zeit vorhersagt, die für das menschliche Sehsystem notwendig ist, um sich an solche Änderungen der Tiefe zu adaptieren, und schlagen vor wie dieses Modell für Schnittoptimierung angewendet werden kann. Danach gehen wir das Problem der Unstimmigkeiten zwischen den zwei Ansichten eines stereoskopischen Bildes, infolge der sichtabhängigen Schattierung von glänzenden Materialien, an. Die vorgeschlagene Lösung eliminiert das visuelle Unbehagen, welches von nicht zusammenpassenden Glanzlichtern verursacht wird, indessen bewahrt sie die Glanzwahrnehmung. Zuletzt behandeln wir das Problem des Filmkornsmanagements in stereoskopischen Produktionen und schlagen eine neue Methode für das Hinzufügen vom Filmkorn vor, die die visuelle Behaglichkeit mit der Idee der Medium-Szenen-Trennung in Einklang bringt

Towards Better Methods of Stereoscopic 3D Media Adjustment and Stylization

Stereoscopic 3D (S3D) media is pervasive in film, photography and art. However, working with S3D media poses a number of interesting challenges arising from capture and editing. In this thesis we address several of these challenges. In particular, we address disparity adjustment and present a layer-based method that can reduce disparity without distorting the scene. Our method was successfully used to repair several images for the 2014 documentary “Soldiers’ Stories” directed by Jonathan Kitzen. We then explore consistent and comfortable methods for stylizing stereo images. Our approach uses a modified version of the layer-based technique used for disparity adjustment and can be used with a variety of stylization filters, including those in Adobe Photoshop. We also present a disparity-aware painterly rendering algorithm. A user study concluded that our layer-based stylization method produced S3D images that were more comfortable than previous methods. Finally, we address S3D line drawing from S3D photographs. Line drawing is a common art style that our layer-based method is not able to reproduce. To improve the depth perception of our line drawings we optionally add stylized shading. An expert survey concluded that our results were comfortable and reproduced a sense of depth

Multicamera workflow applied to a Cultural Heritage Building: Alhambra's Torre de la Cautiva from the inside

It is increasingly necessary to generate accessible and navigable digital representations of historical or heritage buildings. This article explains the workflow that was applied to create such a digital component for one of the least accessible areas of the Alhambra palace in Granada, the so-called Torre de la Cautiva (Tower of the Captive). The main goal of this process was to create affordable, photorealistic 3D models that contribute to the dissemination of cultural heritage, the decision making for its conservation and restoration, and public engagement and entertainment. With enough preparation, the time spent gathering data following a Structure from Motion (SfM) approach can be significantly reduced by using a multi-camera (low cost DSLR) photogrammetric strategy. Without the possibility of artificial lighting, it was essential to use RAW images and calibrate the color in the scene for material and texture characterization. Through processing, the amount of data was reduced by optimizing the model's topology. Thus, a photorealistic result was obtained that could be managed and visualized in immersive Visual Reality (VR) environments, simulating different historical periods and environmental and lighting conditions. The potential of this method allows, with slight modifications, the creation of HBIMs and the adaptation to VR systems development, whose current visualization quality is below the resolution of actionable models in rendering engines.This work was funded by of the Ministry of Science and Innovation of Spain, project number RTI2018-097633-A-I00, Photonic restoration applied to cultural heritage: Application to Dalis painting: Two Figures

Drawing from motion capture : developing visual languages of animation

The work presented in this thesis aims to explore novel approaches of combining motion capture with drawing and 3D animation. As the art form of animation matures, possibilities of hybrid techniques become more feasible, and crosses between traditional and digital media provide new opportunities for artistic expression. 3D computer animation is used for its keyframing and rendering advancements, that result in complex pipelines where different areas of technical and artistic specialists contribute to the end result. Motion capture is mostly used for realistic animation, more often than not for live-action filmmaking, as a visual effect. Realistic animated films depend on retargeting techniques, designed to preserve actors performances with a high degree of accuracy. In this thesis, we investigate alternative production methods that do not depend on retargeting, and provide animators with greater options for experimentation and expressivity. As motion capture data is a great source for naturalistic movements, we aim to combine it with interactive methods such as digital sculpting and 3D drawing. As drawing is predominately used in preproduction, in both the case of realistic animation and visual effects, we embed it instead to alternative production methods, where artists can benefit from improvisation and expression, while emerging in a three-dimensional environment. Additionally, we apply these alternative methods for the visual development of animation, where they become relevant for the creation of specific visual languages that can be used to articulate concrete ideas for storytelling in animation

Fusing Multimedia Data Into Dynamic Virtual Environments

In spite of the dramatic growth of virtual and augmented reality (VR and AR) technology, content creation for immersive and dynamic virtual environments remains a significant challenge. In this dissertation, we present our research in fusing multimedia data, including text, photos, panoramas, and multi-view videos, to create rich and compelling virtual environments. First, we present Social Street View, which renders geo-tagged social media in its natural geo-spatial context provided by 360° panoramas. Our system takes into account visual saliency and uses maximal Poisson-disc placement with spatiotemporal filters to render social multimedia in an immersive setting. We also present a novel GPU-driven pipeline for saliency computation in 360° panoramas using spherical harmonics (SH). Our spherical residual model can be applied to virtual cinematography in 360° videos. We further present Geollery, a mixed-reality platform to render an interactive mirrored world in real time with three-dimensional (3D) buildings, user-generated content, and geo-tagged social media. Our user study has identified several use cases for these systems, including immersive social storytelling, experiencing the culture, and crowd-sourced tourism. We next present Video Fields, a web-based interactive system to create, calibrate, and render dynamic videos overlaid on 3D scenes. Our system renders dynamic entities from multiple videos, using early and deferred texture sampling. Video Fields can be used for immersive surveillance in virtual environments. Furthermore, we present VRSurus and ARCrypt projects to explore the applications of gestures recognition, haptic feedback, and visual cryptography for virtual and augmented reality. Finally, we present our work on Montage4D, a real-time system for seamlessly fusing multi-view video textures with dynamic meshes. We use geodesics on meshes with view-dependent rendering to mitigate spatial occlusion seams while maintaining temporal consistency. Our experiments show significant enhancement in rendering quality, especially for salient regions such as faces. We believe that Social Street View, Geollery, Video Fields, and Montage4D will greatly facilitate several applications such as virtual tourism, immersive telepresence, and remote education

Blickpunktabhängige Computergraphik

Contemporary digital displays feature multi-million pixels at ever-increasing refresh rates. Reality, on the other hand, provides us with a view of the world that is continuous in space and time. The discrepancy between viewing the physical world and its sampled depiction on digital displays gives rise to perceptual quality degradations. By measuring or estimating where we look, gaze-contingent algorithms aim at exploiting the way we visually perceive to remedy visible artifacts. This dissertation presents a variety of novel gaze-contingent algorithms and respective perceptual studies. Chapter 4 and 5 present methods to boost perceived visual quality of conventional video footage when viewed on commodity monitors or projectors. In Chapter 6 a novel head-mounted display with real-time gaze tracking is described. The device enables a large variety of applications in the context of Virtual Reality and Augmented Reality. Using the gaze-tracking VR headset, a novel gaze-contingent render method is described in Chapter 7. The gaze-aware approach greatly reduces computational efforts for shading virtual worlds. The described methods and studies show that gaze-contingent algorithms are able to improve the quality of displayed images and videos or reduce the computational effort for image generation, while display quality perceived by the user does not change.Moderne digitale Bildschirme ermöglichen immer höhere Auflösungen bei ebenfalls steigenden Bildwiederholraten. Die Realität hingegen ist in Raum und Zeit kontinuierlich. Diese Grundverschiedenheit führt beim Betrachter zu perzeptuellen Unterschieden. Die Verfolgung der Aug-Blickrichtung ermöglicht blickpunktabhängige Darstellungsmethoden, die sichtbare Artefakte verhindern können. Diese Dissertation trägt zu vier Bereichen blickpunktabhängiger und wahrnehmungstreuer Darstellungsmethoden bei. Die Verfahren in Kapitel 4 und 5 haben zum Ziel, die wahrgenommene visuelle Qualität von Videos für den Betrachter zu erhöhen, wobei die Videos auf gewöhnlicher Ausgabehardware wie z.B. einem Fernseher oder Projektor dargestellt werden. Kapitel 6 beschreibt die Entwicklung eines neuartigen Head-mounted Displays mit Unterstützung zur Erfassung der Blickrichtung in Echtzeit. Die Kombination der Funktionen ermöglicht eine Reihe interessanter Anwendungen in Bezug auf Virtuelle Realität (VR) und Erweiterte Realität (AR). Das vierte und abschließende Verfahren in Kapitel 7 dieser Dissertation beschreibt einen neuen Algorithmus, der das entwickelte Eye-Tracking Head-mounted Display zum blickpunktabhängigen Rendern nutzt. Die Qualität des Shadings wird hierbei auf Basis eines Wahrnehmungsmodells für jeden Bildpixel in Echtzeit analysiert und angepasst. Das Verfahren hat das Potenzial den Berechnungsaufwand für das Shading einer virtuellen Szene auf ein Bruchteil zu reduzieren. Die in dieser Dissertation beschriebenen Verfahren und Untersuchungen zeigen, dass blickpunktabhängige Algorithmen die Darstellungsqualität von Bildern und Videos wirksam verbessern können, beziehungsweise sich bei gleichbleibender Bildqualität der Berechnungsaufwand des bildgebenden Verfahrens erheblich verringern lässt

Coraline

Coraline (Henry Selick, 2009) is stop-motion studio LAIKA's feature-length debut based on the popular children's novel by British author Neil Gaiman. Heralding a revival in global interest in stop-motion animation, the film is both an international cultural phenomenon and a breakthrough moment in the technological evolution of the craft. This open access collection brings together an international group of practitioners and scholars to examine Coraline’s place in animation history and culture, dissect its politics, and unpack its role in the technological and aesthetic development of its medium. More broadly, it celebrates stop motion as a unique and enduring artform while embracing its capacity to evolve in response to cultural, political, and technological changes, as well as shifting critical and audience demands. Divided into three sections, this volume’s chapters situate Coraline within an interconnected network of historical, industrial, discursive, theoretical, and cultural contexts. They place the film in conversation with the medium’s aesthetic and technological history, broader global intellectual and political traditions, and questions of animation reception and spectatorship. In doing so, they invite recognition – and appreciation – of the fact that Coraline occupies many liminal spaces at once. It straddles the boundary between children’s entertainment and traditional ‘adult’ genres, such as horror and thriller. It complicates a seemingly straight(forward) depiction of normative family life with gestures of queer resistance. Finally, it marks a pivotal point in stop-motion animation’s digital turn. Following the film’s recent tenth anniversary, the time is right to revisit its production history, evaluate its cultural and industry impact, and celebrate its legacy as contemporary stop-motion cinema’s gifted child. As the first book-length academic study of this contemporary animation classic, this volume serves as an authoritative introduction and a primary reference on the film for scholars, students, practitioners, and animation fans. The ebook editions of this book are available open access under a CC BY-NC-ND 4.0 licence on bloomsburycollections.com

Coraline

Coraline (Henry Selick, 2009) is stop-motion studio LAIKA's feature-length debut based on the popular children's novel by British author Neil Gaiman. Heralding a revival in global interest in stop-motion animation, the film is both an international cultural phenomenon and a breakthrough moment in the technological evolution of the craft. This open access collection brings together an international group of practitioners and scholars to examine Coraline’s place in animation history and culture, dissect its politics, and unpack its role in the technological and aesthetic development of its medium. More broadly, it celebrates stop motion as a unique and enduring artform while embracing its capacity to evolve in response to cultural, political, and technological changes, as well as shifting critical and audience demands. Divided into three sections, this volume’s chapters situate Coraline within an interconnected network of historical, industrial, discursive, theoretical, and cultural contexts. They place the film in conversation with the medium’s aesthetic and technological history, broader global intellectual and political traditions, and questions of animation reception and spectatorship. In doing so, they invite recognition – and appreciation – of the fact that Coraline occupies many liminal spaces at once. It straddles the boundary between children’s entertainment and traditional ‘adult’ genres, such as horror and thriller. It complicates a seemingly straight(forward) depiction of normative family life with gestures of queer resistance. Finally, it marks a pivotal point in stop-motion animation’s digital turn. Following the film’s recent tenth anniversary, the time is right to revisit its production history, evaluate its cultural and industry impact, and celebrate its legacy as contemporary stop-motion cinema’s gifted child. As the first book-length academic study of this contemporary animation classic, this volume serves as an authoritative introduction and a primary reference on the film for scholars, students, practitioners, and animation fans. The ebook editions of this book are available open access under a CC BY-NC-ND 4.0 licence on bloomsburycollections.com

The Art of Movies

Movie is considered to be an important art form; films entertain, educate, enlighten and inspire audiences. Film is a term that encompasses motion pictures as individual projects, as well as — in metonymy — the field in general. The origin of the name comes from the fact that photographic film (also called filmstock) has historically been the primary medium for recording and displaying motion pictures. Many other terms exist — motion pictures (or just pictures or “picture”), the silver screen, photoplays, the cinema, picture shows, flicks — and commonly movies