Photorealistic Rendering for LIve-Action Video Integration

Exploring the creative and technical process of rendering realistic images for integration into live action footage

The Virtual Worlds of Cinema Visual Effects, Simulation, and the Aesthetics of Cinematic Immersion

This thesis develops a phenomenology of immersive cinematic spectatorship. During an immersive experience in the cinema, the images, sounds, events, emotions, and characters that form a fictional diegesis become so compelling that our conscious experience of the real world is displaced by a virtual world. Theorists and audiences have long recognized cinema’s ability to momentarily substitute for the lived experience of reality, but it remains an under-theorized aspect of cinematic spectatorship. The first aim of this thesis is therefore to examine these immersive responses to cinema from three perspectives – the formal, the technological, and the neuroscientific – to describe the exact mechanisms through which a spectator’s immersion in a cinematic world is achieved. A second aim is to examine the historical development of the technologies of visual simulation that are used to create these immersive diegetic worlds. My analysis shows a consistent increase in the vividness and transparency of simulative technologies, two factors that are crucial determinants in a spectator’s immersion. In contrast to the cultural anxiety that often surrounds immersive responses to simulative technologies, I examine immersive spectatorship as an aesthetic phenomenon that is central to our engagement with cinema. The ubiquity of narrative – written, verbal, cinematic – shows that the ability to achieve immersion is a fundamental property of the human mind found in cultures diverse in both time and place. This thesis is thus an attempt to illuminate this unique human ability and examine the technologies that allow it to flourish

Lighting and Compositing for Project Hero

With tragedies such as the shooting at Sandy Hook Elementary on the rise in the last few years, brainstorming ideas that would prevent that type of violence in schools has become necessary. One such scenario involves a teleoperated robot, manned by local police, which would track down and subdue an intruder until police could arrive at the location. This thesis follows the process of creating a computer-generated robot and compositing it seamlessly into a real environment for the production, Project Hero. This project is a short film that shows a scenario in which a robot would be used to subdue a perpetrator. The production required the robot to appear photorealistic to convince the viewer that it was part of the live-action footage. The main components to accomplish photorealism included surfacing, lighting and compositing the robot

Hybrid visibility compositing and masking for illustrative rendering

In this paper, we introduce a novel framework for the compositing of interactively rendered 3D layers tailored to the needs of scientific illustration. Currently, traditional scientific illustrations are produced in a series of composition stages, combining different pictorial elements using 2D digital layering. Our approach extends the layer metaphor into 3D without giving up the advantages of 2D methods. The new compositing approach allows for effects such as selective transparency, occlusion overrides, and soft depth buffering. Furthermore, we show how common manipulation techniques such as masking can be integrated into this concept. These tools behave just like in 2D, but their influence extends beyond a single viewpoint. Since the presented approach makes no assumptions about the underlying rendering algorithms, layers can be generated based on polygonal geometry, volumetric data, point-based representations, or others. Our implementation exploits current graphics hardware and permits real-time interaction and rendering.publishedVersio

Seasons: A motion graphics depicts activities of ancient Chinese people in four seasons

Seasons is a 2D motion graphics film that aims to visually depict my interpretation of seasons changing and seasonal activities of ancient Chinese people. It is an attempt to reproduce two classical arts on modern media. I mimic and mix the visual effects of Chinese shadow puppets and Chinese ink wash strokes through digital media technologies. Neither Chinese painting nor shadow puppet emphasize on reproducing the appearance of subject. It is more regarded as a form of expressionistic art, which intends to capture the spirit of the subject and illustrate the perception of the scene. My graphics are not planned to fully copy external forms of shadow puppets or old Chinese paintings, instead, I target to show the beauty of Chinese traditional art forms as well as the beauty of Asian aesthetics and philosophy. Seasons represents my imaginary and visual adaptation of my knowledge and memories about cultures, traditions, and mysteries that related to seasons. The intent of project is to seek a different approach to integrate motion graphics with traditional art forms, and, to discuss more possibilities of transplanting classical arts into digital platform

A robust patch-based synthesis framework for combining inconsistent images

Current methods for combining different images produce visible artifacts when the sources have very different textures and structures, come from far view points, or capture dynamic scenes with motions. In this thesis, we propose a patch-based synthesis algorithm to plausibly combine different images that have color, texture, structural, and geometric inconsistencies. For some applications such as cloning and stitching where a gradual blend is required, we present a new method for synthesizing a transition region between two source images, such that inconsistent properties change gradually from one source to the other. We call this process image melding. For gradual blending, we generalized patch-based optimization foundation with three key generalizations: First, we enrich the patch search space with additional geometric and photometric transformations. Second, we integrate image gradients into the patch representation and replace the usual color averaging with a screened Poisson equation solver. Third, we propose a new energy based on mixed L2/L0 norms for colors and gradients that produces a gradual transition between sources without sacrificing texture sharpness. Together, all three generalizations enable patch-based solutions to a broad class of image melding problems involving inconsistent sources: object cloning, stitching challenging panoramas, hole filling from multiple photos, and image harmonization. We also demonstrate another application which requires us to address inconsistencies across the images: high dynamic range (HDR) reconstruction using sequential exposures. In this application, the results will suffer from objectionable artifacts for dynamic scenes if the inconsistencies caused by significant scene motions are not handled properly. In this thesis, we propose a new approach to HDR reconstruction that uses information in all exposures while being more robust to motion than previous techniques. Our algorithm is based on a novel patch-based energy-minimization formulation that integrates alignment and reconstruction in a joint optimization through an equation we call the HDR image synthesis equation. This allows us to produce an HDR result that is aligned to one of the exposures yet contains information from all of them. These two applications (image melding and high dynamic range reconstruction) show that patch based methods like the one proposed in this dissertation can address inconsistent images and could open the door to many new image editing applications in the future

A Collection of Digital Photo Editing Methods

Ph.DDOCTOR OF PHILOSOPH

Early Pioneers

Rochester, NY has a rich history of innovation and visionary ideas. Many people who hear or talk about Rochester outside of the area are not familiar with the people who called Rochester their home. These pioneers took advantage of the areas strategic placement in the United States and use it to help achieve great things. The use of a short motion graphics piece can help people get a taste of what some of the early pioneers achieved. Content, planning, and organization of this piece has been completed through research on four specific people (George Eastman, John Jacob Bausch, Henry Lomb, and Susan B. Anthony) and the times and areas of Rochester that they resided in. Early Pioneers takes the viewer briefly through each person and some of the accomplishments they made in their lives. The use of design programs Adobe After Effects, Adobe Photoshop, Autodesk Maya, and Maxon Cinema 4D as well as live shot video helped create a visual representation of these accomplishments. The project can be viewed at: earlypioneers.tumblr.co

Structure-Aware Image Compositing

The classic task of image compositing is complicated by the fact that the source and target images need to be carefully aligned and adjusted. Otherwise, it is not possible to achieve convincing results. Visual artifacts are caused by image intensity mismatch, image distortion or structure misalignment even if the images have been globally aligned. In this paper we extend classic Poisson blending by a constrained structure deformation and propagation method. This approach can solve the above-mentioned problems and proves useful for a variety of applications, e.g. in de-ghosting of mosaic images, classic image compositing or other applications such as superresolution from image databases. Our method is based on the following basic steps. First, an optimal partitioning boundary is computed between the input images. Then, features along this boundary are robustly aligned and deformation vectors are computed. Starting at these features, salient edges are traced and aligned, serving as additional constraints for the smooth deformation field, which is propagated robustly and smoothly into the interior of the target image. If very different images are to be stitched, we propose to base the deformation constraints on the curvature of the salient edges for C1-continuity of the structures between the images. We present results that show the robustness of our method on a number of image stitching and compositing tasks.Bildkompositionen werden oft dadurch erschwert, dass die zwei zusammenzufügenden Bilder vorsichtig und genau justiert und angepasst werden müssen. Andernfalls können keine überzeugenden Ergebnisse erreicht werden. Visuelle Artefakte können durch unterschiedliche Intensitäten, Verzerrungen oder strukturelle Missanpassungen entstehen, selbst wenn die Bilder global ausgerichtet wurden. In diesem Report wird das klassische Poisson Blending erweitert durch eine gezielte Strukturdeformation und Propagierung derselben. Dieser Ansatz löst die oben genannten Probleme und ist sinnvoll einsetzbar in einer Reihe von Anwendungen wie Mosaikbilder, Bildkompositionen, oder sogar Super-Resolution durch Bilddatenbanken. Die Methode basiert auf folgenden Schritten: Zunächst wir eine optimale Partitionierung berechnet zwischen den Eingabebildern, dann werden die wichtigen Features entlang dieser Grenze in Einklang gebracht und entsprechende Deformationsvektoren berechnet. Ausgehend von diesen Features werden die auffälligen Kanten weiterverfolgt und in Einklang gebracht. Diese Deformationen werden dann über das gesamte Bild ausgebreitet. Wenn die Bilder stark unterschiedlich sind schlagen wir in diesem Report vor die Verformung auf Basis der Krümmung der Kanten zu durchzuführen. Es werden Ergebnisse präsentiert, die die Robustheit der Methode zeigen

The Dupe

The story happens in the near future. Astronaut Patrick U7\u27s mission is to find members of a space team who have been lost for two months on a Planet M-6 expedition. He travels alone in a spacecraft headed for Planet M-6. Patrick U7 will confront his own fears and mortality when he discovers the truth about the space team\u27s fate and his real mission, which is linked to the deaths of the space team members. I want to use science fiction as a mechanism for conveying a truth about the darker side human nature. Even though the main character is a clone, he is nonetheless a real human being, and he represents those people who have been exploited for scientific experimentation or moral ideology. The Dupe is a thesis film with a total runtime of five minutes and fifteen seconds. It is a 3D animation that was produced primarily in Maya and After Effects. This paper outlines the entire creation process of making this animated film. It details the very beginning of the story development to the final stage. It describes all my intentions, obstacles, challenges and successes, as well as the technical specifics of process