A Temporally Coherent Neural Algorithm for Artistic Style Transfer

Within the fields of visual effects and animation, humans have historically spent countless painstaking hours mastering the skill of drawing frame-by-frame animations. One such animation technique that has been widely used in the animation and visual effects industry is called rotoscoping and has allowed uniquely stylized animations to capture the motion of real life action sequences, however it is a very complex and time consuming process. Automating this arduous technique would free animators from performing frame by frame stylization and allow them to concentrate on their own artistic contributions. This thesis introduces a new artificial system based on an existing neural style transfer method which creates artistically stylized animations that simultaneously reproduce both the motion of the original videos that they are derived from and the unique style of a given artistic work. This system utilizes a convolutional neural network framework to extract a hierarchy of image features used for generating images that appear visually similar to a given artistic style while at the same time faithfully preserving temporal content. The use of optical flow allows the combination of style and content to be integrated directly with the apparent motion over frames of a video to produce smooth and visually appealing transitions. The implementation described in this thesis demonstrates how biologically-inspired systems such as convolutional neural networks are rapidly approaching human-level behavior in tasks that were once thought impossible for computers. Such a complex task elucidates the current and future technical and artistic capabilities of such biologically-inspired neural systems as their horizons expand exponentially. Further, this research provides unique insights into the way that humans perceive and utilize temporal information in everyday tasks. A secondary implementation that is explored in this thesis seeks to improve existing convolutional neural networks using a biological approach to the way these models adapt to their inputs. This implementation shows how these pattern recognition systems can be greatly improved by integrating recent neuroscience research into already biologically inspired systems. Such a novel hybrid activation function model replicates recent findings in the field of neuroscience and shows significant advantages over existing static activation functions

A Natural Image Pointillism with Controlled Ellipse Dots

This paper presents an image-based artistic rendering algorithm for the automatic Pointillism style. At first, ellipse dot locations are randomly generated based on a source image; then dot orientations are precalculated with help of a direction map; a saliency map of the source image decides long and short radius of the ellipse dot. At last, the rendering runs layer-by-layer from large size dots to small size dots so as to reserve the detailed parts of the image. Although only ellipse dot shape is adopted, the final Pointillism style performs well because of variable characteristics of the dot

Structure Preserving regularizer for Neural Style Transfer

The aim of the project is to generate an image in the style of the image by a well-known artist. The experiment will use artificial neural networks to transfer the style of one image onto another. In Computer Vision context: capturing the content invariant that is the style of an image and applying it on the content of another image. Initially captures the tensors that we need from the content and style image and then we pass the input image which will initially be an image with noise and our algorithm will try to minimize the loss between the input and content image and that between input and style image thus capturing the essence of both the images into one. The traditional method of style transfer generated image has an artistic effect that is the model successfully capture the style of the image but does not preserve the structural content of the image. The proposed method uses a segmented version of images to faithfully transfer the style to semantic similar content. Also, a regularizer term modified in loss function that helps in avoiding style spill over and have photographic results

Entwicklung eines semi-automatischen Workflows zur Ableitung ikonographischer Kartenzeichen

Die Verwendung von ikonographischen, bildhaften Kartenzeichen ist sehr beliebt bei der Darstellung von Sehenswürdigkeiten in touristischen Karten sowie bei Kartendarstellungen für Kinder und Jugendliche. Der Begriff des Non-Photorealistic Rendering (NPR) beschreibt einen zentralen Bereich in der Computergrafik, der sich mit der Erzeugung von Bildern auseinandersetzt, die scheinbar handgemacht sind und bewusst nicht dem physikalisch korrekten Abbild eines Modells entsprechen. Ein weiteres Trendthema zur Nachahmung eines bestimmten Stils eines Kunstwerks stellt der Neural Style Transfer (NST) dar. Hierbei werden künstlerische Bilder durch Trennung und Rekombination von Bildinhalt und Stil erzeugt. Im Rahmen der vorliegenden Arbeit ist ein semi-automatischer Workflow zur Erzeugung ikonographischer Gebäudedarstellungen für die Nutzung in zoombaren Webkarten entwickelt und in drei künstlerischen Stilvarianten unter Nutzung von Bildverarbeitungswerkzeugen in dem rasterbasierten Open Source Bildbearbeitungsprogramm GIMP, speziell mit der Filtersammlung G'MIC technisch umgesetzt worden. Außerdem zeigt die Masterarbeit das Potential der Ableitung von ikonographischen Signaturen durch den Style-Transfer mittels neuronaler Netze.:Selbstständigkeitserklärung III Inhaltsverzeichnis 5 Abbildungsverzeichnis 7 Tabellenverzeichnis 8 Abkürzungsverzeichnis 9 1 Einleitung 10 1.1 Motivation 10 1.2 Gliederung der Arbeit 10 2 Literaturstudium 11 2.1 Computergrafik 11 2.2 Non-Photorealistic Rendering 11 2.3 Neural Style Transfer 14 2.3.1 Einleitung 14 2.3.2 Convolutional Neural Network 15 2.3.3 Beschreibung des Algorithmus 17 3 Methodik 19 3.1 Technische Komponenten 19 3.2 Kriterien der Bildauswahl 19 3.3 Workflow „Ölmalerei“ 21 3.4 Workflow „Tuschezeichnung 22 3.5 Workflow „Silhouette“ 22 4 Praktischer Teil 23 4.1 Konkrete Umsetzung 23 4.1.1 Workflow „Ölmalerei“ 24 4.1.2 Workflow „Tuschezeichnung“ 32 4.1.3 Workflow „Silhouette“ 32 4.2 Implementierung eines Automatisierungsprozesses 35 4.3 Anwendung: Karte Dresden 39 4.4 Neural Style Transfer 43 4.4.1 Online-Anwendungen 43 4.4.2 Offline-Implementierung 45 5 Diskussion 51 5.1 Resultate 51 5.1.1 Bildverarbeitung 51 5.1.2 Neural Style Transfer 51 5.2 Ausblick 52 6 Zusammenfassung 52 Literaturverzeichnis 53The use of iconographic, pictorial map symbols is very popular for the representation of places of interest in tourist maps as well as for map presentations for children and young people. The term Non-Photorealistic Rendering (NPR) describes a prominent field in computer graphics that deals with the generation of images that are apparently handmade and deliberately do not correspond to the physically correct image of a model. Neural Style Transfer (NST) is another trend topic for imitating a certain style of an artwork. Here, artistic images are created by separating and recombining image content and style. In the context of the present work, a semi-automatic workflow for the creation of iconographic building representations for use in zoomable web maps has been developed and technically implemented in three artistic style variants using image processing tools in the raster-based open source image processing program GIMP, especially with the filter collection G'MIC. In addition, the master thesis demonstrates the potential of deriving iconographic signatures through style transfer using neural networks.:Selbstständigkeitserklärung III Inhaltsverzeichnis 5 Abbildungsverzeichnis 7 Tabellenverzeichnis 8 Abkürzungsverzeichnis 9 1 Einleitung 10 1.1 Motivation 10 1.2 Gliederung der Arbeit 10 2 Literaturstudium 11 2.1 Computergrafik 11 2.2 Non-Photorealistic Rendering 11 2.3 Neural Style Transfer 14 2.3.1 Einleitung 14 2.3.2 Convolutional Neural Network 15 2.3.3 Beschreibung des Algorithmus 17 3 Methodik 19 3.1 Technische Komponenten 19 3.2 Kriterien der Bildauswahl 19 3.3 Workflow „Ölmalerei“ 21 3.4 Workflow „Tuschezeichnung 22 3.5 Workflow „Silhouette“ 22 4 Praktischer Teil 23 4.1 Konkrete Umsetzung 23 4.1.1 Workflow „Ölmalerei“ 24 4.1.2 Workflow „Tuschezeichnung“ 32 4.1.3 Workflow „Silhouette“ 32 4.2 Implementierung eines Automatisierungsprozesses 35 4.3 Anwendung: Karte Dresden 39 4.4 Neural Style Transfer 43 4.4.1 Online-Anwendungen 43 4.4.2 Offline-Implementierung 45 5 Diskussion 51 5.1 Resultate 51 5.1.1 Bildverarbeitung 51 5.1.2 Neural Style Transfer 51 5.2 Ausblick 52 6 Zusammenfassung 52 Literaturverzeichnis 5