Exploring the structure of a real-time, arbitrary neural artistic stylization network

In this paper, we present a method which combines the flexibility of the neural algorithm of artistic style with the speed of fast style transfer networks to allow real-time stylization using any content/style image pair. We build upon recent work leveraging conditional instance normalization for multi-style transfer networks by learning to predict the conditional instance normalization parameters directly from a style image. The model is successfully trained on a corpus of roughly 80,000 paintings and is able to generalize to paintings previously unobserved. We demonstrate that the learned embedding space is smooth and contains a rich structure and organizes semantic information associated with paintings in an entirely unsupervised manner.Comment: Accepted as an oral presentation at British Machine Vision Conference (BMVC) 201

A jigsaw puzzle framework for homogenization of high porosity foams

An approach to homogenization of high porosity metallic foams is explored. The emphasis is on the \Alporas{} foam and its representation by means of two-dimensional wire-frame models. The guaranteed upper and lower bounds on the effective properties are derived by the first-order homogenization with the uniform and minimal kinematic boundary conditions at heart. This is combined with the method of Wang tilings to generate sufficiently large material samples along with their finite element discretization. The obtained results are compared to experimental and numerical data available in literature and the suitability of the two-dimensional setting itself is discussed.Comment: 11 pages, 7 figures, 3 table

Implicit Decals: Interactive Editing of Repetitive Patterns on Surfaces

11 pagesInternational audienceTexture mapping is an essential component for creating 3D models and is widely used in both the game and the movie industries. Creating texture maps has always been a complex task and existing methods carefully balance flexibility with ease of use. One difficulty in using texturing is the repeated placement of individual textures over larger areas. In this paper we propose a method which uses decals to place images onto a model. Our method allows the decals to compete for space and to deform as they are being pushed by other decals. A spherical field function is used to determine the position and the size of each decal and the deformation applied to fit the decals. The decals may span multiple objects with heterogeneous representations. Our method does not require an explicit parameterization of the model. As such, varieties of patterns including repeated patterns like rocks, tiles, and scales can be mapped. We have implemented the method using the GPU where placement, size, and orientation of thousands of decals are manipulated in real time

Performance Analysis of Different Applications of Image Inpainting Based on Exemplar Technique

In this age of rapidly developing image processing, inpainting has been a popular and practical art. Researchers have paid considerable attention to image inpainting throughout the years due to its enormous significance and effectiveness in a wide range of image processing applications, including the removal of scratches, the elimination of objects, and the modification of faces. It is one of the most challenging issues in image processing, demanding a comprehensive understanding of the image's texture and structure. The quality of inpainted image is a crucial factor which determines how close the inpainted image is to the original image. Many improvements have been implemented in the exemplar-based approach to increase the quality of inpainted regions containing structure and texture information. There are numerous ways to assess the quality of an inpainted image. In this study, the applications of exemplar based inpainting are evaluated using standard analytical measures including Sum of Absolute Difference (SAD), Peak Signal-to-Noise Ratio (PSNR), Correlation Coefficient, and Structural Similarity Index Measure (SSIM)

A Panorama on Multiscale Geometric Representations, Intertwining Spatial, Directional and Frequency Selectivity

The richness of natural images makes the quest for optimal representations in image processing and computer vision challenging. The latter observation has not prevented the design of image representations, which trade off between efficiency and complexity, while achieving accurate rendering of smooth regions as well as reproducing faithful contours and textures. The most recent ones, proposed in the past decade, share an hybrid heritage highlighting the multiscale and oriented nature of edges and patterns in images. This paper presents a panorama of the aforementioned literature on decompositions in multiscale, multi-orientation bases or dictionaries. They typically exhibit redundancy to improve sparsity in the transformed domain and sometimes its invariance with respect to simple geometric deformations (translation, rotation). Oriented multiscale dictionaries extend traditional wavelet processing and may offer rotation invariance. Highly redundant dictionaries require specific algorithms to simplify the search for an efficient (sparse) representation. We also discuss the extension of multiscale geometric decompositions to non-Euclidean domains such as the sphere or arbitrary meshed surfaces. The etymology of panorama suggests an overview, based on a choice of partially overlapping "pictures". We hope that this paper will contribute to the appreciation and apprehension of a stream of current research directions in image understanding.Comment: 65 pages, 33 figures, 303 reference

Fast inpainting algorithm for real-time video inpainting problem

The paper examines a simple and efficient method to solve the digital inpainting problem with a reasonable result by processing the information locally around the painting area. The method is based on a unique matrix transformation algorithm. It can guarantee transforming a non-negative matrix without rows and columns of all zero elements into another matrix with the same size but having both its column and row products equal to 1. The method is time and memory efficient so it can be used in many real time systems like video stream which may have protential inpainting problems

Design of decorative 3D models: from geodesic ornaments to tangible assemblies

L'obiettivo di questa tesi è sviluppare strumenti utili per creare opere d'arte decorative digitali in 3D. Uno dei processi decorativi più comunemente usati prevede la creazione di pattern decorativi, al fine di abbellire gli oggetti. Questi pattern possono essere dipinti sull'oggetto di base o realizzati con l'applicazione di piccoli elementi decorativi. Tuttavia, la loro realizzazione nei media digitali non è banale. Da un lato, gli utenti esperti possono eseguire manualmente la pittura delle texture o scolpire ogni decorazione, ma questo processo può richiedere ore per produrre un singolo pezzo e deve essere ripetuto da zero per ogni modello da decorare. D'altra parte, gli approcci automatici allo stato dell'arte si basano sull'approssimazione di questi processi con texturing basato su esempi o texturing procedurale, o con sistemi di riproiezione 3D. Tuttavia, questi approcci possono introdurre importanti limiti nei modelli utilizzabili e nella qualità dei risultati. Il nostro lavoro sfrutta invece i recenti progressi e miglioramenti delle prestazioni nel campo dell'elaborazione geometrica per creare modelli decorativi direttamente sulle superfici. Presentiamo una pipeline per i pattern 2D e una per quelli 3D, e dimostriamo come ognuna di esse possa ricreare una vasta gamma di risultati con minime modifiche dei parametri. Inoltre, studiamo la possibilità di creare modelli decorativi tangibili. I pattern 3D generati possono essere stampati in 3D e applicati a oggetti realmente esistenti precedentemente scansionati. Discutiamo anche la creazione di modelli con mattoncini da costruzione, e la possibilità di mescolare mattoncini standard e mattoncini custom stampati in 3D. Ciò consente una rappresentazione precisa indipendentemente da quanto la voxelizzazione sia approssimativa. I principali contributi di questa tesi sono l'implementazione di due diverse pipeline decorative, un approccio euristico alla costruzione con mattoncini e un dataset per testare quest'ultimo.The aim of this thesis is to develop effective tools to create digital decorative 3D artworks. Real-world art often involves the use of decorative patterns to enrich objects. These patterns can be painted on the base or might be realized with the application of small decorative elements. However, their creation in digital media is not trivial. On the one hand, users can manually perform texture paint or sculpt each decoration, in a process that can take hours to produce a single piece and needs to be repeated from the ground up for every model that needs to be decorated. On the other hand, automatic approaches in state of the art rely on approximating these processes with procedural or by-example texturing or with 3D reprojection. However, these approaches can introduce significant limitations in the models that can be used and in the quality of the results. Instead, our work exploits the recent advances and performance improvements in the geometry processing field to create decorative patterns directly on surfaces. We present a pipeline for 2D and one for 3D patterns and demonstrate how each of them can recreate a variety of results with minimal tweaking of the parameters. Furthermore, we investigate the possibility of creating decorative tangible models. The 3D patterns we generate can be 3D printed and applied to previously scanned real-world objects. We also discuss the creation of models with standard building bricks and the possibility of mixing standard and custom 3D-printed bricks. This allows for a precise representation regardless of the coarseness of the voxelization. The main contributions of this thesis are the implementation of two different decorative pipelines, a heuristic approach to brick construction, and a dataset to test the latter

Distributed texture-based terrain synthesis

Terrain synthesis is an important field of Computer Graphics that deals with the generation of 3D landscape models for use in virtual environments. The field has evolved to a stage where large and even infinite landscapes can be generated in realtime. However, user control of the generation process is still minimal, as well as the creation of virtual landscapes that mimic real terrain. This thesis investigates the use of texture synthesis techniques on real landscapes to improve realism and the use of sketch-based interfaces to enable intuitive user control

Stereological techniques for synthesizing solid textures from images of aggregate materials

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2005.Includes bibliographical references (leaves 121-130).When creating photorealistic digital scenes, textures are commonly used to depict complex variation in surface appearance. For materials that have spatial variation in three dimensions, such as wood or marble, solid textures offer a natural representation. Unlike 2D textures, which can be easily captured with a photograph, it can be difficult to obtain a 3D material volume. This thesis addresses the challenge of extrapolating tileable 3D solid textures from images of aggregate materials, such as concrete, asphalt, terrazzo or granite. The approach introduced here is inspired by and builds on prior work in stereology--the study of 3D properties of a material based on 2D observations. Unlike ad hoc methods for texture synthesis, this approach has rigorous mathematical foundations that allow for reliable, accurate material synthesis with well-defined assumptions. The algorithm is also driven by psychophysical constraints to insure that slices through the synthesized volume have a perceptually similar appearance to the input image. The texture synthesis algorithm uses a variety of techniques to independently solve for the shape, distribution, and color of the embedded particles, as well as the residual noise. To approximate particle shape, I consider four methods-including two algorithms of my own contribution. I compare these methods under a variety of input conditions using automated, perceptually-motivated metrics as well as a carefully controlled psychophysical experiment. In addition to assessing the relative performance of the four algorithms, I also evaluate the reliability of the automated metrics in predicting the results of the user study. To solve for the particle distribution, I apply traditional stereological methods.(cont.) I first illustrate this approach for aggregate materials of spherical particles and then extend the technique to apply to particles of arbitrary shapes. The particle shape and distribution are used in conjunction to create an explicit 3D material volume using simulated annealing. Particle colors are assigned using a stochastic method, and high-frequency noise is replicated with the assistance of existing algorithms. The data representation is suitable for high-fidelity rendering and physical simulation. I demonstrate the effectiveness of the approach with side-by-side comparisons of real materials and their synthetic counterparts derived from the application of these techniques.by Robert Carl Jagnow.Ph.D