5,941 research outputs found
Exploring the structure of a real-time, arbitrary neural artistic stylization network
In this paper, we present a method which combines the flexibility of the
neural algorithm of artistic style with the speed of fast style transfer
networks to allow real-time stylization using any content/style image pair. We
build upon recent work leveraging conditional instance normalization for
multi-style transfer networks by learning to predict the conditional instance
normalization parameters directly from a style image. The model is successfully
trained on a corpus of roughly 80,000 paintings and is able to generalize to
paintings previously unobserved. We demonstrate that the learned embedding
space is smooth and contains a rich structure and organizes semantic
information associated with paintings in an entirely unsupervised manner.Comment: Accepted as an oral presentation at British Machine Vision Conference
(BMVC) 201
Calipso: Physics-based Image and Video Editing through CAD Model Proxies
We present Calipso, an interactive method for editing images and videos in a
physically-coherent manner. Our main idea is to realize physics-based
manipulations by running a full physics simulation on proxy geometries given by
non-rigidly aligned CAD models. Running these simulations allows us to apply
new, unseen forces to move or deform selected objects, change physical
parameters such as mass or elasticity, or even add entire new objects that
interact with the rest of the underlying scene. In Calipso, the user makes
edits directly in 3D; these edits are processed by the simulation and then
transfered to the target 2D content using shape-to-image correspondences in a
photo-realistic rendering process. To align the CAD models, we introduce an
efficient CAD-to-image alignment procedure that jointly minimizes for rigid and
non-rigid alignment while preserving the high-level structure of the input
shape. Moreover, the user can choose to exploit image flow to estimate scene
motion, producing coherent physical behavior with ambient dynamics. We
demonstrate Calipso's physics-based editing on a wide range of examples
producing myriad physical behavior while preserving geometric and visual
consistency.Comment: 11 page
Automating joiners
Pictures taken from different view points cannot be stitched into a geometrically consistent mosaic, unless the structure of the scene is very special. However, geometrical consistency is not the only criterion for success: incorporating multiple view points into the same picture may produce compelling and informative representations. A multi viewpoint form of visual expression that has recently become highly popular is that of joiners (a term coined by artist David Hockney). Joiners are compositions where photographs are layered on a 2D canvas, with some photographs occluding others and boundaries fully visible.
Composing joiners is currently a tedious manual process, especially when a great number of photographs is involved. We are thus interested in automating their construction. Our approach is based on optimizing a cost function encouraging image-to-image consistency which is measured on point-features and along picture boundaries. The optimization looks for consistency in the 2D composition rather than 3D geometrical scene consistency and explicitly considers occlusion between pictures. We illustrate our ideas with a number of experiments on collections of images of objects, people, and outdoor scenes
Deep Photo Style Transfer
This paper introduces a deep-learning approach to photographic style transfer
that handles a large variety of image content while faithfully transferring the
reference style. Our approach builds upon the recent work on painterly transfer
that separates style from the content of an image by considering different
layers of a neural network. However, as is, this approach is not suitable for
photorealistic style transfer. Even when both the input and reference images
are photographs, the output still exhibits distortions reminiscent of a
painting. Our contribution is to constrain the transformation from the input to
the output to be locally affine in colorspace, and to express this constraint
as a custom fully differentiable energy term. We show that this approach
successfully suppresses distortion and yields satisfying photorealistic style
transfers in a broad variety of scenarios, including transfer of the time of
day, weather, season, and artistic edits
- …