50,678 research outputs found
PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain
We propose a universal image reconstruction method to represent detailed
images purely from binary sparse edge and flat color domain. Inspired by the
procedures of painting, our framework, based on generative adversarial network,
consists of three phases: Imitation Phase aims at initializing networks,
followed by Generating Phase to reconstruct preliminary images. Moreover,
Refinement Phase is utilized to fine-tune preliminary images into final outputs
with details. This framework allows our model generating abundant high
frequency details from sparse input information. We also explore the defects of
disentangling style latent space implicitly from images, and demonstrate that
explicit color domain in our model performs better on controllability and
interpretability. In our experiments, we achieve outstanding results on
reconstructing realistic images and translating hand drawn drafts into
satisfactory paintings. Besides, within the domain of edge-to-image
translation, our model PI-REC outperforms existing state-of-the-art methods on
evaluations of realism and accuracy, both quantitatively and qualitatively.Comment: 15 pages, 13 figure
A Flexible Convolutional Solver with Application to Photorealistic Style Transfer
We propose a new flexible deep convolutional neural network (convnet) to
perform fast visual style transfer. In contrast to existing convnets that
address the same task, our architecture derives directly from the structure of
the gradient descent originally used to solve the style transfer problem [Gatys
et al., 2016]. Like existing convnets, ours approximately solves the original
problem much faster than the gradient descent. However, our network is uniquely
flexible by design: it can be manipulated at runtime to enforce new constraints
on the final solution. In particular, we show how to modify it to obtain a
photorealistic result with no retraining. We study the modifications made by
[Luan et al., 2017] to the original cost function of [Gatys et al., 2016] to
achieve photorealistic style transfer. These modifications affect directly the
gradient descent and can be reported on-the-fly in our network. These
modifications are possible as the proposed architecture stems from unrolling
the gradient descent
Photo Stylistic Brush: Robust Style Transfer via Superpixel-Based Bipartite Graph
With the rapid development of social network and multimedia technology,
customized image and video stylization has been widely used for various
social-media applications. In this paper, we explore the problem of
exemplar-based photo style transfer, which provides a flexible and convenient
way to invoke fantastic visual impression. Rather than investigating some fixed
artistic patterns to represent certain styles as was done in some previous
works, our work emphasizes styles related to a series of visual effects in the
photograph, e.g. color, tone, and contrast. We propose a photo stylistic brush,
an automatic robust style transfer approach based on Superpixel-based BIpartite
Graph (SuperBIG). A two-step bipartite graph algorithm with different
granularity levels is employed to aggregate pixels into superpixels and find
their correspondences. In the first step, with the extracted hierarchical
features, a bipartite graph is constructed to describe the content similarity
for pixel partition to produce superpixels. In the second step, superpixels in
the input/reference image are rematched to form a new superpixel-based
bipartite graph, and superpixel-level correspondences are generated by a
bipartite matching. Finally, the refined correspondence guides SuperBIG to
perform the transformation in a decorrelated color space. Extensive
experimental results demonstrate the effectiveness and robustness of the
proposed method for transferring various styles of exemplar images, even for
some challenging cases, such as night images
Mask-Guided Portrait Editing with Conditional GANs
Portrait editing is a popular subject in photo manipulation. The Generative
Adversarial Network (GAN) advances the generating of realistic faces and allows
more face editing. In this paper, we argue about three issues in existing
techniques: diversity, quality, and controllability for portrait synthesis and
editing. To address these issues, we propose a novel end-to-end learning
framework that leverages conditional GANs guided by provided face masks for
generating faces. The framework learns feature embeddings for every face
component (e.g., mouth, hair, eye), separately, contributing to better
correspondences for image translation, and local face editing. With the mask,
our network is available to many applications, like face synthesis driven by
mask, face Swap+ (including hair in swapping), and local manipulation. It can
also boost the performance of face parsing a bit as an option of data
augmentation.Comment: To appear in CVPR201
Image Inpainting using Block-wise Procedural Training with Annealed Adversarial Counterpart
Recent advances in deep generative models have shown promising potential in
image inpanting, which refers to the task of predicting missing pixel values of
an incomplete image using the known context. However, existing methods can be
slow or generate unsatisfying results with easily detectable flaws. In
addition, there is often perceivable discontinuity near the holes and require
further post-processing to blend the results. We present a new approach to
address the difficulty of training a very deep generative model to synthesize
high-quality photo-realistic inpainting. Our model uses conditional generative
adversarial networks (conditional GANs) as the backbone, and we introduce a
novel block-wise procedural training scheme to stabilize the training while we
increase the network depth. We also propose a new strategy called adversarial
loss annealing to reduce the artifacts. We further describe several losses
specifically designed for inpainting and show their effectiveness. Extensive
experiments and user-study show that our approach outperforms existing methods
in several tasks such as inpainting, face completion and image harmonization.
Finally, we show our framework can be easily used as a tool for interactive
guided inpainting, demonstrating its practical value to solve common real-world
challenges
Superimposition-guided Facial Reconstruction from Skull
We develop a new algorithm to perform facial reconstruction from a given
skull. This technique has forensic application in helping the identification of
skeletal remains when other information is unavailable. Unlike most existing
strategies that directly reconstruct the face from the skull, we utilize a
database of portrait photos to create many face candidates, then perform a
superimposition to get a well matched face, and then revise it according to the
superimposition. To support this pipeline, we build an effective autoencoder
for image-based facial reconstruction, and a generative model for constrained
face inpainting. Our experiments have demonstrated that the proposed pipeline
is stable and accurate.Comment: 14 pages; 14 figure
Computational Parquetry: Fabricated Style Transfer with Wood Pixels
Parquetry is the art and craft of decorating a surface with a pattern of
differently colored veneers of wood, stone or other materials. Traditionally,
the process of designing and making parquetry has been driven by color, using
the texture found in real wood only for stylization or as a decorative effect.
Here, we introduce a computational pipeline that draws from the rich natural
structure of strongly textured real-world veneers as a source of detail in
order to approximate a target image as faithfully as possible using a
manageable number of parts. This challenge is closely related to the
established problems of patch-based image synthesis and stylization in some
ways, but fundamentally different in others. Most importantly, the limited
availability of resources (any piece of wood can only be used once) turns the
relatively simple problem of finding the right piece for the target location
into the combinatorial problem of finding optimal parts while avoiding resource
collisions. We introduce an algorithm that allows to efficiently solve an
approximation to the problem. It further addresses challenges like gamut
mapping, feature characterization and the search for fabricable cuts. We
demonstrate the effectiveness of the system by fabricating a selection of
"photo-realistic" pieces of parquetry from different kinds of unstained wood
veneer
Real-Time User-Guided Image Colorization with Learned Deep Priors
We propose a deep learning approach for user-guided image colorization. The
system directly maps a grayscale image, along with sparse, local user "hints"
to an output colorization with a Convolutional Neural Network (CNN). Rather
than using hand-defined rules, the network propagates user edits by fusing
low-level cues along with high-level semantic information, learned from
large-scale data. We train on a million images, with simulated user inputs. To
guide the user towards efficient input selection, the system recommends likely
colors based on the input image and current user inputs. The colorization is
performed in a single feed-forward pass, enabling real-time use. Even with
randomly simulated user inputs, we show that the proposed system helps novice
users quickly create realistic colorizations, and offers large improvements in
colorization quality with just a minute of use. In addition, we demonstrate
that the framework can incorporate other user "hints" to the desired
colorization, showing an application to color histogram transfer. Our code and
models are available at https://richzhang.github.io/ideepcolor.Comment: Accepted to SIGGRAPH 2017. Project page:
https://richzhang.github.io/ideepcolo
Arbitrary Style Transfer via Multi-Adaptation Network
Arbitrary style transfer is a significant topic with research value and
application prospect. A desired style transfer, given a content image and
referenced style painting, would render the content image with the color tone
and vivid stroke patterns of the style painting while synchronously maintaining
the detailed content structure information. Style transfer approaches would
initially learn content and style representations of the content and style
references and then generate the stylized images guided by these
representations. In this paper, we propose the multi-adaptation network which
involves two self-adaptation (SA) modules and one co-adaptation (CA) module:
the SA modules adaptively disentangle the content and style representations,
i.e., content SA module uses position-wise self-attention to enhance content
representation and style SA module uses channel-wise self-attention to enhance
style representation; the CA module rearranges the distribution of style
representation based on content representation distribution by calculating the
local similarity between the disentangled content and style features in a
non-local fashion. Moreover, a new disentanglement loss function enables our
network to extract main style patterns and exact content structures to adapt to
various input images, respectively. Various qualitative and quantitative
experiments demonstrate that the proposed multi-adaptation network leads to
better results than the state-of-the-art style transfer methods
Visual Attribute Transfer through Deep Image Analogy
We propose a new technique for visual attribute transfer across images that
may have very different appearance but have perceptually similar semantic
structure. By visual attribute transfer, we mean transfer of visual information
(such as color, tone, texture, and style) from one image to another. For
example, one image could be that of a painting or a sketch while the other is a
photo of a real scene, and both depict the same type of scene.
Our technique finds semantically-meaningful dense correspondences between two
input images. To accomplish this, it adapts the notion of "image analogy" with
features extracted from a Deep Convolutional Neutral Network for matching; we
call our technique Deep Image Analogy. A coarse-to-fine strategy is used to
compute the nearest-neighbor field for generating the results. We validate the
effectiveness of our proposed method in a variety of cases, including
style/texture transfer, color/style swap, sketch/painting to photo, and time
lapse.Comment: Accepted by SIGGRAPH 201
- …