2,000 research outputs found
Context-Aware Semantic Inpainting
IEEE In recent times, image inpainting has witnessed rapid progress due to the generative adversarial networks (GANs) that are able to synthesize realistic contents. However, most existing GAN-based methods for semantic inpainting apply an auto-encoder architecture with a fully connected layer, which cannot accurately maintain spatial information. In addition, the discriminator in existing GANs struggles to comprehend high-level semantics within the image context and yields semantically consistent content. Existing evaluation criteria are biased toward blurry results and cannot well characterize edge preservation and visual authenticity in the inpainting results. In this paper, we propose an improved GAN to overcome the aforementioned limitations. Our proposed GAN-based framework consists of a fully convolutional design for the generator which helps to better preserve spatial structures and a joint loss function with a revised perceptual loss to capture high-level semantics in the context. Furthermore, we also introduce two novel measures to better assess the quality of image inpainting results. The experimental results demonstrate that our method outperforms the state-of-the-art under a wide range of criteria
Geometry-Aware Face Completion and Editing
Face completion is a challenging generation task because it requires
generating visually pleasing new pixels that are semantically consistent with
the unmasked face region. This paper proposes a geometry-aware Face Completion
and Editing NETwork (FCENet) by systematically studying facial geometry from
the unmasked region. Firstly, a facial geometry estimator is learned to
estimate facial landmark heatmaps and parsing maps from the unmasked face
image. Then, an encoder-decoder structure generator serves to complete a face
image and disentangle its mask areas conditioned on both the masked face image
and the estimated facial geometry images. Besides, since low-rank property
exists in manually labeled masks, a low-rank regularization term is imposed on
the disentangled masks, enforcing our completion network to manage occlusion
area with various shape and size. Furthermore, our network can generate diverse
results from the same masked input by modifying estimated facial geometry,
which provides a flexible mean to edit the completed face appearance. Extensive
experimental results qualitatively and quantitatively demonstrate that our
network is able to generate visually pleasing face completion results and edit
face attributes as well
Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid
Restoring reasonable and realistic content for arbitrary missing regions in
images is an important yet challenging task. Although recent image inpainting
models have made significant progress in generating vivid visual details, they
can still lead to texture blurring or structural distortions due to contextual
ambiguity when dealing with more complex scenes. To address this issue, we
propose the Semantic Pyramid Network (SPN) motivated by the idea that learning
multi-scale semantic priors from specific pretext tasks can greatly benefit the
recovery of locally missing content in images. SPN consists of two components.
First, it distills semantic priors from a pretext model into a multi-scale
feature pyramid, achieving a consistent understanding of the global context and
local structures. Within the prior learner, we present an optional module for
variational inference to realize probabilistic image inpainting driven by
various learned priors. The second component of SPN is a fully context-aware
image generator, which adaptively and progressively refines low-level visual
representations at multiple scales with the (stochastic) prior pyramid. We
train the prior learner and the image generator as a unified model without any
post-processing. Our approach achieves the state of the art on multiple
datasets, including Places2, Paris StreetView, CelebA, and CelebA-HQ, under
both deterministic and probabilistic inpainting setups.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessibl
- …