8 research outputs found
R-MNet: A Perceptual Adversarial Network for Image Inpainting
Facial image inpainting is a problem that is widely studied, and in recent
years the introduction of Generative Adversarial Networks, has led to
improvements in the field. Unfortunately some issues persists, in particular
when blending the missing pixels with the visible ones. We address the problem
by proposing a Wasserstein GAN combined with a new reverse mask operator,
namely Reverse Masking Network (R-MNet), a perceptual adversarial network for
image inpainting. The reverse mask operator transfers the reverse masked image
to the end of the encoder-decoder network leaving only valid pixels to be
inpainted. Additionally, we propose a new loss function computed in feature
space to target only valid pixels combined with adversarial training. These
then capture data distributions and generate images similar to those in the
training data with achieved realism (realistic and coherent) on the output
images. We evaluate our method on publicly available dataset, and compare with
state-of-the-art methods. We show that our method is able to generalize to
high-resolution inpainting task, and further show more realistic outputs that
are plausible to the human visual system when compared with the
state-of-the-art methods.Comment: 10 pages, 7 figures, 3 table
Edge Guided GANs with Semantic Preserving for Semantic Image Synthesis
We propose a novel Edge guided Generative Adversarial Network (EdgeGAN) for
photo-realistic image synthesis from semantic layouts. Although considerable
improvement has been achieved, the quality of synthesized images is far from
satisfactory due to two largely unresolved challenges. First, the semantic
labels do not provide detailed structural information, making it difficult to
synthesize local details and structures. Second, the widely adopted CNN
operations such as convolution, down-sampling and normalization usually cause
spatial resolution loss and thus are unable to fully preserve the original
semantic information, leading to semantically inconsistent results (e.g.,
missing small objects). To tackle the first challenge, we propose to use the
edge as an intermediate representation which is further adopted to guide image
generation via a proposed attention guided edge transfer module. Edge
information is produced by a convolutional generator and introduces detailed
structure information. Further, to preserve the semantic information, we design
an effective module to selectively highlight class-dependent feature maps
according to the original semantic layout. Extensive experiments on two
challenging datasets show that the proposed EdgeGAN can generate significantly
better results than state-of-the-art methods. The source code and trained
models are available at https://github.com/Ha0Tang/EdgeGAN.Comment: 40 pages, 29 figure
Review on Image Inpainting using Intelligence Mining Techniques
Objective Inpainting is a technique for fixing or removing undesired areas of an image. Methods In present scenario, image plays a vital role in every aspect such as business images, satellite images, and medical images and so on. Results and Conclusion This paper presents a comprehensive review of past traditional image inpainting methods and the present state-of-the-art deep learning methods and also detailed the strengths and weaknesses of each to provide new insights in the field
Theoretical Deep Learning
Deep learning has long been criticised as a black-box model for lacking sound theoretical explanation. During the PhD course, I explore and establish theoretical foundations for deep learning. In this thesis, I present my contributions positioned upon existing literature: (1) analysing the generalizability of the neural networks with residual connections via complexity and capacity-based hypothesis complexity measures; (2) modeling stochastic gradient descent (SGD) by stochastic differential equations (SDEs) and their dynamics, and further characterizing the generalizability of deep learning; (3) understanding the geometrical structures of the loss landscape that drives the trajectories of the dynamic systems, which sheds light in reconciling the over-representation and excellent generalizability of deep learning; and (4) discovering the interplay between generalization, privacy preservation, and adversarial robustness, which have seen rising concerns in deep learning deployment