3,978 research outputs found
Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN
Source at https://proceedings.neurips.cc/paper/2021/hash/151de84cca69258b17375e2f44239191-Abstract.html.Image-based virtual try-on is one of the most promising applications of human-centric image generation due to its tremendous real-world potential. Yet, as most try-on approaches fit in-shop garments onto a target person, they require the laborious and restrictive construction of a paired training dataset, severely limiting their scalability. While a few recent works attempt to transfer garments directly from one person to another, alleviating the need to collect paired datasets, their performance is impacted by the lack of paired (supervised) information. In particular, disentangling style and spatial information of the garment becomes a challenge, which existing methods either address by requiring auxiliary data or extensive online optimization procedures, thereby still inhibiting their scalability. To achieve a scalable virtual try-on system that can transfer arbitrary garments between a source and a target person in an unsupervised manner, we thus propose a texture-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN (PASTA-GAN), that facilitates real-world unpaired virtual try-on. Specifically, to disentangle the style and spatial information of each garment, PASTA-GAN consists of an innovative patch-routed disentanglement module for successfully retaining garment texture and shape characteristics. Guided by the source person's keypoints, the patch-routed disentanglement module first decouples garments into normalized patches, thus eliminating the inherent spatial information of the garment, and then reconstructs the normalized patches to the warped garment complying with the target person pose. Given the warped garment, PASTA-GAN further introduces novel spatially-adaptive residual blocks that guide the generator to synthesize more realistic garment details. Extensive comparisons with paired and unpaired approaches demonstrate the superiority of PASTA-GAN, highlighting its ability to generate high-quality try-on images when faced with a large variety of garments(e.g. vests, shirts, pants), taking a crucial step towards real-world scalable try-on
DI-Net : Decomposed Implicit Garment Transfer Network for Digital Clothed 3D Human
3D virtual try-on enjoys many potential applications and hence has attracted
wide attention. However, it remains a challenging task that has not been
adequately solved. Existing 2D virtual try-on methods cannot be directly
extended to 3D since they lack the ability to perceive the depth of each pixel.
Besides, 3D virtual try-on approaches are mostly built on the fixed topological
structure and with heavy computation. To deal with these problems, we propose a
Decomposed Implicit garment transfer network (DI-Net), which can effortlessly
reconstruct a 3D human mesh with the newly try-on result and preserve the
texture from an arbitrary perspective. Specifically, DI-Net consists of two
modules: 1) A complementary warping module that warps the reference image to
have the same pose as the source image through dense correspondence learning
and sparse flow learning; 2) A geometry-aware decomposed transfer module that
decomposes the garment transfer into image layout based transfer and texture
based transfer, achieving surface and texture reconstruction by constructing
pixel-aligned implicit functions. Experimental results show the effectiveness
and superiority of our method in the 3D virtual try-on task, which can yield
more high-quality results over other existing methods
Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis
Deep person generation has attracted extensive research attention due to its
wide applications in virtual agents, video conferencing, online shopping and
art/movie production. With the advancement of deep learning, visual appearances
(face, pose, cloth) of a person image can be easily generated or manipulated on
demand. In this survey, we first summarize the scope of person generation, and
then systematically review recent progress and technical trends in deep person
generation, covering three major tasks: talking-head generation (face),
pose-guided person generation (pose) and garment-oriented person generation
(cloth). More than two hundred papers are covered for a thorough overview, and
the milestone works are highlighted to witness the major technical
breakthrough. Based on these fundamental tasks, a number of applications are
investigated, e.g., virtual fitting, digital human, generative data
augmentation. We hope this survey could shed some light on the future prospects
of deep person generation, and provide a helpful foundation for full
applications towards digital human
PG-VTON: A Novel Image-Based Virtual Try-On Method via Progressive Inference Paradigm
Virtual try-on is a promising computer vision topic with a high commercial
value wherein a new garment is visually worn on a person with a photo-realistic
effect. Previous studies conduct their shape and content inference at one
stage, employing a single-scale warping mechanism and a relatively
unsophisticated content inference mechanism. These approaches have led to
suboptimal results in terms of garment warping and skin reservation under
challenging try-on scenarios. To address these limitations, we propose a novel
virtual try-on method via progressive inference paradigm (PGVTON) that
leverages a top-down inference pipeline and a general garment try-on strategy.
Specifically, we propose a robust try-on parsing inference method by
disentangling semantic categories and introducing consistency. Exploiting the
try-on parsing as the shape guidance, we implement the garment try-on via
warping-mapping-composition. To facilitate adaptation to a wide range of try-on
scenarios, we adopt a covering more and selecting one warping strategy and
explicitly distinguish tasks based on alignment. Additionally, we regulate
StyleGAN2 to implement re-naked skin inpainting, conditioned on the target skin
shape and spatial-agnostic skin features. Experiments demonstrate that our
method has state-of-the-art performance under two challenging scenarios. The
code will be available at https://github.com/NerdFNY/PGVTON
SieveNet: A Unified Framework for Robust Image-Based Virtual Try-On
Image-based virtual try-on for fashion has gained considerable attention
recently. The task requires trying on a clothing item on a target model image.
An efficient framework for this is composed of two stages: (1) warping
(transforming) the try-on cloth to align with the pose and shape of the target
model, and (2) a texture transfer module to seamlessly integrate the warped
try-on cloth onto the target model image. Existing methods suffer from
artifacts and distortions in their try-on output. In this work, we present
SieveNet, a framework for robust image-based virtual try-on. Firstly, we
introduce a multi-stage coarse-to-fine warping network to better model
fine-grained intricacies (while transforming the try-on cloth) and train it
with a novel perceptual geometric matching loss. Next, we introduce a try-on
cloth conditioned segmentation mask prior to improve the texture transfer
network. Finally, we also introduce a dueling triplet loss strategy for
training the texture translation network which further improves the quality of
the generated try-on results. We present extensive qualitative and quantitative
evaluations of each component of the proposed pipeline and show significant
performance improvements against the current state-of-the-art method.Comment: Accepted at IEEE WACV 202
- …