32,336 research outputs found
Pose-Guided High-Resolution Appearance Transfer via Progressive Training
We propose a novel pose-guided appearance transfer network for transferring a
given reference appearance to a target pose in unprecedented image resolution
(1024 * 1024), given respectively an image of the reference and target person.
No 3D model is used. Instead, our network utilizes dense local descriptors
including local perceptual loss and local discriminators to refine details,
which is trained progressively in a coarse-to-fine manner to produce the
high-resolution output to faithfully preserve complex appearance of garment
textures and geometry, while hallucinating seamlessly the transferred
appearances including those with dis-occlusion. Our progressive encoder-decoder
architecture can learn the reference appearance inherent in the input image at
multiple scales. Extensive experimental results on the Human3.6M dataset, the
DeepFashion dataset, and our dataset collected from YouTube show that our model
produces high-quality images, which can be further utilized in useful
applications such as garment transfer between people and pose-guided human
video generation.Comment: 10 pages, 10 figures, 2 table
Bidirectionally Deformable Motion Modulation For Video-based Human Pose Transfer
Video-based human pose transfer is a video-to-video generation task that
animates a plain source human image based on a series of target human poses.
Considering the difficulties in transferring highly structural patterns on the
garments and discontinuous poses, existing methods often generate
unsatisfactory results such as distorted textures and flickering artifacts. To
address these issues, we propose a novel Deformable Motion Modulation (DMM)
that utilizes geometric kernel offset with adaptive weight modulation to
simultaneously perform feature alignment and style transfer. Different from
normal style modulation used in style transfer, the proposed modulation
mechanism adaptively reconstructs smoothed frames from style codes according to
the object shape through an irregular receptive field of view. To enhance the
spatio-temporal consistency, we leverage bidirectional propagation to extract
the hidden motion information from a warped image sequence generated by noisy
poses. The proposed feature propagation significantly enhances the motion
prediction ability by forward and backward propagation. Both quantitative and
qualitative experimental results demonstrate superiority over the
state-of-the-arts in terms of image fidelity and visual continuity. The source
code is publicly available at github.com/rocketappslab/bdmm.Comment: ICCV 202
- …