133 research outputs found
Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis
In this paper, we propose a novel dual-branch Transformation-Synthesis
network (TS-Net), for video motion retargeting. Given one subject video and one
driving video, TS-Net can produce a new plausible video with the subject
appearance of the subject video and motion pattern of the driving video. TS-Net
consists of a warp-based transformation branch and a warp-free synthesis
branch. The novel design of dual branches combines the strengths of
deformation-grid-based transformation and warp-free generation for better
identity preservation and robustness to occlusion in the synthesized videos. A
mask-aware similarity module is further introduced to the transformation branch
to reduce computational overhead. Experimental results on face and dance
datasets show that TS-Net achieves better performance in video motion
retargeting than several state-of-the-art models as well as its single-branch
variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.Comment: WACV 202
Saliency-based image enhancement
Ph.DDOCTOR OF PHILOSOPH
Light field image processing: an overview
Light field imaging has emerged as a technology allowing to capture richer visual information from our world. As opposed to traditional photography, which captures a 2D projection of the light in the scene integrating the angular domain, light fields collect radiance from rays in all directions, demultiplexing the angular information lost in conventional photography. On the one hand, this higher dimensional representation of visual data offers powerful capabilities for scene understanding, and substantially improves the performance of traditional computer vision problems such as depth sensing, post-capture refocusing, segmentation, video stabilization, material classification, etc. On the other hand, the high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing, and display. Taking these two elements together, research in light field image processing has become increasingly popular in the computer vision, computer graphics, and signal processing communities. In this paper, we present a comprehensive overview and discussion of research in this field over the past 20 years. We focus on all aspects of light field image processing, including basic light field representation and theory, acquisition, super-resolution, depth estimation, compression, editing, processing algorithms for light field display, and computer vision applications of light field data
- …