423 research outputs found
High-Resolution Deep Image Matting
Image matting is a key technique for image and video editing and composition.
Conventionally, deep learning approaches take the whole input image and an
associated trimap to infer the alpha matte using convolutional neural networks.
Such approaches set state-of-the-arts in image matting; however, they may fail
in real-world matting applications due to hardware limitations, since
real-world input images for matting are mostly of very high resolution. In this
paper, we propose HDMatt, a first deep learning based image matting approach
for high-resolution inputs. More concretely, HDMatt runs matting in a
patch-based crop-and-stitch manner for high-resolution inputs with a novel
module design to address the contextual dependency and consistency issues
between different patches. Compared with vanilla patch-based inference which
computes each patch independently, we explicitly model the cross-patch
contextual dependency with a newly-proposed Cross-Patch Contextual module (CPC)
guided by the given trimap. Extensive experiments demonstrate the effectiveness
of the proposed method and its necessity for high-resolution inputs. Our HDMatt
approach also sets new state-of-the-art performance on Adobe Image Matting and
AlphaMatting benchmarks and produce impressive visual results on more
real-world high-resolution images.Comment: AAAI 202
Natural Image Matting via Guided Contextual Attention
Over the last few years, deep learning based approaches have achieved
outstanding improvements in natural image matting. Many of these methods can
generate visually plausible alpha estimations, but typically yield blurry
structures or textures in the semitransparent area. This is due to the local
ambiguity of transparent objects. One possible solution is to leverage the
far-surrounding information to estimate the local opacity. Traditional
affinity-based methods often suffer from the high computational complexity,
which are not suitable for high resolution alpha estimation. Inspired by
affinity-based method and the successes of contextual attention in inpainting,
we develop a novel end-to-end approach for natural image matting with a guided
contextual attention module, which is specifically designed for image matting.
Guided contextual attention module directly propagates high-level opacity
information globally based on the learned low-level affinity. The proposed
method can mimic information flow of affinity-based methods and utilize rich
features learned by deep neural networks simultaneously. Experiment results on
Composition-1k testing set and alphamatting.com benchmark dataset demonstrate
that our method outperforms state-of-the-art approaches in natural image
matting. Code and models are available at
https://github.com/Yaoyi-Li/GCA-Matting.Comment: AAAI-2
FADE: Fusing the Assets of Decoder and Encoder for Task-Agnostic Upsampling
We consider the problem of task-agnostic feature upsampling in dense
prediction where an upsampling operator is required to facilitate both
region-sensitive tasks like semantic segmentation and detail-sensitive tasks
such as image matting. Existing upsampling operators often can work well in
either type of the tasks, but not both. In this work, we present FADE, a novel,
plug-and-play, and task-agnostic upsampling operator. FADE benefits from three
design choices: i) considering encoder and decoder features jointly in
upsampling kernel generation; ii) an efficient semi-shift convolutional
operator that enables granular control over how each feature point contributes
to upsampling kernels; iii) a decoder-dependent gating mechanism for enhanced
detail delineation. We first study the upsampling properties of FADE on toy
data and then evaluate it on large-scale semantic segmentation and image
matting. In particular, FADE reveals its effectiveness and task-agnostic
characteristic by consistently outperforming recent dynamic upsampling
operators in different tasks. It also generalizes well across convolutional and
transformer architectures with little computational overhead. Our work
additionally provides thoughtful insights on what makes for task-agnostic
upsampling. Code is available at: http://lnkiy.in/fade_inComment: Accepted to ECCV 2022. Code is available at http://lnkiy.in/fade_i
- …