329 research outputs found
Fast Deep Matting for Portrait Animation on Mobile Phone
Image matting plays an important role in image and video editing. However,
the formulation of image matting is inherently ill-posed. Traditional methods
usually employ interaction to deal with the image matting problem with trimaps
and strokes, and cannot run on the mobile phone in real-time. In this paper, we
propose a real-time automatic deep matting approach for mobile devices. By
leveraging the densely connected blocks and the dilated convolution, a light
full convolutional network is designed to predict a coarse binary mask for
portrait images. And a feathering block, which is edge-preserving and matting
adaptive, is further developed to learn the guided filter and transform the
binary mask into alpha matte. Finally, an automatic portrait animation system
based on fast deep matting is built on mobile devices, which does not need any
interaction and can realize real-time matting with 15 fps. The experiments show
that the proposed approach achieves comparable results with the
state-of-the-art matting solvers.Comment: ACM Multimedia Conference (MM) 2017 camera-read
High-Resolution Deep Image Matting
Image matting is a key technique for image and video editing and composition.
Conventionally, deep learning approaches take the whole input image and an
associated trimap to infer the alpha matte using convolutional neural networks.
Such approaches set state-of-the-arts in image matting; however, they may fail
in real-world matting applications due to hardware limitations, since
real-world input images for matting are mostly of very high resolution. In this
paper, we propose HDMatt, a first deep learning based image matting approach
for high-resolution inputs. More concretely, HDMatt runs matting in a
patch-based crop-and-stitch manner for high-resolution inputs with a novel
module design to address the contextual dependency and consistency issues
between different patches. Compared with vanilla patch-based inference which
computes each patch independently, we explicitly model the cross-patch
contextual dependency with a newly-proposed Cross-Patch Contextual module (CPC)
guided by the given trimap. Extensive experiments demonstrate the effectiveness
of the proposed method and its necessity for high-resolution inputs. Our HDMatt
approach also sets new state-of-the-art performance on Adobe Image Matting and
AlphaMatting benchmarks and produce impressive visual results on more
real-world high-resolution images.Comment: AAAI 202
Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimation
Natural image matting is an important problem in computer vision and
graphics. It is an ill-posed problem when only an input image is available
without any external information. While the recent deep learning approaches
have shown promising results, they only estimate the alpha matte. This paper
presents a context-aware natural image matting method for simultaneous
foreground and alpha matte estimation. Our method employs two encoder networks
to extract essential information for matting. Particularly, we use a matting
encoder to learn local features and a context encoder to obtain more global
context information. We concatenate the outputs from these two encoders and
feed them into decoder networks to simultaneously estimate the foreground and
alpha matte. To train this whole deep neural network, we employ both the
standard Laplacian loss and the feature loss: the former helps to achieve high
numerical performance while the latter leads to more perceptually plausible
results. We also report several data augmentation strategies that greatly
improve the network's generalization performance. Our qualitative and
quantitative experiments show that our method enables high-quality matting for
a single natural image. Our inference codes and models have been made publicly
available at https://github.com/hqqxyy/Context-Aware-Matting.Comment: This is the camera ready version of ICCV2019 pape
Rethinking Context Aggregation in Natural Image Matting
For natural image matting, context information plays a crucial role in
estimating alpha mattes especially when it is challenging to distinguish
foreground from its background. Exiting deep learning-based methods exploit
specifically designed context aggregation modules to refine encoder features.
However, the effectiveness of these modules has not been thoroughly explored.
In this paper, we conduct extensive experiments to reveal that the context
aggregation modules are actually not as effective as expected. We also
demonstrate that when learned on large image patches, basic encoder-decoder
networks with a larger receptive field can effectively aggregate context to
achieve better performance.Upon the above findings, we propose a simple yet
effective matting network, named AEMatter, which enlarges the receptive field
by incorporating an appearance-enhanced axis-wise learning block into the
encoder and adopting a hybrid-transformer decoder. Experimental results on four
datasets demonstrate that our AEMatter significantly outperforms
state-of-the-art matting methods (e.g., on the Adobe Composition-1K dataset,
\textbf{25\%} and \textbf{40\%} reduction in terms of SAD and MSE,
respectively, compared against MatteFormer). The code and model are available
at \url{https://github.com/QLYoo/AEMatter}
- …