5,795 research outputs found
Fast Deep Matting for Portrait Animation on Mobile Phone
Image matting plays an important role in image and video editing. However,
the formulation of image matting is inherently ill-posed. Traditional methods
usually employ interaction to deal with the image matting problem with trimaps
and strokes, and cannot run on the mobile phone in real-time. In this paper, we
propose a real-time automatic deep matting approach for mobile devices. By
leveraging the densely connected blocks and the dilated convolution, a light
full convolutional network is designed to predict a coarse binary mask for
portrait images. And a feathering block, which is edge-preserving and matting
adaptive, is further developed to learn the guided filter and transform the
binary mask into alpha matte. Finally, an automatic portrait animation system
based on fast deep matting is built on mobile devices, which does not need any
interaction and can realize real-time matting with 15 fps. The experiments show
that the proposed approach achieves comparable results with the
state-of-the-art matting solvers.Comment: ACM Multimedia Conference (MM) 2017 camera-read
Deep Image Matting: A Comprehensive Survey
Image matting refers to extracting precise alpha matte from natural images,
and it plays a critical role in various downstream applications, such as image
editing. Despite being an ill-posed problem, traditional methods have been
trying to solve it for decades. The emergence of deep learning has
revolutionized the field of image matting and given birth to multiple new
techniques, including automatic, interactive, and referring image matting. This
paper presents a comprehensive review of recent advancements in image matting
in the era of deep learning. We focus on two fundamental sub-tasks: auxiliary
input-based image matting, which involves user-defined input to predict the
alpha matte, and automatic image matting, which generates results without any
manual intervention. We systematically review the existing methods for these
two tasks according to their task settings and network structures and provide a
summary of their advantages and disadvantages. Furthermore, we introduce the
commonly used image matting datasets and evaluate the performance of
representative matting methods both quantitatively and qualitatively. Finally,
we discuss relevant applications of image matting and highlight existing
challenges and potential opportunities for future research. We also maintain a
public repository to track the rapid development of deep image matting at
https://github.com/JizhiziLi/matting-survey
3D Matting: A Soft Segmentation Method Applied in Computed Tomography
Three-dimensional (3D) images, such as CT, MRI, and PET, are common in
medical imaging applications and important in clinical diagnosis. Semantic
ambiguity is a typical feature of many medical image labels. It can be caused
by many factors, such as the imaging properties, pathological anatomy, and the
weak representation of the binary masks, which brings challenges to accurate 3D
segmentation. In 2D medical images, using soft masks instead of binary masks
generated by image matting to characterize lesions can provide rich semantic
information, describe the structural characteristics of lesions more
comprehensively, and thus benefit the subsequent diagnoses and analyses. In
this work, we introduce image matting into the 3D scenes to describe the
lesions in 3D medical images. The study of image matting in 3D modality is
limited, and there is no high-quality annotated dataset related to 3D matting,
therefore slowing down the development of data-driven deep-learning-based
methods. To address this issue, we constructed the first 3D medical matting
dataset and convincingly verified the validity of the dataset through quality
control and downstream experiments in lung nodules classification. We then
adapt the four selected state-of-the-art 2D image matting algorithms to 3D
scenes and further customize the methods for CT images. Also, we propose the
first end-to-end deep 3D matting network and implement a solid 3D medical image
matting benchmark, which will be released to encourage further research.Comment: 12 pages, 7 figure
High-Resolution Deep Image Matting
Image matting is a key technique for image and video editing and composition.
Conventionally, deep learning approaches take the whole input image and an
associated trimap to infer the alpha matte using convolutional neural networks.
Such approaches set state-of-the-arts in image matting; however, they may fail
in real-world matting applications due to hardware limitations, since
real-world input images for matting are mostly of very high resolution. In this
paper, we propose HDMatt, a first deep learning based image matting approach
for high-resolution inputs. More concretely, HDMatt runs matting in a
patch-based crop-and-stitch manner for high-resolution inputs with a novel
module design to address the contextual dependency and consistency issues
between different patches. Compared with vanilla patch-based inference which
computes each patch independently, we explicitly model the cross-patch
contextual dependency with a newly-proposed Cross-Patch Contextual module (CPC)
guided by the given trimap. Extensive experiments demonstrate the effectiveness
of the proposed method and its necessity for high-resolution inputs. Our HDMatt
approach also sets new state-of-the-art performance on Adobe Image Matting and
AlphaMatting benchmarks and produce impressive visual results on more
real-world high-resolution images.Comment: AAAI 202
Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimation
Natural image matting is an important problem in computer vision and
graphics. It is an ill-posed problem when only an input image is available
without any external information. While the recent deep learning approaches
have shown promising results, they only estimate the alpha matte. This paper
presents a context-aware natural image matting method for simultaneous
foreground and alpha matte estimation. Our method employs two encoder networks
to extract essential information for matting. Particularly, we use a matting
encoder to learn local features and a context encoder to obtain more global
context information. We concatenate the outputs from these two encoders and
feed them into decoder networks to simultaneously estimate the foreground and
alpha matte. To train this whole deep neural network, we employ both the
standard Laplacian loss and the feature loss: the former helps to achieve high
numerical performance while the latter leads to more perceptually plausible
results. We also report several data augmentation strategies that greatly
improve the network's generalization performance. Our qualitative and
quantitative experiments show that our method enables high-quality matting for
a single natural image. Our inference codes and models have been made publicly
available at https://github.com/hqqxyy/Context-Aware-Matting.Comment: This is the camera ready version of ICCV2019 pape
Natural Image Matting via Guided Contextual Attention
Over the last few years, deep learning based approaches have achieved
outstanding improvements in natural image matting. Many of these methods can
generate visually plausible alpha estimations, but typically yield blurry
structures or textures in the semitransparent area. This is due to the local
ambiguity of transparent objects. One possible solution is to leverage the
far-surrounding information to estimate the local opacity. Traditional
affinity-based methods often suffer from the high computational complexity,
which are not suitable for high resolution alpha estimation. Inspired by
affinity-based method and the successes of contextual attention in inpainting,
we develop a novel end-to-end approach for natural image matting with a guided
contextual attention module, which is specifically designed for image matting.
Guided contextual attention module directly propagates high-level opacity
information globally based on the learned low-level affinity. The proposed
method can mimic information flow of affinity-based methods and utilize rich
features learned by deep neural networks simultaneously. Experiment results on
Composition-1k testing set and alphamatting.com benchmark dataset demonstrate
that our method outperforms state-of-the-art approaches in natural image
matting. Code and models are available at
https://github.com/Yaoyi-Li/GCA-Matting.Comment: AAAI-2
- …