38 research outputs found
์ด์ ์คํ์์ 3D ๊น์ด ์ฌ๊ตฌ์ฑ ๋ฐ ๊น์ด ๊ฐ์
ํ์๋
ผ๋ฌธ (๋ฐ์ฌ) -- ์์ธ๋ํ๊ต ๋ํ์ : ๊ณต๊ณผ๋ํ ์ ๊ธฐยท์ปดํจํฐ๊ณตํ๋ถ, 2021. 2. ์ ์๊ธธ.Three-dimensional (3D) depth recovery from two-dimensional images is a fundamental and challenging objective in computer vision, and is one of the most important prerequisites for many applications such as 3D measurement, robot location and navigation, self-driving, and so on. Depth-from-focus (DFF) is one of the important methods to reconstruct a 3D depth in the use of focus information. Reconstructing a 3D depth from texture-less regions is a typical issue associated with the conventional DFF. Further more, it is difficult for the conventional DFF reconstruction techniques to preserve depth edges and fine details while maintaining spatial consistency. In this dissertation, we address these problems and propose an DFF depth recovery framework which is robust over texture-less regions, and can reconstruct a depth image with clear edges and fine details.
The depth recovery framework proposed in this dissertation is composed of two processes: depth reconstruction and depth refinement. To recovery an accurate 3D depth, We first formulate the depth reconstruction as a maximum a posterior (MAP) estimation problem with the inclusion of matting Laplacian prior. The nonlocal principle is adopted during the construction stage of the matting Laplacian matrix to preserve depth edges and fine details. Additionally, a depth variance based confidence measure with the combination of the reliability measure of focus measure is proposed to maintain the spatial smoothness, such that the smooth depth regions in initial depth could have high confidence value and the reconstructed depth could be more derived from the initial depth. As the nonlocal principle breaks the spatial consistency, the reconstructed depth image is spatially inconsistent. Meanwhile, it suffers from texture-copy artifacts. To smooth the noise and suppress the texture-copy artifacts introduced in the reconstructed depth image, we propose a closed-form edge-preserving depth refinement algorithm that formulates the depth refinement as a MAP estimation problem using Markov random fields (MRFs). With the incorporation of pre-estimated depth edges and mutual structure information into our energy function and the specially designed smoothness weight, the proposed refinement method can effectively suppress noise and texture-copy artifacts while preserving depth edges. Additionally, with the construction of undirected weighted graph representing the energy function, a closed-form solution is obtained by using the Laplacian matrix corresponding to the graph.
The proposed framework presents a novel method of 3D depth recovery from a focal stack. The proposed algorithm shows the superiority in depth recovery over texture-less regions owing to the effective variance based confidence level computation and the matting Laplacian prior. Additionally, this proposed reconstruction method can obtain a depth image with clear edges and fine details due to the adoption of nonlocal principle in the construct]ion of matting Laplacian matrix. The proposed closed-form depth refinement approach shows that the ability in noise removal while preserving object structure with the usage of common edges. Additionally, it is able to effectively suppress texture-copy artifacts by utilizing mutual structure information. The proposed depth refinement provides a general idea for edge-preserving image smoothing, especially for depth related refinement such as stereo vision.
Both quantitative and qualitative experimental results show the supremacy of the proposed method in terms of robustness in texture-less regions, accuracy, and ability to preserve object structure while maintaining spatial smoothness.Chapter 1 Introduction 1
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Chapter 2 Related Works 9
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Principle of depth-from-focus . . . . . . . . . . . . . . . . . . . . 9
2.2.1 Focus measure operators . . . . . . . . . . . . . . . . . . . 12
2.3 Depth-from-focus reconstruction . . . . . . . . . . . . . . . . . . 14
2.4 Edge-preserving image denoising . . . . . . . . . . . . . . . . . . 23
Chapter 3 Depth-from-Focus Reconstruction using Nonlocal Matting Laplacian Prior 38
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2 Image matting and matting Laplacian . . . . . . . . . . . . . . . 40
3.3 Depth-from-focus . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.4 Depth reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.4.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . 47
3.4.2 Likelihood model . . . . . . . . . . . . . . . . . . . . . . . 48
3.4.3 Nonlocal matting Laplacian prior model . . . . . . . . . . 50
3.5 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5.2 Data configuration . . . . . . . . . . . . . . . . . . . . . . 55
3.5.3 Reconstruction results . . . . . . . . . . . . . . . . . . . . 56
3.5.4 Comparison between reconstruction using local and nonlocal matting Laplacian . . . . . . . . . . . . . . . . . . . 56
3.5.5 Spatial consistency analysis . . . . . . . . . . . . . . . . . 59
3.5.6 Parameter setting and analysis . . . . . . . . . . . . . . . 59
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Chapter 4 Closed-form MRF-based Depth Refinement 63
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.3 Closed-form solution . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.4 Edge preservation . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.5 Texture-copy artifacts suppression . . . . . . . . . . . . . . . . . 73
4.6 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Chapter 5 Evaluation 82
5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.2 Evaluation metrics . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.3 Evaluation on synthetic datasets . . . . . . . . . . . . . . . . . . 84
5.4 Evaluation on real scene datasets . . . . . . . . . . . . . . . . . . 89
5.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.6 Computational performances . . . . . . . . . . . . . . . . . . . . 93
Chapter 6 Conclusion 96
Bibliography 99Docto
Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimation
Natural image matting is an important problem in computer vision and
graphics. It is an ill-posed problem when only an input image is available
without any external information. While the recent deep learning approaches
have shown promising results, they only estimate the alpha matte. This paper
presents a context-aware natural image matting method for simultaneous
foreground and alpha matte estimation. Our method employs two encoder networks
to extract essential information for matting. Particularly, we use a matting
encoder to learn local features and a context encoder to obtain more global
context information. We concatenate the outputs from these two encoders and
feed them into decoder networks to simultaneously estimate the foreground and
alpha matte. To train this whole deep neural network, we employ both the
standard Laplacian loss and the feature loss: the former helps to achieve high
numerical performance while the latter leads to more perceptually plausible
results. We also report several data augmentation strategies that greatly
improve the network's generalization performance. Our qualitative and
quantitative experiments show that our method enables high-quality matting for
a single natural image. Our inference codes and models have been made publicly
available at https://github.com/hqqxyy/Context-Aware-Matting.Comment: This is the camera ready version of ICCV2019 pape
Graph Spectral Image Processing
Recent advent of graph signal processing (GSP) has spurred intensive studies
of signals that live naturally on irregular data kernels described by graphs
(e.g., social networks, wireless sensor networks). Though a digital image
contains pixels that reside on a regularly sampled 2D grid, if one can design
an appropriate underlying graph connecting pixels with weights that reflect the
image structure, then one can interpret the image (or image patch) as a signal
on a graph, and apply GSP tools for processing and analysis of the signal in
graph spectral domain. In this article, we overview recent graph spectral
techniques in GSP specifically for image / video processing. The topics covered
include image compression, image restoration, image filtering and image
segmentation
Privileged Prior Information Distillation for Image Matting
Performance of trimap-free image matting methods is limited when trying to
decouple the deterministic and undetermined regions, especially in the scenes
where foregrounds are semantically ambiguous, chromaless, or high
transmittance. In this paper, we propose a novel framework named Privileged
Prior Information Distillation for Image Matting (PPID-IM) that can effectively
transfer privileged prior environment-aware information to improve the
performance of students in solving hard foregrounds. The prior information of
trimap regulates only the teacher model during the training stage, while not
being fed into the student network during actual inference. In order to achieve
effective privileged cross-modality (i.e. trimap and RGB) information
distillation, we introduce a Cross-Level Semantic Distillation (CLSD) module
that reinforces the trimap-free students with more knowledgeable semantic
representations and environment-aware information. We also propose an
Attention-Guided Local Distillation module that efficiently transfers
privileged local attributes from the trimap-based teacher to trimap-free
students for the guidance of local-region optimization. Extensive experiments
demonstrate the effectiveness and superiority of our PPID framework on the task
of image matting. In addition, our trimap-free IndexNet-PPID surpasses the
other competing state-of-the-art methods by a large margin, especially in
scenarios with chromaless, weak texture, or irregular objects.Comment: 15 pages, 7 figure