5,172 research outputs found
Image interpolation with edge-preserving differential motion refinement
Motion estimation (ME) methods based on differential techniques provide useful information for video analysis, and moreover it is relatively easy to embed into them regularity constraints en- forcing for example, contour preservation. On the other hand, these techniques are rarely employed for video compression since, though accurate, the dense motion vector field (MVF) they produce requires too much coding resource and computational effort. However, this kind of algorithm could be useful in the framework of distributed video coding (DVC), where the motion vector are computed at the decoder side, so that no bit-rate is needed to transmit them. More- over usually the decoder has enough computational power to face with the increased complexity of differential ME. In this paper we introduce a new image interpolation algorithm to be used in the context of DVC. This algorithm combines a popular DVC technique with differential ME. We adapt a pel-recursive differential ME algorithm to the DVC context; moreover we insert a regularity constraint which allows more consistent MVFs. The experimental results are encouraging: the quality of interpolated images is improved of up to 1.1 dB w.r.t. to state-of-the-art techniques. These results prove to be consistent when we use different GOP sizes
초점 스택에서 3D 깊이 재구성 및 깊이 개선
학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·컴퓨터공학부, 2021. 2. 신영길.Three-dimensional (3D) depth recovery from two-dimensional images is a fundamental and challenging objective in computer vision, and is one of the most important prerequisites for many applications such as 3D measurement, robot location and navigation, self-driving, and so on. Depth-from-focus (DFF) is one of the important methods to reconstruct a 3D depth in the use of focus information. Reconstructing a 3D depth from texture-less regions is a typical issue associated with the conventional DFF. Further more, it is difficult for the conventional DFF reconstruction techniques to preserve depth edges and fine details while maintaining spatial consistency. In this dissertation, we address these problems and propose an DFF depth recovery framework which is robust over texture-less regions, and can reconstruct a depth image with clear edges and fine details.
The depth recovery framework proposed in this dissertation is composed of two processes: depth reconstruction and depth refinement. To recovery an accurate 3D depth, We first formulate the depth reconstruction as a maximum a posterior (MAP) estimation problem with the inclusion of matting Laplacian prior. The nonlocal principle is adopted during the construction stage of the matting Laplacian matrix to preserve depth edges and fine details. Additionally, a depth variance based confidence measure with the combination of the reliability measure of focus measure is proposed to maintain the spatial smoothness, such that the smooth depth regions in initial depth could have high confidence value and the reconstructed depth could be more derived from the initial depth. As the nonlocal principle breaks the spatial consistency, the reconstructed depth image is spatially inconsistent. Meanwhile, it suffers from texture-copy artifacts. To smooth the noise and suppress the texture-copy artifacts introduced in the reconstructed depth image, we propose a closed-form edge-preserving depth refinement algorithm that formulates the depth refinement as a MAP estimation problem using Markov random fields (MRFs). With the incorporation of pre-estimated depth edges and mutual structure information into our energy function and the specially designed smoothness weight, the proposed refinement method can effectively suppress noise and texture-copy artifacts while preserving depth edges. Additionally, with the construction of undirected weighted graph representing the energy function, a closed-form solution is obtained by using the Laplacian matrix corresponding to the graph.
The proposed framework presents a novel method of 3D depth recovery from a focal stack. The proposed algorithm shows the superiority in depth recovery over texture-less regions owing to the effective variance based confidence level computation and the matting Laplacian prior. Additionally, this proposed reconstruction method can obtain a depth image with clear edges and fine details due to the adoption of nonlocal principle in the construct]ion of matting Laplacian matrix. The proposed closed-form depth refinement approach shows that the ability in noise removal while preserving object structure with the usage of common edges. Additionally, it is able to effectively suppress texture-copy artifacts by utilizing mutual structure information. The proposed depth refinement provides a general idea for edge-preserving image smoothing, especially for depth related refinement such as stereo vision.
Both quantitative and qualitative experimental results show the supremacy of the proposed method in terms of robustness in texture-less regions, accuracy, and ability to preserve object structure while maintaining spatial smoothness.Chapter 1 Introduction 1
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Chapter 2 Related Works 9
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Principle of depth-from-focus . . . . . . . . . . . . . . . . . . . . 9
2.2.1 Focus measure operators . . . . . . . . . . . . . . . . . . . 12
2.3 Depth-from-focus reconstruction . . . . . . . . . . . . . . . . . . 14
2.4 Edge-preserving image denoising . . . . . . . . . . . . . . . . . . 23
Chapter 3 Depth-from-Focus Reconstruction using Nonlocal Matting Laplacian Prior 38
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2 Image matting and matting Laplacian . . . . . . . . . . . . . . . 40
3.3 Depth-from-focus . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.4 Depth reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.4.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . 47
3.4.2 Likelihood model . . . . . . . . . . . . . . . . . . . . . . . 48
3.4.3 Nonlocal matting Laplacian prior model . . . . . . . . . . 50
3.5 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5.2 Data configuration . . . . . . . . . . . . . . . . . . . . . . 55
3.5.3 Reconstruction results . . . . . . . . . . . . . . . . . . . . 56
3.5.4 Comparison between reconstruction using local and nonlocal matting Laplacian . . . . . . . . . . . . . . . . . . . 56
3.5.5 Spatial consistency analysis . . . . . . . . . . . . . . . . . 59
3.5.6 Parameter setting and analysis . . . . . . . . . . . . . . . 59
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Chapter 4 Closed-form MRF-based Depth Refinement 63
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.3 Closed-form solution . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.4 Edge preservation . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.5 Texture-copy artifacts suppression . . . . . . . . . . . . . . . . . 73
4.6 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Chapter 5 Evaluation 82
5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.2 Evaluation metrics . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.3 Evaluation on synthetic datasets . . . . . . . . . . . . . . . . . . 84
5.4 Evaluation on real scene datasets . . . . . . . . . . . . . . . . . . 89
5.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.6 Computational performances . . . . . . . . . . . . . . . . . . . . 93
Chapter 6 Conclusion 96
Bibliography 99Docto
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
We present a compact but effective CNN model for optical flow, called
PWC-Net. PWC-Net has been designed according to simple and well-established
principles: pyramidal processing, warping, and the use of a cost volume. Cast
in a learnable feature pyramid, PWC-Net uses the cur- rent optical flow
estimate to warp the CNN features of the second image. It then uses the warped
features and features of the first image to construct a cost volume, which is
processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller in
size and easier to train than the recent FlowNet2 model. Moreover, it
outperforms all published optical flow methods on the MPI Sintel final pass and
KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution (1024x436)
images. Our models are available on https://github.com/NVlabs/PWC-Net.Comment: CVPR 2018 camera ready version (with github link to Caffe and PyTorch
code
PVR: Patch-to-Volume Reconstruction for Large Area Motion Correction of Fetal MRI
In this paper we present a novel method for the correction of motion
artifacts that are present in fetal Magnetic Resonance Imaging (MRI) scans of
the whole uterus. Contrary to current slice-to-volume registration (SVR)
methods, requiring an inflexible anatomical enclosure of a single investigated
organ, the proposed patch-to-volume reconstruction (PVR) approach is able to
reconstruct a large field of view of non-rigidly deforming structures. It
relaxes rigid motion assumptions by introducing a specific amount of redundant
information that is exploited with parallelized patch-wise optimization,
super-resolution, and automatic outlier rejection. We further describe and
provide an efficient parallel implementation of PVR allowing its execution
within reasonable time on commercially available graphics processing units
(GPU), enabling its use in the clinical practice. We evaluate PVR's
computational overhead compared to standard methods and observe improved
reconstruction accuracy in presence of affine motion artifacts of approximately
30% compared to conventional SVR in synthetic experiments. Furthermore, we have
evaluated our method qualitatively and quantitatively on real fetal MRI data
subject to maternal breathing and sudden fetal movements. We evaluate
peak-signal-to-noise ratio (PSNR), structural similarity index (SSIM), and
cross correlation (CC) with respect to the originally acquired data and provide
a method for visual inspection of reconstruction uncertainty. With these
experiments we demonstrate successful application of PVR motion compensation to
the whole uterus, the human fetus, and the human placenta.Comment: 10 pages, 13 figures, submitted to IEEE Transactions on Medical
Imaging. v2: wadded funders acknowledgements to preprin
High-order Discretization of a Gyrokinetic Vlasov Model in Edge Plasma Geometry
We present a high-order spatial discretization of a continuum gyrokinetic
Vlasov model in axisymmetric tokamak edge plasma geometries. Such models
describe the phase space advection of plasma species distribution functions in
the absence of collisions. The gyrokinetic model is posed in a four-dimensional
phase space, upon which a grid is imposed when discretized. To mitigate the
computational cost associated with high-dimensional grids, we employ a
high-order discretization to reduce the grid size needed to achieve a given
level of accuracy relative to lower-order methods. Strong anisotropy induced by
the magnetic field motivates the use of mapped coordinate grids aligned with
magnetic flux surfaces. The natural partitioning of the edge geometry by the
separatrix between the closed and open field line regions leads to the
consideration of multiple mapped blocks, in what is known as a mapped
multiblock (MMB) approach. We describe the specialization of a more general
formalism that we have developed for the construction of high-order,
finite-volume discretizations on MMB grids, yielding the accurate evaluation of
the gyrokinetic Vlasov operator, the metric factors resulting from the MMB
coordinate mappings, and the interaction of blocks at adjacent boundaries. Our
conservative formulation of the gyrokinetic Vlasov model incorporates the fact
that the phase space velocity has zero divergence, which must be preserved
discretely to avoid truncation error accumulation. We describe an approach for
the discrete evaluation of the gyrokinetic phase space velocity that preserves
the divergence-free property to machine precision
Calipso: Physics-based Image and Video Editing through CAD Model Proxies
We present Calipso, an interactive method for editing images and videos in a
physically-coherent manner. Our main idea is to realize physics-based
manipulations by running a full physics simulation on proxy geometries given by
non-rigidly aligned CAD models. Running these simulations allows us to apply
new, unseen forces to move or deform selected objects, change physical
parameters such as mass or elasticity, or even add entire new objects that
interact with the rest of the underlying scene. In Calipso, the user makes
edits directly in 3D; these edits are processed by the simulation and then
transfered to the target 2D content using shape-to-image correspondences in a
photo-realistic rendering process. To align the CAD models, we introduce an
efficient CAD-to-image alignment procedure that jointly minimizes for rigid and
non-rigid alignment while preserving the high-level structure of the input
shape. Moreover, the user can choose to exploit image flow to estimate scene
motion, producing coherent physical behavior with ambient dynamics. We
demonstrate Calipso's physics-based editing on a wide range of examples
producing myriad physical behavior while preserving geometric and visual
consistency.Comment: 11 page
- …