46 research outputs found
A Geometric Approach to Color Image Regularization
We present a new vectorial total variation method that addresses the problem
of color consistent image filtering. Our approach is inspired from the
double-opponent cell representation in the human visual cortex. Existing
methods of vectorial total variation regularizers have insufficient (or no)
coupling between the color channels and thus may introduce color artifacts. We
address this problem by introducing a novel coupling between the color channels
related to a pullback-metric from the opponent space to the data (RGB color)
space. Our energy is a non-convex, non-smooth higher-order vectorial total
variation approach and promotes color consistent image filtering via a coupling
term. For a convex variant, we show well-posedness and existence of a solution
in the space of vectorial bounded variation. For the higher-order scheme we
employ a half-quadratic strategy, which model the non-convex energy terms as
the infimum of a sequence of quadratic functions. In experiments, we elaborate
on traditional image restoration applications of inpainting, deblurring and
denoising. Regarding the latter, we demonstrate state of the art restoration
quality with respect to structure coherence and color consistency.Comment: 30 page
Discontinuous and Smooth Depth Completion with Binary Anisotropic Diffusion Tensor
We propose an unsupervised real-time dense depth completion from a sparse
depth map guided by a single image. Our method generates a smooth depth map
while preserving discontinuity between different objects. Our key idea is a
Binary Anisotropic Diffusion Tensor (B-ADT) which can completely eliminate
smoothness constraint at intended positions and directions by applying it to
variational regularization. We also propose an Image-guided Nearest Neighbor
Search (IGNNS) to derive a piecewise constant depth map which is used for B-ADT
derivation and in the data term of the variational energy. Our experiments show
that our method can outperform previous unsupervised and semi-supervised depth
completion methods in terms of accuracy. Moreover, since our resulting depth
map preserves the discontinuity between objects, the result can be converted to
a visually plausible point cloud. This is remarkable since previous methods
generate unnatural surface-like artifacts between discontinuous objects.Comment: 8 pages 6 figure
3dDepthNet: Point Cloud Guided Depth Completion Network for Sparse Depth and Single Color Image
In this paper, we propose an end-to-end deep learning network named
3dDepthNet, which produces an accurate dense depth image from a single pair of
sparse LiDAR depth and color image for robotics and autonomous driving tasks.
Based on the dimensional nature of depth images, our network offers a novel
3D-to-2D coarse-to-fine dual densification design that is both accurate and
lightweight. Depth densification is first performed in 3D space via point cloud
completion, followed by a specially designed encoder-decoder structure that
utilizes the projected dense depth from 3D completion and the original RGB-D
images to perform 2D image completion. Experiments on the KITTI dataset show
our network achieves state-of-art accuracy while being more efficient. Ablation
and generalization tests prove that each module in our network has positive
influences on the final results, and furthermore, our network is resilient to
even sparser depth.Comment: 8 pages, 10 figures and 4 table
Explicit Edge Inconsistency Evaluation Model for Color-Guided Depth Map Enhancement
© 2016 IEEE. Color-guided depth enhancement is used to refine depth maps according to the assumption that the depth edges and the color edges at the corresponding locations are consistent. In methods on such low-level vision tasks, the Markov random field (MRF), including its variants, is one of the major approaches that have dominated this area for several years. However, the assumption above is not always true. To tackle the problem, the state-of-the-art solutions are to adjust the weighting coefficient inside the smoothness term of the MRF model. These methods lack an explicit evaluation model to quantitatively measure the inconsistency between the depth edge map and the color edge map, so they cannot adaptively control the efforts of the guidance from the color image for depth enhancement, leading to various defects such as texture-copy artifacts and blurring depth edges. In this paper, we propose a quantitative measurement on such inconsistency and explicitly embed it into the smoothness term. The proposed method demonstrates promising experimental results compared with the benchmark and state-of-the-art methods on the Middlebury ToF-Mark, and NYU data sets
Intelligent Imaging of Perfusion Using Arterial Spin Labelling
Arterial spin labelling (ASL) is a powerful magnetic resonance imaging technique, which can be used to noninvasively measure perfusion in the brain and other organs of the body. Promising research results show how ASL might be used in stroke, tumours, dementia and paediatric medicine, in addition to many other areas. However, significant obstacles remain to prevent widespread use: ASL images have an inherently low signal to noise ratio, and are susceptible to corrupting artifacts from motion and other sources. The objective of the work in this thesis is to move towards an "intelligent imaging" paradigm: one in which the image acquisition, reconstruction and processing are mutually coupled, and tailored to the individual patient. This thesis explores how ASL images may be improved at several stages of the imaging pipeline. We review the relevant ASL literature, exploring details of ASL acquisitions, parameter inference and artifact post-processing. We subsequently present original work: we use the framework of Bayesian experimental design to generate optimised ASL acquisitions, we present original methods to improve parameter inference through anatomically-driven modelling of spatial correlation, and we describe a novel deep learning approach for simultaneous denoising and artifact filtering. Using a mixture of theoretical derivation, simulation results and imaging experiments, the work in this thesis presents several new approaches for ASL, and hopefully will shape future research and future ASL usage
Deep Depth Super-Resolution : Learning Depth Super-Resolution using Deep Convolutional Neural Network
Depth image super-resolution is an extremely challenging task due to the
information loss in sub-sampling. Deep convolutional neural network have been
widely applied to color image super-resolution. Quite surprisingly, this
success has not been matched to depth super-resolution. This is mainly due to
the inherent difference between color and depth images. In this paper, we
bridge up the gap and extend the success of deep convolutional neural network
to depth super-resolution. The proposed deep depth super-resolution method
learns the mapping from a low-resolution depth image to a high resolution one
in an end-to-end style. Furthermore, to better regularize the learned depth
map, we propose to exploit the depth field statistics and the local correlation
between depth image and color image. These priors are integrated in an energy
minimization formulation, where the deep neural network learns the unary term,
the depth field statistics works as global model constraint and the color-depth
correlation is utilized to enforce the local structure in depth images.
Extensive experiments on various depth super-resolution benchmark datasets show
that our method outperforms the state-of-the-art depth image super-resolution
methods with a margin.Comment: 13 pages, 10 figure
Collaborative Total Variation: A General Framework for Vectorial TV Models
Even after over two decades, the total variation (TV) remains one of the most
popular regularizations for image processing problems and has sparked a
tremendous amount of research, particularly to move from scalar to
vector-valued functions. In this paper, we consider the gradient of a color
image as a three dimensional matrix or tensor with dimensions corresponding to
the spatial extend, the differences to other pixels, and the spectral channels.
The smoothness of this tensor is then measured by taking different norms along
the different dimensions. Depending on the type of these norms one obtains very
different properties of the regularization, leading to novel models for color
images. We call this class of regularizations collaborative total variation
(CTV). On the theoretical side, we characterize the dual norm, the
subdifferential and the proximal mapping of the proposed regularizers. We
further prove, with the help of the generalized concept of singular vectors,
that an channel coupling makes the most prior assumptions and
has the greatest potential to reduce color artifacts. Our practical
contributions consist of an extensive experimental section where we compare the
performance of a large number of collaborative TV methods for inverse problems
like denoising, deblurring and inpainting
Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Data
Depth completion aims to predict a dense depth map from a sparse depth input.
The acquisition of dense ground truth annotations for depth completion settings
can be difficult and, at the same time, a significant domain gap between real
LiDAR measurements and synthetic data has prevented from successful training of
models in virtual settings. We propose a domain adaptation approach for
sparse-to-dense depth completion that is trained from synthetic data, without
annotations in the real domain or additional sensors. Our approach simulates
the real sensor noise in an RGB+LiDAR set-up, and consists of three modules:
simulating the real LiDAR input in the synthetic domain via projections,
filtering the real noisy LiDAR for supervision and adapting the synthetic RGB
image using a CycleGAN approach. We extensively evaluate these modules against
the state-of-the-art in the KITTI depth completion benchmark, showing
significant improvements
Global Auto-regressive Depth Recovery via Iterative Non-local Filtering
Existing depth sensing techniques have many shortcomings in terms of resolution, completeness, and accuracy. The performance of 3-D broadcasting systems is therefore limited by the challenges of capturing high-resolution depth data. In this paper, we present a novel framework for obtaining high-quality depth images and multi-view depth videos from simple acquisition systems. We first propose a single depth image recovery algorithm based on auto-regressive (AR) correlations. A fixed-point iteration algorithm under the global AR modeling is derived to efficiently solve the large-scale quadratic programming. Each iteration is equivalent to a nonlocal filtering process with a residue feedback. Then, we extend our framework to an AR-based multi-view depth video recovery framework, where each depth map is recovered from low-quality measurements with the help of the corresponding color image, depth maps from neighboring views, and depth maps of temporally adjacent frames. AR coefficients on nonlocal spatiotemporal neighborhoods in the algorithm are designed to improve the recovery performance. We further discuss the connections between our model and other methods like graph-based tools, and demonstrate that our algorithms enjoy the advantages of both global and local methods. Experimental results on both the Middleburry datasets and other captured datasets finally show that our method is able to improve the performances of depth images and multi-view depth videos recovery compared with state-of-the-art approaches