4,061 research outputs found
Advanced Restoration Techniques for Images and Disparity Maps
With increasing popularity of digital cameras, the field of Computa-
tional Photography emerges as one of the most demanding areas of
research. In this thesis we study and develop novel priors and op-
timization techniques to solve inverse problems, including disparity
estimation and image restoration.
The disparity map estimation method proposed in this thesis incor-
porates multiple frames of a stereo video sequence to ensure temporal
coherency. To enforce smoothness, we use spatio-temporal connec-
tions between the pixels of the disparity map to constrain our solution.
Apart from smoothness, we enforce a consistency constraint for the
disparity assignments by using connections between the left and right
views. These constraints are then formulated in a graphical model,
which we solve using mean-field approximation. We use a filter-based
mean-field optimization that perform efficiently by updating the dis-
parity variables in parallel. The parallel updates scheme, however, is
not guaranteed to converge to a stationary point. To compare and
demonstrate the effectiveness of our approach, we developed a new
optimization technique that uses sequential updates, which runs ef-
ficiently and guarantees convergence. Our empirical results indicate
that with proper initialization, we can employ the parallel update
scheme and efficiently optimize our disparity maps without loss of
quality. Our method ranks amongst the state of the art in common
benchmarks, and significantly reduces the temporal flickering artifacts
in the disparity maps.
In the second part of this thesis, we address several image restora-
tion problems such as image deblurring, demosaicing and super-
resolution. We propose to use denoising autoencoders to learn an
approximation of the true natural image distribution. We parametrize
our denoisers using deep neural networks and show that they learn
the gradient of the smoothed density of natural images. Based on
this analysis, we propose a restoration technique that moves the so-
lution towards the local extrema of this distribution by minimizing
the difference between the input and output of our denoiser. Weii
demonstrate the effectiveness of our approach using a single trained
neural network in several restoration tasks such as deblurring and
super-resolution. In a more general framework, we define a new
Bayes formulation for the restoration problem, which leads to a more
efficient and robust estimator. The proposed framework achieves state
of the art performance in various restoration tasks such as deblurring
and demosaicing, and also for more challenging tasks such as noise-
and kernel-blind image deblurring.
Keywords. disparity map estimation, stereo matching, mean-field
optimization, graphical models, image processing, linear inverse prob-
lems, image restoration, image deblurring, image denoising, single
image super-resolution, image demosaicing, deep neural networks,
denoising autoencoder
One at A Time: Multi-step Volumetric Probability Distribution Diffusion for Depth Estimation
Recent works have explored the fundamental role of depth estimation in
multi-view stereo (MVS) and semantic scene completion (SSC). They generally
construct 3D cost volumes to explore geometric correspondence in depth, and
estimate such volumes in a single step relying directly on the ground truth
approximation. However, such problem cannot be thoroughly handled in one step
due to complex empirical distributions, especially in challenging regions like
occlusions, reflections, etc. In this paper, we formulate the depth estimation
task as a multi-step distribution approximation process, and introduce a new
paradigm of modeling the Volumetric Probability Distribution progressively
(step-by-step) following a Markov chain with Diffusion models (VPDD).
Specifically, to constrain the multi-step generation of volume in VPDD, we
construct a meta volume guidance and a confidence-aware contextual guidance as
conditional geometry priors to facilitate the distribution approximation. For
the sampling process, we further investigate an online filtering strategy to
maintain consistency in volume representations for stable training. Experiments
demonstrate that our plug-and-play VPDD outperforms the state-of-the-arts for
tasks of MVS and SSC, and can also be easily extended to different baselines to
get improvement. It is worth mentioning that we are the first camera-based work
that surpasses LiDAR-based methods on the SemanticKITTI dataset
Graph Spectral Image Processing
Recent advent of graph signal processing (GSP) has spurred intensive studies
of signals that live naturally on irregular data kernels described by graphs
(e.g., social networks, wireless sensor networks). Though a digital image
contains pixels that reside on a regularly sampled 2D grid, if one can design
an appropriate underlying graph connecting pixels with weights that reflect the
image structure, then one can interpret the image (or image patch) as a signal
on a graph, and apply GSP tools for processing and analysis of the signal in
graph spectral domain. In this article, we overview recent graph spectral
techniques in GSP specifically for image / video processing. The topics covered
include image compression, image restoration, image filtering and image
segmentation
Test-Time Adaptation for Point Cloud Upsampling Using Meta-Learning
Affordable 3D scanners often produce sparse and non-uniform point clouds that
negatively impact downstream applications in robotic systems. While existing
point cloud upsampling architectures have demonstrated promising results on
standard benchmarks, they tend to experience significant performance drops when
the test data have different distributions from the training data. To address
this issue, this paper proposes a test-time adaption approach to enhance model
generality of point cloud upsampling. The proposed approach leverages
meta-learning to explicitly learn network parameters for test-time adaption.
Our method does not require any prior information about the test data. During
meta-training, the model parameters are learned from a collection of
instance-level tasks, each of which consists of a sparse-dense pair of point
clouds from the training data. During meta-testing, the trained model is
fine-tuned with a few gradient updates to produce a unique set of network
parameters for each test instance. The updated model is then used for the final
prediction. Our framework is generic and can be applied in a plug-and-play
manner with existing backbone networks in point cloud upsampling. Extensive
experiments demonstrate that our approach improves the performance of
state-of-the-art models.Comment: Accepted at IROS 202
Electrification of Smart Cities
Electrification plays a key role in decarbonizing energy consumption for various sectors, including transportation, heating, and cooling. There are several essential infrastructures for a smart city, including smart grids and transportation networks. These infrastructures are the complementary solutions to successfully developing novel services, with enhanced energy efficiency and energy security. Five papers are published in this Special Issue that cover various key areas expanding the state-of-the-art in smart cities’ electrification, including transportation, healthcare, and advanced closed-circuit televisions for smart city surveillance
- …