214 research outputs found
Multimodal Deep Unfolding for Guided Image Super-Resolution
The reconstruction of a high resolution image given a low resolution
observation is an ill-posed inverse problem in imaging. Deep learning methods
rely on training data to learn an end-to-end mapping from a low-resolution
input to a high-resolution output. Unlike existing deep multimodal models that
do not incorporate domain knowledge about the problem, we propose a multimodal
deep learning design that incorporates sparse priors and allows the effective
integration of information from another image modality into the network
architecture. Our solution relies on a novel deep unfolding operator,
performing steps similar to an iterative algorithm for convolutional sparse
coding with side information; therefore, the proposed neural network is
interpretable by design. The deep unfolding architecture is used as a core
component of a multimodal framework for guided image super-resolution. An
alternative multimodal design is investigated by employing residual learning to
improve the training efficiency. The presented multimodal approach is applied
to super-resolution of near-infrared and multi-spectral images as well as depth
upsampling using RGB images as side information. Experimental results show that
our model outperforms state-of-the-art methods
Unsharp Mask Guided Filtering
The goal of this paper is guided image filtering, which emphasizes the
importance of structure transfer during filtering by means of an additional
guidance image. Where classical guided filters transfer structures using
hand-designed functions, recent guided filters have been considerably advanced
through parametric learning of deep networks. The state-of-the-art leverages
deep networks to estimate the two core coefficients of the guided filter. In
this work, we posit that simultaneously estimating both coefficients is
suboptimal, resulting in halo artifacts and structure inconsistencies. Inspired
by unsharp masking, a classical technique for edge enhancement that requires
only a single coefficient, we propose a new and simplified formulation of the
guided filter. Our formulation enjoys a filtering prior from a low-pass filter
and enables explicit structure transfer by estimating a single coefficient.
Based on our proposed formulation, we introduce a successive guided filtering
network, which provides multiple filtering results from a single network,
allowing for a trade-off between accuracy and efficiency. Extensive ablations,
comparisons and analysis show the effectiveness and efficiency of our
formulation and network, resulting in state-of-the-art results across filtering
tasks like upsampling, denoising, and cross-modality filtering. Code is
available at \url{https://github.com/shizenglin/Unsharp-Mask-Guided-Filtering}.Comment: IEEE Transactions on Image Processing, 202
- …