2 research outputs found
Bridging the Gap between Multi-focus and Multi-modal: A Focused Integration Framework for Multi-modal Image Fusion
Multi-modal image fusion (MMIF) integrates valuable information from
different modality images into a fused one. However, the fusion of multiple
visible images with different focal regions and infrared images is a
unprecedented challenge in real MMIF applications. This is because of the
limited depth of the focus of visible optical lenses, which impedes the
simultaneous capture of the focal information within the same scene. To address
this issue, in this paper, we propose a MMIF framework for joint focused
integration and modalities information extraction. Specifically, a
semi-sparsity-based smoothing filter is introduced to decompose the images into
structure and texture components. Subsequently, a novel multi-scale operator is
proposed to fuse the texture components, capable of detecting significant
information by considering the pixel focus attributes and relevant data from
various modal images. Additionally, to achieve an effective capture of scene
luminance and reasonable contrast maintenance, we consider the distribution of
energy information in the structural components in terms of multi-directional
frequency variance and information entropy. Extensive experiments on existing
MMIF datasets, as well as the object detection and depth estimation tasks,
consistently demonstrate that the proposed algorithm can surpass the
state-of-the-art methods in visual perception and quantitative evaluation. The
code is available at https://github.com/ixilai/MFIF-MMIF.Comment: Accepted to IEEE/CVF Winter Conference on Applications of Computer
Vision (WACV) 202
Semi-Sparsity for Smoothing Filters
In this paper, we propose an interesting semi-sparsity smoothing algorithm
based on a novel sparsity-inducing optimization framework. This method is
derived from the multiple observations, that is, semi-sparsity prior knowledge
is more universally applicable, especially in areas where sparsity is not fully
admitted, such as polynomial-smoothing surfaces. We illustrate that this
semi-sparsity can be identified into a generalized -norm minimization in
higher-order gradient domains, thereby giving rise to a new "feature-aware"
filtering method with a powerful simultaneous-fitting ability in both sparse
features (singularities and sharpening edges) and non-sparse regions
(polynomial-smoothing surfaces). Notice that a direct solver is always
unavailable due to the non-convexity and combinatorial nature of -norm
minimization. Instead, we solve the model based on an efficient half-quadratic
splitting minimization with fast Fourier transforms (FFTs) for acceleration. We
finally demonstrate its versatility and many benefits to a series of
signal/image processing and computer vision applications