Comparative study of Image Fusion Methods: A Review

As the size and cost of sensors decrease, sensor networks are increasingly becoming an attractive method to collect information in a given area. However, one single sensor is not capable of providing all the required information,either because of their design or because of observational constraints. One possible solution to get all the required information about a particular scene or subject is data fusion.. A small number of metrics proposed so far provide only a rough, numerical estimate of fusion performance with limited understanding of the relative merits of different fusion schemes. This paper proposes a method for comprehensive, objective, image fusion performance characterization using a fusion evaluation framework based on gradient information representation. We give the framework of the overallnbsp system and explain its USAge method. The system has many functions: image denoising, image enhancement, image registration, image segmentation, image fusion, and fusion evaluation. This paper presents a literature review on some of the image fusion techniques for image fusion like, Laplace transform, Discrete Wavelet transform based fusion, Principal component analysis (PCA) based fusion etc. Comparison of all the techniques can be the better approach fornbsp future research

DILF: Differentiable Rendering-Based Multi-View Image-Language Fusion for Zero-Shot 3D Shape Understanding

Zero-shot 3D shape understanding aims to recognize “unseen” 3D categories that are not present in training data. Recently, Contrastive Language–Image Pre-training (CLIP) has shown promising open-world performance in zero-shot 3D shape understanding tasks by information fusion among language and 3D modality. It first renders 3D objects into multiple 2D image views and then learns to understand the semantic relationships between the textual descriptions and images, enabling the model to generalize to new and unseen categories. However, existing studies in zero-shot 3D shape understanding rely on predefined rendering parameters, resulting in repetitive, redundant, and low-quality views. This limitation hinders the model’s ability to fully comprehend 3D shapes and adversely impacts the text–image fusion in a shared latent space. To this end, we propose a novel approach called Differentiable rendering-based multi-view Image–Language Fusion (DILF) for zero-shot 3D shape understanding. Specifically, DILF leverages large-scale language models (LLMs) to generate textual prompts enriched with 3D semantics and designs a differentiable renderer with learnable rendering parameters to produce representative multi-view images. These rendering parameters can be iteratively updated using a text–image fusion loss, which aids in parameters’ regression, allowing the model to determine the optimal viewpoint positions for each 3D object. Then a group-view mechanism is introduced to model interdependencies across views, enabling efficient information fusion to achieve a more comprehensive 3D shape understanding. Experimental results can demonstrate that DILF outperforms state-of-the-art methods for zero-shot 3D classification while maintaining competitive performance for standard 3D classification. The code is available at https://github.com/yuzaiyang123/DILP

Evaluation of optical and synthetic aperture radar image fusion methods: a case study applied to Sentinel imagery

This paper evaluates different optical and synthetic aperture radar (SAR) image fusion methods applied to open-access Sentinel images with global coverage. The objective of this research was to evaluate the potential of image fusion methods to get a greater visual difference in land cover, especially in oil palm crops with natural forest areas that are difficult to differentiate visually. The application of the image fusion methods: Brovey (BR), high-frequency modulation (HFM), Gram-Schmidt (GS), and principal components (PC) was evaluated on Sentinel-2 optical and Sentinel-1 SAR images using a cloud computing environment. The results show that the application of the implemented optical/SAR image fusion methods allows the creation of a synthetic image with the characteristics of both data sources. The multispectral information provided by the optical image and information associated with the geometry and texture/roughness of the land covers, provided by the SAR image, allows a greater differentiation in the visualization of the various land covers, achieving a better understanding of the study area. The fusion methods that visually presented greater characteristics associated with the SAR image were the BR and GS methods. The HFM method reached the best statistical indicators; however, this method did not present significant visual changes in the SAR contribution

Infrared and Visible Image Fusion Based on Oversampled Graph Filter Banks

The infrared image (RI) and visible image (VI) fusion method merges complementary information from the infrared and visible imaging sensors to provide an effective way for understanding the scene. The graph filter bank-based graph wavelet transform possesses the advantages of the classic wavelet filter bank and graph representation of a signal. Therefore, we propose an RI and VI fusion method based on oversampled graph filter banks. Specifically, we consider the source images as signals on the regular graph and decompose them into the multiscale representations with M-channel oversampled graph filter banks. Then, the fusion rule for the low-frequency subband is constructed using the modified local coefficient of variation and the bilateral filter. The fusion maps of detail subbands are formed using the standard deviation-based local properties. Finally, the fusion image is obtained by applying the inverse transform on the fusion subband coefficients. The experimental results on benchmark images show the potential of the proposed method in the image fusion applications

A new framework for the integrative analytics of intravascular ultrasound and optical coherence tomography images

Abstract:The integrative analysis of multimodal medical images plays an important role in the diagnosis of coronary artery disease by providing additional comprehensive information that cannot be found in an individual source image. Intravascular ultrasound (IVUS) and optical coherence tomography (IV-OCT) are two imaging modalities that have been widely used in the medical practice for the assessment of arterial health and the detection of vascular lumen lesions. IV-OCT has a high resolution and poor penetration, while IVUS has a low resolution and high detection depth. This paper proposes a new approach for the fusion of intravascular ultrasound and optical coherence tomography pullbacks to significantly improve the use of those two types of medical images. It also presents a new two-phase multimodal fusion framework using a coarse-to-fine registration and a wavelet fusion method. In the coarse-registration process, we define a set of new feature points to match the IVUS image and IV-OCT image. Then, the improved quality image is obtained based on the integration of the mutual information of two types of images. Finally, the matched registered images are fused with an approach based on the new proposed wavelet algorithm. The experimental results demonstrate the performance of the proposed new approach for significantly enhancing both the precision and computational stability. The proposed approach is shown to be promising for providing additional information to enhance the diagnosis and enable a deeper understanding of atherosclerosis