106 research outputs found
Medical Diagnosis with Multimodal Image Fusion Techniques
Image Fusion is an effective approach utilized to draw out all the significant information from the source images, which supports experts in evaluation and quick decision making. Multi modal medical image fusion produces a composite fused image utilizing various sources to improve quality and extract complementary information. It is extremely challenging to gather every piece of information needed using just one imaging method. Therefore, images obtained from different modalities are fused Additional clinical information can be gleaned through the fusion of several types of medical image pairings. This study's main aim is to present a thorough review of medical image fusion techniques which also covers steps in fusion process, levels of fusion, various imaging modalities with their pros and cons, and the major scientific difficulties encountered in the area of medical image fusion. This paper also summarizes the quality assessments fusion metrics. The various approaches used by image fusion algorithms that are presently available in the literature are classified into four broad categories i) Spatial fusion methods ii) Multiscale Decomposition based methods iii) Neural Network based methods and iv) Fuzzy Logic based methods. the benefits and pitfalls of the existing literature are explored and Future insights are suggested. Moreover, this study is anticipated to create a solid platform for the development of better fusion techniques in medical applications
Deraining and Desnowing Algorithm on Adaptive Tolerance and Dual-tree Complex Wavelet Fusion
Severe weather conditions such as rain and snow often reduce the visual perception quality of the video image system, the traditional methods of deraining and desnowing usually rarely consider adaptive parameters. In order to enhance the effect of video deraining and desnowing, this paper proposes a video deraining and desnowing algorithm based on adaptive tolerance and dual-tree complex wavelet. This algorithm can be widely used in security surveillance, military defense, biological monitoring, remote sensing and other fields. First, this paper introduces the main work of the adaptive tolerance method for the video of dynamic scenes. Second, the algorithm of dual-tree complex wavelet fusion is analyzed and introduced. Using principal component analysis fusion rules to process low-frequency sub-bands, the fusion rule of local energy matching is used to process the high-frequency sub-bands. Finally, this paper used various rain and snow videos to verify the validity and superiority of image reconstruction. Experimental results show that the algorithm has achieved good results in improving the image clarity and restoring the image details obscured by raindrops and snows
Spectral Graphormer: Spectral Graph-based Transformer for Egocentric Two-Hand Reconstruction using Multi-View Color Images
We propose a novel transformer-based framework that reconstructs two high
fidelity hands from multi-view RGB images. Unlike existing hand pose estimation
methods, where one typically trains a deep network to regress hand model
parameters from single RGB image, we consider a more challenging problem
setting where we directly regress the absolute root poses of two-hands with
extended forearm at high resolution from egocentric view. As existing datasets
are either infeasible for egocentric viewpoints or lack background variations,
we create a large-scale synthetic dataset with diverse scenarios and collect a
real dataset from multi-calibrated camera setup to verify our proposed
multi-view image feature fusion strategy. To make the reconstruction physically
plausible, we propose two strategies: (i) a coarse-to-fine spectral graph
convolution decoder to smoothen the meshes during upsampling and (ii) an
optimisation-based refinement stage at inference to prevent self-penetrations.
Through extensive quantitative and qualitative evaluations, we show that our
framework is able to produce realistic two-hand reconstructions and demonstrate
the generalisation of synthetic-trained models to real data, as well as
real-time AR/VR applications.Comment: Accepted to ICCV 202
Encoding Enhanced Complex CNN for Accurate and Highly Accelerated MRI
Magnetic resonance imaging (MRI) using hyperpolarized noble gases provides a
way to visualize the structure and function of human lung, but the long imaging
time limits its broad research and clinical applications. Deep learning has
demonstrated great potential for accelerating MRI by reconstructing images from
undersampled data. However, most existing deep conventional neural networks
(CNN) directly apply square convolution to k-space data without considering the
inherent properties of k-space sampling, limiting k-space learning efficiency
and image reconstruction quality. In this work, we propose an encoding enhanced
(EN2) complex CNN for highly undersampled pulmonary MRI reconstruction. EN2
employs convolution along either the frequency or phase-encoding direction,
resembling the mechanisms of k-space sampling, to maximize the utilization of
the encoding correlation and integrity within a row or column of k-space. We
also employ complex convolution to learn rich representations from the complex
k-space data. In addition, we develop a feature-strengthened modularized unit
to further boost the reconstruction performance. Experiments demonstrate that
our approach can accurately reconstruct hyperpolarized 129Xe and 1H lung MRI
from 6-fold undersampled k-space data and provide lung function measurements
with minimal biases compared with fully-sampled image. These results
demonstrate the effectiveness of the proposed algorithmic components and
indicate that the proposed approach could be used for accelerated pulmonary MRI
in research and clinical lung disease patient care
A multimodal fusion method for Alzheimer’s disease based on DCT convolutional sparse representation
IntroductionThe medical information contained in magnetic resonance imaging (MRI) and positron emission tomography (PET) has driven the development of intelligent diagnosis of Alzheimer’s disease (AD) and multimodal medical imaging. To solve the problems of severe energy loss, low contrast of fused images and spatial inconsistency in the traditional multimodal medical image fusion methods based on sparse representation. A multimodal fusion algorithm for Alzheimer’ s disease based on the discrete cosine transform (DCT) convolutional sparse representation is proposed.MethodsThe algorithm first performs a multi-scale DCT decomposition of the source medical images and uses the sub-images of different scales as training images, respectively. Different sparse coefficients are obtained by optimally solving the sub-dictionaries at different scales using alternating directional multiplication method (ADMM). Secondly, the coefficients of high-frequency and low-frequency subimages are inverse DCTed using an improved L1 parametric rule combined with improved spatial frequency novel sum-modified SF (NMSF) to obtain the final fused images.Results and discussionThrough extensive experimental results, we show that our proposed method has good performance in contrast enhancement, texture and contour information retention
- …