756 research outputs found
A Plug-and-Play Approach To Multiparametric Quantitative MRI:Image Reconstruction Using Pre-Trained Deep Denoisers
Current spatiotemporal deep learning approaches to Magnetic Resonance
Fingerprinting (MRF) build artefact-removal models customised to a particular
k-space subsampling pattern which is used for fast (compressed) acquisition.
This may not be useful when the acquisition process is unknown during training
of the deep learning model and/or changes during testing time. This paper
proposes an iterative deep learning plug-and-play reconstruction approach to
MRF which is adaptive to the forward acquisition process. Spatiotemporal image
priors are learned by an image denoiser i.e. a Convolutional Neural Network
(CNN), trained to remove generic white gaussian noise (not a particular
subsampling artefact) from data. This CNN denoiser is then used as a
data-driven shrinkage operator within the iterative reconstruction algorithm.
This algorithm with the same denoiser model is then tested on two simulated
acquisition processes with distinct subsampling patterns. The results show
consistent de-aliasing performance against both acquisition schemes and
accurate mapping of tissues' quantitative bio-properties. Software available:
https://github.com/ketanfatania/QMRI-PnP-Recon-PO
High-ISO long-exposure image denoising based on quantitative blob characterization
Blob detection and image denoising are fundamental, sometimes related tasks in computer vision. In this paper, we present a computational method to quantitatively measure blob characteristics using normalized unilateral second-order Gaussian kernels. This method suppresses non-blob structures while yielding a quantitative measurement of the position, prominence and scale of blobs, which can facilitate the tasks of blob reconstruction and blob reduction. Subsequently, we propose a denoising scheme to address high-ISO long-exposure noise, which sometimes spatially shows a blob appearance, employing a blob reduction procedure as a cheap preprocessing for conventional denoising methods. We apply the proposed denoising methods to real-world noisy images as well as standard images that are corrupted by real noise. The experimental results demonstrate the superiority of the proposed methods over state-of-the-art denoising methods
Deep Representation Learning with Limited Data for Biomedical Image Synthesis, Segmentation, and Detection
Biomedical imaging requires accurate expert annotation and interpretation that can aid medical staff and clinicians in automating differential diagnosis and solving underlying health conditions. With the advent of Deep learning, it has become a standard for reaching expert-level performance in non-invasive biomedical imaging tasks by training with large image datasets. However, with the need for large publicly available datasets, training a deep learning model to learn intrinsic representations becomes harder. Representation learning with limited data has introduced new learning techniques, such as Generative Adversarial Networks, Semi-supervised Learning, and Self-supervised Learning, that can be applied to various biomedical applications. For example, ophthalmologists use color funduscopy (CF) and fluorescein angiography (FA) to diagnose retinal degenerative diseases. However, fluorescein angiography requires injecting a dye, which can create adverse reactions in the patients. So, to alleviate this, a non-invasive technique needs to be developed that can translate fluorescein angiography from fundus images. Similarly, color funduscopy and optical coherence tomography (OCT) are also utilized to semantically segment the vasculature and fluid build-up in spatial and volumetric retinal imaging, which can help with the future prognosis of diseases. Although many automated techniques have been proposed for medical image segmentation, the main drawback is the model's precision in pixel-wise predictions. Another critical challenge in the biomedical imaging field is accurately segmenting and quantifying dynamic behaviors of calcium signals in cells. Calcium imaging is a widely utilized approach to studying subcellular calcium activity and cell function; however, large datasets have yielded a profound need for fast, accurate, and standardized analyses of calcium signals. For example, image sequences from calcium signals in colonic pacemaker cells ICC (Interstitial cells of Cajal) suffer from motion artifacts and high periodic and sensor noise, making it difficult to accurately segment and quantify calcium signal events. Moreover, it is time-consuming and tedious to annotate such a large volume of calcium image stacks or videos and extract their associated spatiotemporal maps. To address these problems, we propose various deep representation learning architectures that utilize limited labels and annotations to address the critical challenges in these biomedical applications. To this end, we detail our proposed semi-supervised, generative adversarial networks and transformer-based architectures for individual learning tasks such as retinal image-to-image translation, vessel and fluid segmentation from fundus and OCT images, breast micro-mass segmentation, and sub-cellular calcium events tracking from videos and spatiotemporal map quantification. We also illustrate two multi-modal multi-task learning frameworks with applications that can be extended to other domains of biomedical applications. The main idea is to incorporate each of these as individual modules to our proposed multi-modal frameworks to solve the existing challenges with 1) Fluorescein angiography synthesis, 2) Retinal vessel and fluid segmentation, 3) Breast micro-mass segmentation, and 4) Dynamic quantification of calcium imaging datasets
H2-Stereo: High-Speed, High-Resolution Stereoscopic Video System
High-speed, high-resolution stereoscopic (H2-Stereo) video allows us to
perceive dynamic 3D content at fine granularity. The acquisition of H2-Stereo
video, however, remains challenging with commodity cameras. Existing spatial
super-resolution or temporal frame interpolation methods provide compromised
solutions that lack temporal or spatial details, respectively. To alleviate
this problem, we propose a dual camera system, in which one camera captures
high-spatial-resolution low-frame-rate (HSR-LFR) videos with rich spatial
details, and the other captures low-spatial-resolution high-frame-rate
(LSR-HFR) videos with smooth temporal details. We then devise a Learned
Information Fusion network (LIFnet) that exploits the cross-camera redundancies
to enhance both camera views to high spatiotemporal resolution (HSTR) for
reconstructing the H2-Stereo video effectively. We utilize a disparity network
to transfer spatiotemporal information across views even in large disparity
scenes, based on which, we propose disparity-guided flow-based warping for
LSR-HFR view and complementary warping for HSR-LFR view. A multi-scale fusion
method in feature domain is proposed to minimize occlusion-induced warping
ghosts and holes in HSR-LFR view. The LIFnet is trained in an end-to-end manner
using our collected high-quality Stereo Video dataset from YouTube. Extensive
experiments demonstrate that our model outperforms existing state-of-the-art
methods for both views on synthetic data and camera-captured real data with
large disparity. Ablation studies explore various aspects, including
spatiotemporal resolution, camera baseline, camera desynchronization,
long/short exposures and applications, of our system to fully understand its
capability for potential applications
Knowledge-driven deep learning for fast MR imaging: undersampled MR image reconstruction from supervised to un-supervised learning
Deep learning (DL) has emerged as a leading approach in accelerating MR
imaging. It employs deep neural networks to extract knowledge from available
datasets and then applies the trained networks to reconstruct accurate images
from limited measurements. Unlike natural image restoration problems, MR
imaging involves physics-based imaging processes, unique data properties, and
diverse imaging tasks. This domain knowledge needs to be integrated with
data-driven approaches. Our review will introduce the significant challenges
faced by such knowledge-driven DL approaches in the context of fast MR imaging
along with several notable solutions, which include learning neural networks
and addressing different imaging application scenarios. The traits and trends
of these techniques have also been given which have shifted from supervised
learning to semi-supervised learning, and finally, to unsupervised learning
methods. In addition, MR vendors' choices of DL reconstruction have been
provided along with some discussions on open questions and future directions,
which are critical for the reliable imaging systems.Comment: 46 pages, 5figures, 1 tabl
Physics-Informed Computer Vision: A Review and Perspectives
Incorporation of physical information in machine learning frameworks are
opening and transforming many application domains. Here the learning process is
augmented through the induction of fundamental knowledge and governing physical
laws. In this work we explore their utility for computer vision tasks in
interpreting and understanding visual data. We present a systematic literature
review of formulation and approaches to computer vision tasks guided by
physical laws. We begin by decomposing the popular computer vision pipeline
into a taxonomy of stages and investigate approaches to incorporate governing
physical equations in each stage. Existing approaches in each task are analyzed
with regard to what governing physical processes are modeled, formulated and
how they are incorporated, i.e. modify data (observation bias), modify networks
(inductive bias), and modify losses (learning bias). The taxonomy offers a
unified view of the application of the physics-informed capability,
highlighting where physics-informed learning has been conducted and where the
gaps and opportunities are. Finally, we highlight open problems and challenges
to inform future research. While still in its early days, the study of
physics-informed computer vision has the promise to develop better computer
vision models that can improve physical plausibility, accuracy, data efficiency
and generalization in increasingly realistic applications
- …