2 research outputs found
Convolutional 3D to 2D Patch Conversion for Pixel-wise Glioma Segmentation in MRI Scans
Structural magnetic resonance imaging (MRI) has been widely utilized for
analysis and diagnosis of brain diseases. Automatic segmentation of brain
tumors is a challenging task for computer-aided diagnosis due to low-tissue
contrast in the tumor subregions. To overcome this, we devise a novel
pixel-wise segmentation framework through a convolutional 3D to 2D MR patch
conversion model to predict class labels of the central pixel in the input
sliding patches. Precisely, we first extract 3D patches from each modality to
calibrate slices through the squeeze and excitation (SE) block. Then, the
output of the SE block is fed directly into subsequent bottleneck layers to
reduce the number of channels. Finally, the calibrated 2D slices are
concatenated to obtain multimodal features through a 2D convolutional neural
network (CNN) for prediction of the central pixel. In our architecture, both
local inter-slice and global intra-slice features are jointly exploited to
predict class label of the central voxel in a given patch through the 2D CNN
classifier. We implicitly apply all modalities through trainable parameters to
assign weights to the contributions of each sequence for segmentation.
Experimental results on the segmentation of brain tumors in multimodal MRI
scans (BraTS'19) demonstrate that our proposed method can efficiently segment
the tumor regions
Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation
In large studies involving multi protocol Magnetic Resonance Imaging (MRI),
it can occur to miss one or more sub-modalities for a given patient owing to
poor quality (e.g. imaging artifacts), failed acquisitions, or hallway
interrupted imaging examinations. In some cases, certain protocols are
unavailable due to limited scan time or to retrospectively harmonise the
imaging protocols of two independent studies. Missing image modalities pose a
challenge to segmentation frameworks as complementary information contributed
by the missing scans is then lost. In this paper, we propose a novel model,
Multi-modal Gaussian Process Prior Variational Autoencoder (MGP-VAE), to impute
one or more missing sub-modalities for a patient scan. MGP-VAE can leverage the
Gaussian Process (GP) prior on the Variational Autoencoder (VAE) to utilize the
subjects/patients and sub-modalities correlations. Instead of designing one
network for each possible subset of present sub-modalities or using frameworks
to mix feature maps, missing data can be generated from a single model based on
all the available samples. We show the applicability of MGP-VAE on brain tumor
segmentation where either, two, or three of four sub-modalities may be missing.
Our experiments against competitive segmentation baselines with missing
sub-modality on BraTS'19 dataset indicate the effectiveness of the MGP-VAE
model for segmentation tasks.Comment: Accepted in MICCAI 202