36 research outputs found
Manifold-Aware CycleGAN for High-Resolution Structural-to-DTI Synthesis
Unpaired image-to-image translation has been applied successfully to natural
images but has received very little attention for manifold-valued data such as
in diffusion tensor imaging (DTI). The non-Euclidean nature of DTI prevents
current generative adversarial networks (GANs) from generating plausible images
and has mainly limited their application to diffusion MRI scalar maps, such as
fractional anisotropy (FA) or mean diffusivity (MD). Even if these scalar maps
are clinically useful, they mostly ignore fiber orientations and therefore have
limited applications for analyzing brain fibers. Here, we propose a
manifold-aware CycleGAN that learns the generation of high-resolution DTI from
unpaired T1w images. We formulate the objective as a Wasserstein distance
minimization problem of data distributions on a Riemannian manifold of
symmetric positive definite 3x3 matrices SPD(3), using adversarial and
cycle-consistency losses. To ensure that the generated diffusion tensors lie on
the SPD(3) manifold, we exploit the theoretical properties of the exponential
and logarithm maps of the Log-Euclidean metric. We demonstrate that, unlike
standard GANs, our method is able to generate realistic high-resolution DTI
that can be used to compute diffusion-based metrics and potentially run fiber
tractography algorithms. To evaluate our model's performance, we compute the
cosine similarity between the generated tensors principal orientation and their
ground-truth orientation, the mean squared error (MSE) of their derived FA
values and the Log-Euclidean distance between the tensors. We demonstrate that
our method produces 2.5 times better FA MSE than a standard CycleGAN and up to
30% better cosine similarity than a manifold-aware Wasserstein GAN while
synthesizing sharp high-resolution DTI.Comment: Accepted at MICCAI 2020 International Workshop on Computational
Diffusion MR
Anatomically-aware Uncertainty for Semi-supervised Image Segmentation
Semi-supervised learning relaxes the need of large pixel-wise labeled
datasets for image segmentation by leveraging unlabeled data. A prominent way
to exploit unlabeled data is to regularize model predictions. Since the
predictions of unlabeled data can be unreliable, uncertainty-aware schemes are
typically employed to gradually learn from meaningful and reliable predictions.
Uncertainty estimation methods, however, rely on multiple inferences from the
model predictions that must be computed for each training step, which is
computationally expensive. Moreover, these uncertainty maps capture pixel-wise
disparities and do not consider global information. This work proposes a novel
method to estimate segmentation uncertainty by leveraging global information
from the segmentation masks. More precisely, an anatomically-aware
representation is first learnt to model the available segmentation masks. The
learnt representation thereupon maps the prediction of a new segmentation into
an anatomically-plausible segmentation. The deviation from the plausible
segmentation aids in estimating the underlying pixel-level uncertainty in order
to further guide the segmentation network. The proposed method consequently
estimates the uncertainty using a single inference from our representation,
thereby reducing the total computation. We evaluate our method on two publicly
available segmentation datasets of left atria in cardiac MRIs and of multiple
organs in abdominal CTs. Our anatomically-aware method improves the
segmentation accuracy over the state-of-the-art semi-supervised methods in
terms of two commonly used evaluation metrics.Comment: Accepted at Medical Image Analysis. Code is available at:
$\href{https://github.com/adigasu/Anatomically-aware_Uncertainty_for_Semi-supervised_Segmentation}{Github}
HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentation
Recently, dense connections have attracted substantial attention in computer
vision because they facilitate gradient flow and implicit deep supervision
during training. Particularly, DenseNet, which connects each layer to every
other layer in a feed-forward fashion, has shown impressive performances in
natural image classification tasks. We propose HyperDenseNet, a 3D fully
convolutional neural network that extends the definition of dense connectivity
to multi-modal segmentation problems. Each imaging modality has a path, and
dense connections occur not only between the pairs of layers within the same
path, but also between those across different paths. This contrasts with the
existing multi-modal CNN approaches, in which modeling several modalities
relies entirely on a single joint layer (or level of abstraction) for fusion,
typically either at the input or at the output of the network. Therefore, the
proposed network has total freedom to learn more complex combinations between
the modalities, within and in-between all the levels of abstraction, which
increases significantly the learning representation. We report extensive
evaluations over two different and highly competitive multi-modal brain tissue
segmentation challenges, iSEG 2017 and MRBrainS 2013, with the former focusing
on 6-month infant data and the latter on adult images. HyperDenseNet yielded
significant improvements over many state-of-the-art segmentation networks,
ranking at the top on both benchmarks. We further provide a comprehensive
experimental analysis of features re-use, which confirms the importance of
hyper-dense connections in multi-modal representation learning. Our code is
publicly available at https://www.github.com/josedolz/HyperDenseNet.Comment: Paper accepted at IEEE TMI in October 2018. Last version of this
paper updates the reference to the IEEE TMI paper which compares the
submissions to the iSEG 2017 MICCAI Challeng
Spectral Graph Transformer Networks for Brain Surface Parcellation
The analysis of the brain surface modeled as a graph mesh is a challenging
task. Conventional deep learning approaches often rely on data lying in the
Euclidean space. As an extension to irregular graphs, convolution operations
are defined in the Fourier or spectral domain. This spectral domain is obtained
by decomposing the graph Laplacian, which captures relevant shape information.
However, the spectral decomposition across different brain graphs causes
inconsistencies between the eigenvectors of individual spectral domains,
causing the graph learning algorithm to fail. Current spectral graph
convolution methods handle this variance by separately aligning the
eigenvectors to a reference brain in a slow iterative step. This paper presents
a novel approach for learning the transformation matrix required for aligning
brain meshes using a direct data-driven approach. Our alignment and graph
processing method provides a fast analysis of brain surfaces. The novel
Spectral Graph Transformer (SGT) network proposed in this paper uses very few
randomly sub-sampled nodes in the spectral domain to learn the alignment matrix
for multiple brain surfaces. We validate the use of this SGT network along with
a graph convolution network to perform cortical parcellation. Our method on 101
manually-labeled brain surfaces shows improved parcellation performance over a
no-alignment strategy, gaining a significant speed (1400 fold) over traditional
iterative alignment approaches.Comment: Equal contribution of R. He and K. Gopinat
Adversarial normalization for multi domain image segmentation
Image normalization is a critical step in medical imaging. This step is often
done on a per-dataset basis, preventing current segmentation algorithms from
the full potential of exploiting jointly normalized information across multiple
datasets. To solve this problem, we propose an adversarial normalization
approach for image segmentation which learns common normalizing functions
across multiple datasets while retaining image realism. The adversarial
training provides an optimal normalizer that improves both the segmentation
accuracy and the discrimination of unrealistic normalizing functions. Our
contribution therefore leverages common imaging information from multiple
domains. The optimality of our common normalizer is evaluated by combining
brain images from both infants and adults. Results on the challenging iSEG and
MRBrainS datasets reveal the potential of our adversarial normalization
approach for segmentation, with Dice improvements of up to 59.6% over the
baseline.Comment: Submitted to ISBI 202
Spectral Forests: Learning of Surface Data, Application to Cortical Parcellation
International audienceThis paper presents a new method for classifying surface datavia spectral representations of shapes. Our approach benefits classificationproblems that involve data living on surfaces, such as in cortical parcellation.For instance, current methods for labeling cortical points into surface parcelsoften involve a slow mesh deformation toward pre-labeled atlases, requiringas much as 4 hours with the established FreeSurfer. This may burden neurosciencestudies involving region-specific measurements. Learning techniquesoffer an attractive computational advantage, however, their representation ofspatial information, typically defined in a Euclidean domain, may be inadequatefor cortical parcellation. Indeed, cortical data resides on surfaces thatare highly variable in space and shape. Consequently, Euclidean representationsof surface data may be inconsistent across individuals. We proposeto fundamentally change the spatial representation of surface data, by exploitingspectral coordinates derived from the Laplacian eigenfunctions ofshapes. They have the advantage over Euclidean coordinates, to be geometryaware and to parameterize surfaces explicitly. This change of paradigm,from Euclidean to spectral representations, enables a classifier to be applieddirectly on surface data via spectral coordinates. In this paper, we decide tobuild upon the successful Random Decision Forests algorithm and improve itsspatial representation with spectral features. Our method, Spectral Forests,is shown to significantly improve the accuracy of cortical parcellations overstandard Random Decision Forests (74% versus 28% Dice overlaps), and produceaccuracy equivalent to FreeSurfer in a fraction of its time (23 secondsversus 3 to 4 hours)