11 research outputs found
Improving Cytoarchitectonic Segmentation of Human Brain Areas with Self-supervised Siamese Networks
Cytoarchitectonic parcellations of the human brain serve as anatomical
references in multimodal atlas frameworks. They are based on analysis of
cell-body stained histological sections and the identification of borders
between brain areas. The de-facto standard involves a semi-automatic,
reproducible border detection, but does not scale with high-throughput imaging
in large series of sections at microscopical resolution. Automatic
parcellation, however, is extremely challenging due to high variation in the
data, and the need for a large field of view at microscopic resolution. The
performance of a recently proposed Convolutional Neural Network model that
addresses this problem especially suffers from the naturally limited amount of
expert annotations for training. To circumvent this limitation, we propose to
pre-train neural networks on a self-supervised auxiliary task, predicting the
3D distance between two patches sampled from the same brain. Compared to a
random initialization, fine-tuning from these networks results in significantly
better segmentations. We show that the self-supervised model has implicitly
learned to distinguish several cortical brain areas -- a strong indicator that
the proposed auxiliary task is appropriate for cytoarchitectonic mapping.Comment: Accepted at MICCAI 201
Self-supervised Contrastive Video-Speech Representation Learning for Ultrasound
In medical imaging, manual annotations can be expensive to acquire and
sometimes infeasible to access, making conventional deep learning-based models
difficult to scale. As a result, it would be beneficial if useful
representations could be derived from raw data without the need for manual
annotations. In this paper, we propose to address the problem of
self-supervised representation learning with multi-modal ultrasound
video-speech raw data. For this case, we assume that there is a high
correlation between the ultrasound video and the corresponding narrative speech
audio of the sonographer. In order to learn meaningful representations, the
model needs to identify such correlation and at the same time understand the
underlying anatomical features. We designed a framework to model the
correspondence between video and audio without any kind of human annotations.
Within this framework, we introduce cross-modal contrastive learning and an
affinity-aware self-paced learning scheme to enhance correlation modelling.
Experimental evaluations on multi-modal fetal ultrasound video and audio show
that the proposed approach is able to learn strong representations and
transfers well to downstream tasks of standard plane detection and eye-gaze
prediction.Comment: MICCAI 2020 (early acceptance
Revisiting Rubik's Cube: Self-supervised Learning with Volume-wise Transformation for 3D Medical Image Segmentation
Deep learning highly relies on the quantity of annotated data. However, the
annotations for 3D volumetric medical data require experienced physicians to
spend hours or even days for investigation. Self-supervised learning is a
potential solution to get rid of the strong requirement of training data by
deeply exploiting raw data information. In this paper, we propose a novel
self-supervised learning framework for volumetric medical images. Specifically,
we propose a context restoration task, i.e., Rubik's cube++, to pre-train 3D
neural networks. Different from the existing context-restoration-based
approaches, we adopt a volume-wise transformation for context permutation,
which encourages network to better exploit the inherent 3D anatomical
information of organs. Compared to the strategy of training from scratch,
fine-tuning from the Rubik's cube++ pre-trained weight can achieve better
performance in various tasks such as pancreas segmentation and brain tissue
segmentation. The experimental results show that our self-supervised learning
method can significantly improve the accuracy of 3D deep learning networks on
volumetric medical datasets without the use of extra data.Comment: Accepted by MICCAI 202
Improving Cytoarchitectonic Segmentation of Human Brain Areas with Self-supervised Siamese Networks
Cytoarchitectonic parcellations of the human brain serve as anatomical
references in multimodal atlas frameworks. They are based on analysis of
cell-body stained histological sections and the identification of borders
between brain areas. The de-facto standard involves a semi-automatic,
reproducible border detection, but does not scale with high-throughput imaging
in large series of sections at microscopical resolution. Automatic
parcellation, however, is extremely challenging due to high variation in the
data, and the need for a large field of view at microscopic resolution. The
performance of a recently proposed Convolutional Neural Network model that
addresses this problem especially suffers from the naturally limited amount of
expert annotations for training. To circumvent this limitation, we propose to
pre-train neural networks on a self-supervised auxiliary task, predicting the
3D distance between two patches sampled from the same brain. Compared to a
random initialization, fine-tuning from these networks results in significantly
better segmentations. We show that the self-supervised model has implicitly
learned to distinguish several cortical brain areas -- a strong indicator that
the proposed auxiliary task is appropriate for cytoarchitectonic mapping.Comment: Accepted at MICCAI 201