3D Convolutional Neural Networks for Tumor Segmentation using Long-range 2D Context

We present an efficient deep learning approach for the challenging task of tumor segmentation in multisequence MR images. In recent years, Convolutional Neural Networks (CNN) have achieved state-of-the-art performances in a large variety of recognition tasks in medical imaging. Because of the considerable computational cost of CNNs, large volumes such as MRI are typically processed by subvolumes, for instance slices (axial, coronal, sagittal) or small 3D patches. In this paper we introduce a CNN-based model which efficiently combines the advantages of the short-range 3D context and the long-range 2D context. To overcome the limitations of specific choices of neural network architectures, we also propose to merge outputs of several cascaded 2D-3D models by a voxelwise voting strategy. Furthermore, we propose a network architecture in which the different MR sequences are processed by separate subnetworks in order to be more robust to the problem of missing MR sequences. Finally, a simple and efficient algorithm for training large CNN models is introduced. We evaluate our method on the public benchmark of the BRATS 2017 challenge on the task of multiclass segmentation of malignant brain tumors. Our method achieves good performances and produces accurate segmentations with median Dice scores of 0.918 (whole tumor), 0.883 (tumor core) and 0.854 (enhancing core). Our approach can be naturally applied to various tasks involving segmentation of lesions or organs.Comment: Submitted to the journal Computerized Medical Imaging and Graphic

Duality, rigidity and planar parallax

We investigate the geometry of two views of seven points, four of which are coplanar, and the geometry of three views of six points, four of which are coplanar. We prove that the two are dual, and that the fundamental geometric constraints in each case are encapsulated by a planar homology. The work unifies a number of previously diverse results related to planar parallax, duality and planar homologies. In addition, we make a number of practical contributions, including formulae for computing the distance of the cameras from a distinguished world plane and formulae for structure computations. We show that the trifocal tensor is obtained uniquely from three views of six points, four of which are coplanar, and give a simple interpretation of the trifocal geometry. We give examples of these computations on real images

Immediate ROI search for 3-D medical images

The objective of this work is a scalable, real-time, visual search engine for 3-D medical images, where a user is able to select a query Region Of Interest (ROI) and automatically detect the corresponding regions within all returned images. We make three contributions: (i) we show that with appropriate off-line processing, images can be retrieved and ROIs registered in real time; (ii) we propose and evaluate a number of scalable exemplar-based image registration schemes; (iii) we propose a discriminative method for learning to rank the returned images based on the content of the ROI. The retrieval system is demonstrated on MRI data from the ADNI dataset, and it is shown that the learnt ranking function outperforms the baseline

Atlas encoding by randomized forests for efficient label propagation

Abstract We propose a method for multi-atlas label propagation based on encoding the individual atlases by randomized classification forests. Most current approaches perform a non-linear registration between all atlases and the target image, followed by a sophisticated fusion scheme. While these approaches can achieve high accuracy, in general they do so at high computational cost. This negatively affects the scalability to large databases and experimentation. To tackle this issue, we propose to use a small and deep classification forest to encode each atlas individually in reference to an aligned probabilistic atlas, resulting in an Atlas Forest (AF). At test time, each AF yields a probabilistic label estimate, and fusion is done by averaging. Our scheme performs only one registration per target image, achieves good results with a simple fusion scheme, and allows for efficient experimentation. In contrast to standard forest schemes, incorporation of new scans is possible without retraining, and target-specific selection of atlases remains possible. The evaluation on three different databases shows accuracy at the level of the state of the art, at a significantly lower runtime

Spectral Forests: Learning of Surface Data, Application to Cortical Parcellation

International audienceThis paper presents a new method for classifying surface datavia spectral representations of shapes. Our approach benefits classificationproblems that involve data living on surfaces, such as in cortical parcellation.For instance, current methods for labeling cortical points into surface parcelsoften involve a slow mesh deformation toward pre-labeled atlases, requiringas much as 4 hours with the established FreeSurfer. This may burden neurosciencestudies involving region-specific measurements. Learning techniquesoffer an attractive computational advantage, however, their representation ofspatial information, typically defined in a Euclidean domain, may be inadequatefor cortical parcellation. Indeed, cortical data resides on surfaces thatare highly variable in space and shape. Consequently, Euclidean representationsof surface data may be inconsistent across individuals. We proposeto fundamentally change the spatial representation of surface data, by exploitingspectral coordinates derived from the Laplacian eigenfunctions ofshapes. They have the advantage over Euclidean coordinates, to be geometryaware and to parameterize surfaces explicitly. This change of paradigm,from Euclidean to spectral representations, enables a classifier to be applieddirectly on surface data via spectral coordinates. In this paper, we decide tobuild upon the successful Random Decision Forests algorithm and improve itsspatial representation with spectral features. Our method, Spectral Forests,is shown to significantly improve the accuracy of cortical parcellations overstandard Random Decision Forests (74% versus 28% Dice overlaps), and produceaccuracy equivalent to FreeSurfer in a fraction of its time (23 secondsversus 3 to 4 hours)

Deep Learning with Mixed Supervision for Brain Tumor Segmentation

International audienceMost of the current state-of-the-art methods for tumor segmentation are based on machine learning models trained on manually segmented images. This type of training data is particularly costly, as manual delineation of tumors is not only time-consuming but also requires medical expertise. On the other hand, images with a provided global label (indicating presence or absence of a tumor) are less informative but can be obtained at a substantially lower cost. In this paper, we propose to use both types of training data (fully-annotated and weakly-annotated) to train a deep learning model for segmentation. The idea of our approach is to extend segmentation networks with an additional branch performing image-level classification. The model is jointly trained for segmentation and classification tasks in order to exploit information contained in weakly-annotated images while preventing the network to learn features which are irrelevant for the segmentation task. We evaluate our method on the challenging task of brain tumor seg-mentation in Magnetic Resonance images from BRATS 2018 challenge. We show that the proposed approach provides a significant improvement of seg-mentation performance compared to the standard supervised learning. The observed improvement is proportional to the ratio between weakly-annotated and fully-annotated images available for training

Recognizing cardiac magnetic resonance acquisition planes

International audienceIn this paper we propose a method for automatic wrangling of missing or noisy acquisition plane information of cardiac magnetic resonance images in order to simplify case filtering and image lookup in large collections of cardiac data. To recognize standard cardiac planes we use features based on image miniatures combined with a decision forest classifier. We show that augmenting the dataset with a set of nondestructive transformations can improve classification accuracy. Our approach compares favorably to the state of the art while requiring fewer manual annotations

Technical Appendix on Sparse Bayesian Regression

Technical appendix to a MICCAI 2014 conference paper. Loïc Le Folgoc, Hervé Delingette, Antonio Criminisi, and Nicholas Ayache: Sparse Bayesian RegistrationInternational audienceWe report the technical details for a sparse bayesian approach to regression. It can be seen as an extension of the Relevance Vector Machine of Tipping to a more general setting that can handle vector-valued regression and generic quadratic priors

Deep Neural Decision Forests

Abstract We present Deep Neural Decision Forests -a novel approach that unifies classification trees with the representation learning functionality known from deep convolutional networks, by training them in an end-to-end manner. To combine these two worlds, we introduce a stochastic and differentiable decision tree model, which steers the representation learning usually conducted in the initial layers of a (deep) convolutional network. Our model differs from conventional deep networks because a decision forest provides the final predictions and it differs from conventional decision forests since we propose a principled, joint and global optimization of split and leaf node parameters. We show experimental results on benchmark machine learning datasets like MNIST and ImageNet and find onpar or superior results when compared to state-of-the-art deep models. Most remarkably, we obtain Top5-Errors of only 7.84%/6.38% on ImageNet validation data when integrating our forests in a single-crop, single/seven model GoogLeNet architecture, respectively. Thus, even without any form of training data set augmentation we are improving on the 6.67% error obtained by the best GoogLeNet architecture (7 models, 144 crops)