Brain tumour segmentation plays a key role in computer-assisted surgery. Deep
neural networks have increased the accuracy of automatic segmentation
significantly, however these models tend to generalise poorly to different
imaging modalities than those for which they have been designed, thereby
limiting their applications. For example, a network architecture initially
designed for brain parcellation of monomodal T1 MRI can not be easily
translated into an efficient tumour segmentation network that jointly utilises
T1, T1c, Flair and T2 MRI. To tackle this, we propose a novel scalable
multimodal deep learning architecture using new nested structures that
explicitly leverage deep features within or across modalities. This aims at
making the early layers of the architecture structured and sparse so that the
final architecture becomes scalable to the number of modalities. We evaluate
the scalable architecture for brain tumour segmentation and give evidence of
its regularisation effect compared to the conventional concatenation approach.Comment: Paper accepted at MICCAI 201
