The limited availability of large image datasets, mainly due to data privacy
and differences in acquisition protocols or hardware, is a significant issue in
the development of accurate and generalizable machine learning methods in
medicine. This is especially the case for Magnetic Resonance (MR) images, where
different MR scanners introduce a bias that limits the performance of a machine
learning model. We present a novel method that learns to ignore the
scanner-related features present in MR images, by introducing specific
additional constraints on the latent space. We focus on a real-world
classification scenario, where only a small dataset provides images of all
classes. Our method \textit{Learn to Ignore (L2I)} outperforms state-of-the-art
domain adaptation methods on a multi-site MR dataset for a classification task
between multiple sclerosis patients and healthy controls