The lack of standardization is a prominent issue in magnetic resonance (MR)
imaging. This often causes undesired contrast variations due to differences in
hardware and acquisition parameters. In recent years, MR harmonization using
image synthesis with disentanglement has been proposed to compensate for the
undesired contrast variations. Despite the success of existing methods, we
argue that three major improvements can be made. First, most existing methods
are built upon the assumption that multi-contrast MR images of the same subject
share the same anatomy. This assumption is questionable since different MR
contrasts are specialized to highlight different anatomical features. Second,
these methods often require a fixed set of MR contrasts for training (e.g.,
both Tw-weighted and T2-weighted images must be available), which limits their
applicability. Third, existing methods generally are sensitive to imaging
artifacts. In this paper, we present a novel approach, Harmonization with
Attention-based Contrast, Anatomy, and Artifact Awareness (HACA3), to address
these three issues. We first propose an anatomy fusion module that enables
HACA3 to respect the anatomical differences between MR contrasts. HACA3 is also
robust to imaging artifacts and can be trained and applied to any set of MR
contrasts. Experiments show that HACA3 achieves state-of-the-art performance
under multiple image quality metrics. We also demonstrate the applicability of
HACA3 on downstream tasks with diverse MR datasets acquired from 21 sites with
different field strengths, scanner platforms, and acquisition protocols