Label smoothing -- using softened labels instead of hard ones -- is a widely
adopted regularization method for deep learning, showing diverse benefits such
as enhanced generalization and calibration. Its implications for preserving
model privacy, however, have remained unexplored. To fill this gap, we
investigate the impact of label smoothing on model inversion attacks (MIAs),
which aim to generate class-representative samples by exploiting the knowledge
encoded in a classifier, thereby inferring sensitive information about its
training data. Through extensive analyses, we uncover that traditional label
smoothing fosters MIAs, thereby increasing a model's privacy leakage. Even
more, we reveal that smoothing with negative factors counters this trend,
impeding the extraction of class-related information and leading to privacy
preservation, beating state-of-the-art defenses. This establishes a practical
and powerful novel way for enhancing model resilience against MIAs.Comment: 23 pages, 8 tables, 8 figure