Fine-tuning diffusion models through personalized datasets is an acknowledged
method for improving generation quality across downstream tasks, which,
however, often inadvertently generates unintended concepts such as watermarks
and QR codes, attributed to the limitations in image sources and collecting
methods within specific downstream tasks. Existing solutions suffer from
eliminating these unintentionally learned implicit concepts, primarily due to
the dependency on the model's ability to recognize concepts that it actually
cannot discern. In this work, we introduce Geom-Erasing, a novel approach that
successfully removes the implicit concepts with either an additional accessible
classifier or detector model to encode geometric information of these concepts
into text domain. Moreover, we propose Implicit Concept, a novel image-text
dataset imbued with three implicit concepts (i.e., watermarks, QR codes, and
text) for training and evaluation. Experimental results demonstrate that
Geom-Erasing not only identifies but also proficiently eradicates implicit
concepts, revealing a significant improvement over the existing methods. The
integration of geometric information marks a substantial progression in the
precise removal of implicit concepts in diffusion models