Model Inversion (MI) attacks aim to recover the private training data from
the target model, which has raised security concerns about the deployment of
DNNs in practice. Recent advances in generative adversarial models have
rendered them particularly effective in MI attacks, primarily due to their
ability to generate high-fidelity and perceptually realistic images that
closely resemble the target data. In this work, we propose a novel Dynamic
Memory Model Inversion Attack (DMMIA) to leverage historically learned
knowledge, which interacts with samples (during the training) to induce diverse
generations. DMMIA constructs two types of prototypes to inject the information
about historically learned knowledge: Intra-class Multicentric Representation
(IMR) representing target-related concepts by multiple learnable prototypes,
and Inter-class Discriminative Representation (IDR) characterizing the
memorized samples as learned prototypes to capture more privacy-related
information. As a result, our DMMIA has a more informative representation,
which brings more diverse and discriminative generated results. Experiments on
multiple benchmarks show that DMMIA performs better than state-of-the-art MI
attack methods