4 research outputs found
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe
Privacy concerns have attracted increasing attention in data-driven products
due to the tendency of machine learning models to memorize sensitive training
data. Generating synthetic versions of such data with a formal privacy
guarantee, such as differential privacy (DP), provides a promising path to
mitigating these privacy concerns, but previous approaches in this direction
have typically failed to produce synthetic data of high quality. In this work,
we show that a simple and practical recipe in the text domain is effective:
simply fine-tuning a pretrained generative language model with DP enables the
model to generate useful synthetic text with strong privacy protection. Through
extensive empirical analyses on both benchmark and private customer data, we
demonstrate that our method produces synthetic text that is competitive in
terms of utility with its non-private counterpart, meanwhile providing strong
protection against potential privacy leakages.Comment: ACL 2023 Main Conference (Honorable Mention