Generative models have demonstrated impressive results in vision, language,
and speech. However, even with massive datasets, they struggle with precision,
generating physically invalid or factually incorrect data. This is particularly
problematic when the generated data must satisfy constraints, for example, to
meet product specifications in engineering design or to adhere to the laws of
physics in a natural scene. To improve precision while preserving diversity and
fidelity, we propose a novel training mechanism that leverages datasets of
constraint-violating data points, which we consider invalid. Our approach
minimizes the divergence between the generative distribution and the valid
prior while maximizing the divergence with the invalid distribution. We
demonstrate how generative models like GANs and DDPMs that we augment to train
with invalid data vastly outperform their standard counterparts which solely
train on valid data points. For example, our training procedure generates up to
98 % fewer invalid samples on 2D densities, improves connectivity and stability
four-fold on a stacking block problem, and improves constraint satisfaction by
15 % on a structural topology optimization benchmark in engineering design. We
also analyze how the quality of the invalid data affects the learning procedure
and the generalization properties of models. Finally, we demonstrate
significant improvements in sample efficiency, showing that a tenfold increase
in valid samples leads to a negligible difference in constraint satisfaction,
while less than 10 % invalid samples lead to a tenfold improvement. Our
proposed mechanism offers a promising solution for improving precision in
generative models while preserving diversity and fidelity, particularly in
domains where constraint satisfaction is critical and data is limited, such as
engineering design, robotics, and medicine