Normalizing flows are tractable density models that can approximate
complicated target distributions, e.g. Boltzmann distributions of physical
systems. However, current methods for training flows either suffer from
mode-seeking behavior, use samples from the target generated beforehand by
expensive MCMC simulations, or use stochastic losses that have very high
variance. To avoid these problems, we augment flows with annealed importance
sampling (AIS) and minimize the mass covering α-divergence with
α=2, which minimizes importance weight variance. Our method, Flow AIS
Bootstrap (FAB), uses AIS to generate samples in regions where the flow is a
poor approximation of the target, facilitating the discovery of new modes. We
target with AIS the minimum variance distribution for the estimation of the
α-divergence via importance sampling. We also use a prioritized buffer
to store and reuse AIS samples. These two features significantly improve FAB's
performance. We apply FAB to complex multimodal targets and show that we can
approximate them very accurately where previous methods fail. To the best of
our knowledge, we are the first to learn the Boltzmann distribution of the
alanine dipeptide molecule using only the unnormalized target density and
without access to samples generated via Molecular Dynamics (MD) simulations:
FAB produces better results than training via maximum likelihood on MD samples
while using 100 times fewer target evaluations. After reweighting samples with
importance weights, we obtain unbiased histograms of dihedral angles that are
almost identical to the ground truth ones