5 research outputs found
Embracing the chaos: analysis and diagnosis of numerical instability in variational flows
In this paper, we investigate the impact of numerical instability on the
reliability of sampling, density evaluation, and evidence lower bound (ELBO)
estimation in variational flows. We first empirically demonstrate that common
flows can exhibit a catastrophic accumulation of error: the numerical flow map
deviates significantly from the exact map -- which affects sampling -- and the
numerical inverse flow map does not accurately recover the initial input --
which affects density and ELBO computations. Surprisingly though, we find that
results produced by flows are often accurate enough for applications despite
the presence of serious numerical instability. In this work, we treat
variational flows as dynamical systems, and leverage shadowing theory to
elucidate this behavior via theoretical guarantees on the error of sampling,
density evaluation, and ELBO estimation. Finally, we develop and empirically
test a diagnostic procedure that can be used to validate results produced by
numerically unstable flows in practice