Deep generative models have become useful for synthetic data generation,
particularly population synthesis. The models implicitly learn the probability
distribution of a dataset and can draw samples from a distribution. Several
models have been proposed, but their performance is only tested on a single
cross-sectional sample. The implementation of population synthesis on single
datasets is seen as a drawback that needs further studies to explore the
robustness of the models on multiple datasets. While comparing with the real
data can increase trust and interpretability of the models, techniques to
evaluate deep generative models' robustness for population synthesis remain
underexplored. In this study, we present bootstrap confidence interval for the
deep generative models, an approach that computes efficient confidence
intervals for mean errors predictions to evaluate the robustness of the models
to multiple datasets. Specifically, we adopt the tabular-based Composite Travel
Generative Adversarial Network (CTGAN) and Variational Autoencoder (VAE), to
estimate the distribution of the population, by generating agents that have
tabular data using several samples over time from the same study area. The
models are implemented on multiple travel diaries of Montreal Origin-
Destination Survey of 2008, 2013, and 2018 and compare the predictive
performance under varying sample sizes from multiple surveys. Results show that
the predictive errors of CTGAN have narrower confidence intervals indicating
its robustness to multiple datasets of the varying sample sizes when compared
to VAE. Again, the evaluation of model robustness against varying sample size
shows a minimal decrease in model performance with decrease in sample size.
This study directly supports agent-based modelling by enabling finer synthetic
generation of populations in a reliable environment.Comment: arXiv admin note: text overlap with arXiv:2203.03489,
arXiv:1909.07689 by other author