Mini-batch optimal transport (m-OT) has been successfully used in practical
applications that involve probability measures with intractable density, or
probability measures with a very high number of supports. The m-OT solves
several sparser optimal transport problems and then returns the average of
their costs and transportation plans. Despite its scalability advantage, the
m-OT does not consider the relationship between mini-batches which leads to
undesirable estimation. Moreover, the m-OT does not approximate a proper metric
between probability measures since the identity property is not satisfied. To
address these problems, we propose a novel mini-batching scheme for optimal
transport, named Batch of Mini-batches Optimal Transport (BoMb-OT), that finds
the optimal coupling between mini-batches and it can be seen as an
approximation to a well-defined distance on the space of probability measures.
Furthermore, we show that the m-OT is a limit of the entropic regularized
version of the BoMb-OT when the regularized parameter goes to infinity.
Finally, we carry out extensive experiments to show that the BoMb-OT can
estimate a better transportation plan between two original measures than the
m-OT. It leads to a favorable performance of the BoMb-OT in the matching and
color transfer tasks. Furthermore, we observe that the BoMb-OT also provides a
better objective loss than the m-OT for doing approximate Bayesian computation,
estimating parameters of interest in parametric generative models, and learning
non-parametric generative models with gradient flow.Comment: 36 pages, 20 figure
