3 research outputs found
Efficient and Accurate Optimal Transport with Mirror Descent and Conjugate Gradients
We design a novel algorithm for optimal transport by drawing from the
entropic optimal transport, mirror descent and conjugate gradients literatures.
Our scalable and GPU parallelizable algorithm is able to compute the
Wasserstein distance with extreme precision, reaching relative error rates of
without numerical stability issues. Empirically, the algorithm
converges to high precision solutions more quickly in terms of wall-clock time
than a variety of algorithms including log-domain stabilized Sinkhorn's
Algorithm. We provide careful ablations with respect to algorithm and problem
parameters, and present benchmarking over upsampled MNIST images, comparing to
various recent algorithms over high-dimensional problems. The results suggest
that our algorithm can be a useful addition to the practitioner's optimal
transport toolkit
Maximum Entropy Model Correction in Reinforcement Learning
We propose and theoretically analyze an approach for planning with an
approximate model in reinforcement learning that can reduce the adverse impact
of model error. If the model is accurate enough, it accelerates the convergence
to the true value function too. One of its key components is the MaxEnt Model
Correction (MoCo) procedure that corrects the model's next-state distributions
based on a Maximum Entropy density estimation formulation. Based on MoCo, we
introduce the Model Correcting Value Iteration (MoCoVI) algorithm, and its
sampled-based variant MoCoDyna. We show that MoCoVI and MoCoDyna's convergence
can be much faster than the conventional model-free algorithms. Unlike
traditional model-based algorithms, MoCoVI and MoCoDyna effectively utilize an
approximate model and still converge to the correct value function