Markov decision processes continue to gain in popularity for modeling a wide
range of applications ranging from analysis of supply chains and queuing
networks to cognitive science and control of autonomous vehicles. Nonetheless,
they tend to become numerically intractable as the size of the model grows
fast. Recent works use machine learning techniques to overcome this crucial
issue, but with no convergence guarantee. This note provides a brief overview
of literature on solving large Markov decision processes, and exploiting them
to solve important combinatorial optimization problems