Large Markov Decision Processes and Combinatorial Optimization

Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend to become numerically intractable as the size of the model grows fast. Recent works use machine learning techniques to overcome this crucial issue, but with no convergence guarantee. This note provides a brief overview of literature on solving large Markov decision processes, and exploiting them to solve important combinatorial optimization problems

How green is a lean supply chain?

This article presents a supply chain planning model that can be used to investigate tradeoffs between cost and environmental degradation including carbon emissions, energy consumption and waste generation. The model also incorporates other aspects of real world supply chains such as multiple transport lot sizing and flexible holding capacity of warehouses. The application of the model and solution method is investigated in an actual case problem. Our analysis of the numerical results focuses on investigating relationship between lean practices and green outcomes. We find that (1) not all lean interventions at the tactical supply chain planning level result in green benefits, and (2) an agile supply chain is the greenest and most efficient alternative when compared to strictly lean and centralized situations

Hamiltonian cycles and subsets of discounted occupational measures

We study a certain polytope arising from embedding the Hamiltonian cycle problem in a discounted Markov decision process. The Hamiltonian cycle problem can be reduced to finding particular extreme points of a certain polytope associated with the input graph. This polytope is a subset of the space of discounted occupational measures. We characterize the feasible bases of the polytope for a general input graph $G$, and determine the expected numbers of different types of feasible bases when the underlying graph is random. We utilize these results to demonstrate that augmenting certain additional constraints to reduce the polyhedral domain can eliminate a large number of feasible bases that do not correspond to Hamiltonian cycles. Finally, we develop a random walk algorithm on the feasible bases of the reduced polytope and present some numerical results. We conclude with a conjecture on the feasible bases of the reduced polytope.Comment: revised based on referees comment

Tactical supply chain planning under a carbon tax policy scheme: a case study

Greenhouse gas emissions are receiving greater scrutiny in many countries due to international forces to reduce anthropogenic global climate change. Industry and their supply chains represent a major source of these emissions. This paper presents a tactical supply chain planning model that integrates economic and carbon emission objectives under a carbon tax policy scheme. A modified Cross-Entropy solution method is adopted to solve the proposed nonlinear supply chain planning model. Numerical experiments are completed utilizing data from an actual organization in Australia where a carbon tax is in operation. The analyses of the numerical results provide important organizational and policy insights on (1) the financial and emissions reduction impacts of a carbon tax at the tactical planning level, (2) the use of cost/emission tradeoff analysis for making informed decisions on investments, (3) the way to price carbon for maximum environmental returns per dollar increase in supply chain cost

On transition matrices of Markov chains corresponding to Hamiltonian cycles

International audienceIn this paper, we present some algebraic properties of a particular class of probability transition matrices, namely, Hamiltonian transition matrices. Each matrix P in this class corresponds to a Hamiltonian cycle in a given graph G on n nodes and to an irreducible, periodic, Markov chain. We show that a number of important matrices traditionally associated with Markov chains, namely, the stationary, fundamental, deviation and the hitting time matrix all have elegant expansions in the first n−1 powers of P , whose coefficients can be explicitly derived. We also consider the resolvent-like matrices associated with any given Hamiltonian cycle and its reverse cycle and prove an identity about the product of these matrices

Tactical supply chain planning under a carbon tax policy scheme: A case study

This paper was accepted for publication in the journal International Journal of Production Economics and the definitive published version is available at http://dx.doi.org/10.1016/j.ijpe.2014.12.015Greenhouse gas emissions are receiving greater scrutiny in many countries due to international forces to reduce anthropogenic global climate change. Industry and their supply chains represent a major source of these emissions. This paper presents a tactical supply chain planning model that integrates economic and carbon emission objectives under a carbon tax policy scheme. A modified Cross-Entropy solution method is adopted to solve the proposed nonlinear supply chain planning model. Numerical experiments are completed utilizing data from an actual organization in Australia where a carbon tax is in operation. The analyses of the numerical results provide important organizational and policy insights on (1) the financial and emissions reduction impacts of a carbon tax at the tactical planning level, (2) the use of cost/emission tradeoff analysis for making informed decisions on investments, (3) the way to price carbon for maximum environmental returns per dollar increase in supply chain cost