A Neuroevolutionary Approach to Stochastic Inventory Control in Multi-Echelon Systems

Stochastic inventory control in multi-echelon systems poses hard problems in optimisation under uncertainty. Stochastic programming can solve small instances optimally, and approximately solve larger instances via scenario reduction techniques, but it cannot handle arbitrary nonlinear constraints or other non-standard features. Simulation optimisation is an alternative approach that has recently been applied to such problems, using policies that require only a few decision variables to be determined. However, to find optimal or near-optimal solutions we must consider exponentially large scenario trees with a corresponding number of decision variables. We propose instead a neuroevolutionary approach: using an artificial neural network to compactly represent the scenario tree, and training the network by a simulation-based evolutionary algorithm. We show experimentally that this method can quickly find high-quality plans using networks of a very simple form

Stochastic regret minimization for revenue management problems with nonstationary demands

We study an admission control model in revenue management with nonstationary and correlated demands over a finite discrete time horizon. The arrival probabilities are updated by current available information, that is, past customer arrivals and some other exogenous information. We develop a regret‐based framework, which measures the difference in revenue between a clairvoyant optimal policy that has access to all realizations of randomness a priori and a given feasible policy which does not have access to this future information. This regret minimization framework better spells out the trade‐offs of each accept/reject decision. We proceed using the lens of approximation algorithms to devise a conceptually simple regret‐parity policy. We show the proposed policy achieves 2‐approximation of the optimal policy in terms of total regret for a two‐class problem, and then extend our results to a multiclass problem with a fairness constraint. Our goal in this article is to make progress toward understanding the marriage between stochastic regret minimization and approximation algorithms in the realm of revenue management and dynamic resource allocation. © 2016 Wiley Periodicals, Inc. Naval Research Logistics 63: 433–448, 2016Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/135128/1/nav21704.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/135128/2/nav21704_am.pd

Dynamic Pricing through Sampling Based Optimization

In this paper we develop an approach to dynamic pricing that combines ideas from data-driven and robust optimization to address the uncertain and dynamic aspects of the problem. In our setting, a firm off ers multiple products to be sold over a fixed discrete time horizon. Each product sold consumes one or more resources, possibly sharing the same resources among di fferent products. The firm is given a fixed initial inventory of these resources and cannot replenish this inventory during the selling season. We assume there is uncertainty about the demand seen by the fi rm for each product and seek to determine a robust and dynamic pricing strategy that maximizes revenue over the time horizon. While the traditional robust optimization models are tractable, they give rise to static policies and are often too conservative. The main contribution of this paper is the exploration of closed-loop pricing policies for di fferent robust objectives, such as MaxMin, MinMax Regret and MaxMin Ratio. We introduce a sampling based optimization approach that can solve this problem in a tractable way, with a con fidence level and a robustness level based on the number of samples used. We will show how this methodology can be used for data-driven pricing or adapted for a random sampling optimization approach when limited information is known about the demand uncertainty. Finally, we compare the revenue performance of the di fferent models using numerical simulations, exploring the behavior of each model under diff erent sample sizes and sampling distributions.National Science Foundation (U.S.) (Grant 0556106-CMII)National Science Foundation (U.S.) (Grant 0824674-CMII)Singapore-MIT Allianc

Efficient Real-time Policies for Revenue Management Problems

This dissertation studies the development of provably near-optimal real-time prescriptive analytics solutions that are easily implementable in a dynamic business environment. We consider several stochastic control problems that are motivated by different applications of the practice of pricing and revenue management. Due to high dimensionality and the need for real-time decision making, it is computationally prohibitive to characterize the optimal controls for these problems. Therefore, we develop heuristic controls with simple decision rules that can be deployed in real-time at large scale, and then show theirs good theoretical and empirical performances. In particular, the first chapter studies the joint dynamic pricing and order fulfillment problem in the context of online retail, where a retailer sells multiple products to customers from different locations and fulfills orders through multiple fulfillment centers. The objective is to maximize the total expected profits, defined as the revenue minus the shipping cost. We propose heuristics where the real-time computations of pricing and fulfillment decisions are partially decoupled, and show their good performances compared to reasonable benchmarks. The second chapter studies a dynamic pricing problem where a firm faces price-sensitive customers arriving stochastically over time. Each customer consumes one unit of resource for a deterministic amount of time, after which the resource can be immediately used to serve new customers. We develop two heuristic controls and show that both are asymptotically optimal in the regime with large demand and supply. We further generalize both of the heuristic controls to the settings with multiple service types requiring different service times and with advance reservation. Lastly, the third chapter considers a general class of single-product dynamic pricing problems with inventory constraints, where the price-dependent demand function is unknown to the firm. We develop nonparametric dynamic pricing algorithms that do not assume any functional form of the demand model and show that, for one of the algorithm, its revenue loss compared to a clairvoyant matches the theoretic lower bound in asymptotic regime. In particular, the proposed algorithms generalize the classic bisection search method to a constrained setting with noisy observations.PHDBusiness AdministrationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/145995/1/leiyz_1.pd

Bounds for Markov Decision Processes

We consider the problem of producing lower bounds on the optimal cost-to-go function of a Markov decision problem. We present two approaches to this problem: one based on the methodology of approximate linear programming (ALP) and another based on the so-called martingale duality approach. We show that these two approaches are intimately connected. Exploring this connection leads us to the problem of finding "optimal" martingale penalties within the martingale duality approach which we dub the pathwise optimization (PO) problem. We show interesting cases where the PO problem admits a tractable solution and establish that these solutions produce tighter approximations than the ALP approach. © 2013 The Institute of Electrical and Electronics Engineers, Inc

Provably near-optimal algorithms for multi-stage stochastic optimization models in operations management

Thesis (Ph. D.)--Massachusetts Institute of Technology, Sloan School of Management, Operations Research Center, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 157-165).Many if not most of the core problems studied in operations management fall into the category of multi-stage stochastic optimization models, whereby one considers multiple, often correlated decisions to optimize a particular objective function under uncertainty on the system evolution over the future horizon. Unfortunately, computing the optimal policies is usually computationally intractable due to curse of dimensionality. This thesis is focused on providing provably near-optimal and tractable policies for some of these challenging models arising in the context of inventory control, capacity planning and revenue management; specifically, on the design of approximation algorithms that admit worst-case performance guarantees. In the first chapter, we develop new algorithmic approaches to compute provably near-optimal policies for multi-period stochastic lot-sizing inventory models with positive lead times, general demand distributions and dynamic forecast updates. The proposed policies have worst-case performance guarantees of 3 and typically perform very close to optimal in extensive computational experiments. We also describe a 6-approximation algorithm for the counterpart model under uniform capacity constraints. In the second chapter, we study a class of revenue management problems in systems with reusable resources and advanced reservations. A simple control policy called the class selection policy (CSP) is proposed based on solving a knapsack-type linear program (LP). We show that the CSP and its variants perform provably near-optimal in the Halfin- Whitt regime. The analysis is based on modeling the problem as loss network systems with advanced reservations. In particular, asymptotic upper bounds on the blocking probabilities are derived. In the third chapter, we examine the problem of capacity planning in joint ventures to meet stochastic demand in a newsvendor-type setting. When resources are heterogeneous, there exists a unique revenue-sharing contract such that the corresponding Nash Bargaining Solution, the Strong Nash Equilibrium, and the system optimal solution coincide. The optimal scheme rewards every participant proportionally to her marginal cost. When resources are homogeneous, there does not exist a revenue-sharing scheme which induces the system optimum. Nonetheless, we propose provably good revenue-sharing contracts which suggests that the reward should be inversely proportional to the marginal cost of each participant.by Cong Shi.Ph.D

Learning to Order for Inventory Systems with Lost Sales and Uncertain Supplies

We consider a stochastic lost-sales inventory control system with a lead time $L$ over a planning horizon $T$. Supply is uncertain, and is a function of the order quantity (due to random yield/capacity, etc). We aim to minimize the $T$-period cost, a problem that is known to be computationally intractable even under known distributions of demand and supply. In this paper, we assume that both the demand and supply distributions are unknown and develop a computationally efficient online learning algorithm. We show that our algorithm achieves a regret (i.e. the performance gap between the cost of our algorithm and that of an optimal policy over $T$ periods) of $O(L+\sqrt{T})$ when $L\geq\log(T)$. We do so by 1) showing our algorithm cost is higher by at most $O(L+\sqrt{T})$ for any $L\geq 0$ compared to an optimal constant-order policy under complete information (a well-known and widely-used algorithm) and 2) leveraging its known performance guarantee from the existing literature. To the best of our knowledge, a finite-sample $O(\sqrt{T})$ (and polynomial in $L$) regret bound when benchmarked against an optimal policy is not known before in the online inventory control literature. A key challenge in this learning problem is that both demand and supply data can be censored; hence only truncated values are observable. We circumvent this challenge by showing that the data generated under an order quantity $q^2$ allows us to simulate the performance of not only $q^2$ but also $q^1$ for all $q^1<q^2$, a key observation to obtain sufficient information even under data censoring. By establishing a high probability coupling argument, we are able to evaluate and compare the performance of different order policies at their steady state within a finite time horizon. Since the problem lacks convexity, we develop an active elimination method that adaptively rules out suboptimal solutions

Lp-Based Artificial Dependency for Probabilistic Etail Order Fulfillment

We consider an online multi-item retailer with multiple fulfillment facilities and finite inventory, with the objective of minimizing the expected shipping cost of fulfilling customer orders over a finite horizon. We approximate the stochastic dynamic programming formulation of the problem with an equivalent deterministic linear program, which we use to develop a probabilistic fulfillment heuristic that is provably optimal in the asymptotic sense. This first heuristic, however, relies on solving an LP that is exponential in the size of the input. Therefore, we subsequently provide another heuristic which solves an LP that is polynomial in the size of the input, and prove an upper bound on its asymptotic competitive ratio. This heuristic works by modifying the LP solution with artificial dependencies, with the resulting fractional variables used to probabilistically fulfill orders. A hardness result shows that asymptotically optimal policies that are computationally efficient cannot exist. Finally, we conduct numerical experiments that show that our heuristic's performance is very close to optimal for a range of parameters.http://deepblue.lib.umich.edu/bitstream/2027.42/108712/1/1250_ASinha.pd