A New Optimal Stepsize For Approximate Dynamic Programming

Approximate dynamic programming (ADP) has proven itself in a wide range of applications spanning large-scale transportation problems, health care, revenue management, and energy systems. The design of effective ADP algorithms has many dimensions, but one crucial factor is the stepsize rule used to update a value function approximation. Many operations research applications are computationally intensive, and it is important to obtain good results quickly. Furthermore, the most popular stepsize formulas use tunable parameters and can produce very poor results if tuned improperly. We derive a new stepsize rule that optimizes the prediction error in order to improve the short-term performance of an ADP algorithm. With only one, relatively insensitive tunable parameter, the new rule adapts to the level of noise in the problem and produces faster convergence in numerical experiments.Comment: Matlab files are included with the paper sourc

Near-Optimal Bisection Search for Nonparametric Dynamic Pricing with Inventory Constraint

We consider a single-product revenue management problem with an inventory constraint and unknown, noisy, demand function. The objective of the fi rm is to dynamically adjust the prices to maximize total expected revenue. We restrict our scope to the nonparametric approach where we only assume some common regularity conditions on the demand function instead of a speci fic functional form. We propose a family of pricing heuristics that successfully balance the tradeo ff between exploration and exploitation. The idea is to generalize the classic bisection search method to a problem that is a ffected both by stochastic noise and an inventory constraint. Our algorithm extends the bisection method to produce a sequence of pricing intervals that converge to the optimal static price with high probability. Using regret (the revenue loss compared to the deterministic pricing problem for a clairvoyant) as the performance metric, we show that one of our heuristics exactly matches the theoretical asymptotic lower bound that has been previously shown to hold for any feasible pricing heuristic. Although the results are presented in the context of revenue management problems, our analysis of the bisection technique for stochastic optimization with learning can be potentially applied to other application areas.http://deepblue.lib.umich.edu/bitstream/2027.42/108717/1/1252_Sinha.pd

Efficient Real-time Policies for Revenue Management Problems

This dissertation studies the development of provably near-optimal real-time prescriptive analytics solutions that are easily implementable in a dynamic business environment. We consider several stochastic control problems that are motivated by different applications of the practice of pricing and revenue management. Due to high dimensionality and the need for real-time decision making, it is computationally prohibitive to characterize the optimal controls for these problems. Therefore, we develop heuristic controls with simple decision rules that can be deployed in real-time at large scale, and then show theirs good theoretical and empirical performances. In particular, the first chapter studies the joint dynamic pricing and order fulfillment problem in the context of online retail, where a retailer sells multiple products to customers from different locations and fulfills orders through multiple fulfillment centers. The objective is to maximize the total expected profits, defined as the revenue minus the shipping cost. We propose heuristics where the real-time computations of pricing and fulfillment decisions are partially decoupled, and show their good performances compared to reasonable benchmarks. The second chapter studies a dynamic pricing problem where a firm faces price-sensitive customers arriving stochastically over time. Each customer consumes one unit of resource for a deterministic amount of time, after which the resource can be immediately used to serve new customers. We develop two heuristic controls and show that both are asymptotically optimal in the regime with large demand and supply. We further generalize both of the heuristic controls to the settings with multiple service types requiring different service times and with advance reservation. Lastly, the third chapter considers a general class of single-product dynamic pricing problems with inventory constraints, where the price-dependent demand function is unknown to the firm. We develop nonparametric dynamic pricing algorithms that do not assume any functional form of the demand model and show that, for one of the algorithm, its revenue loss compared to a clairvoyant matches the theoretic lower bound in asymptotic regime. In particular, the proposed algorithms generalize the classic bisection search method to a constrained setting with noisy observations.PHDBusiness AdministrationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/145995/1/leiyz_1.pd