A Structured Multiarmed Bandit Problem and the Greedy Policy

We consider a multiarmed bandit problem where the expected reward of each arm is a linear function of an unknown scalar with a prior distribution. The objective is to choose a sequence of arms that maximizes the expected total (or discounted total) reward. We demonstrate the effectiveness of a greedy policy that takes advantage of the known statistical correlation structure among the arms. In the infinite horizon discounted reward setting, we show that the greedy and optimal policies eventually coincide, and both settle on the best arm. This is in contrast with the Incomplete Learning Theorem for the case of independent arms. In the total reward setting, we show that the cumulative Bayes risk after T periods under the greedy policy is at most O(logT), which is smaller than the lower bound of Omega(log[superscript 2] T) established by Lai for a general, but different, class of bandit problems. We also establish the tightness of our bounds. Theoretical and numerical results show that the performance of our policy scales independently of the number of arms.National Science Foundation (Grants DMS-0732196, CMMI-0746844, and ECCS-0701623)Kenan-Flagler Business SchoolUniversity of Chicago. Graduate School of Busines

Evaluating Count Prioritization Procedures for Improving Inventory Accuracy in Retail Stores

We compare several approaches for generating a prioritized list of products to be counted in a retail store, with the objective of detecting inventory record inaccuracy and unknown out-of-stocks. Our study evaluates these approaches using data from inventory audits we conducted at European home and personal care retailer dm-drogerie markt. We consider both "rule-based" approaches, which sort products based on heuristic indices, and "model-based" approaches, which maintain probability distributions for the true inventory levels updated based on sales and replenishment observations. Our results support arguments for both rule-based and model-based approaches. We find that model-based approaches provide versatile visibility into inventory states and are useful for a broad range of objectives, but that rule-based approaches are also effective as long as they are matched to the retailer's goal. We find that "high-activity" rule-based policies that favor items with high sales volumes, inventory levels, and past errors are more effective at detecting inventory discrepancies. A "low-activity" rule-based policy based on low recorded inventory levels, on the other hand, is more effective at detecting unknown out-of-stocks. Our approach can be replicated at other retailers interested in customized optimization of their counting programs

Retail Inventory Management When Records Are Inaccurate

Inventory record inaccuracy is a significant problem for retailers using automated inventory management systems. In this paper, we consider an intelligent inventory management tool that accounts for record inaccuracy using a Bayesian belief of the physical inventory level. We assume that excess demands are lost and unobserved, in which case sales data reveal information about physical inventory levels. We show that a probability distribution on physical inventory levels is a sufficient summary of past sales and replenishment observations, and that this probability distribution can be efficiently updated in a Bayesian fashion as observations are accumulated. We also demonstrate the use of this distribution as the basis for practical replenishment and inventory audit policies and illustrate how the needed parameters can be estimated using data from a large national retailer. Our replenishment policies avoid the problem of "freezing," in which a physical inventory position persists at zero while the corresponding record is positive. In addition, simulation studies show that our replenishment policies recoup much of the cost of inventory record inaccuracy, and that our audit policy significantly outperforms the popular "zero balance walk" audit policy.retail execution, inventory control, record inaccuracy, inventory shrinkage, Bayes rule