Ranking and Selection under Input Uncertainty: Fixed Confidence and Fixed Budget

In stochastic simulation, input uncertainty (IU) is caused by the error in estimating the input distributions using finite real-world data. When it comes to simulation-based Ranking and Selection (R&S), ignoring IU could lead to the failure of many existing selection procedures. In this paper, we study R&S under IU by allowing the possibility of acquiring additional data. Two classical R&S formulations are extended to account for IU: (i) for fixed confidence, we consider when data arrive sequentially so that IU can be reduced over time; (ii) for fixed budget, a joint budget is assumed to be available for both collecting input data and running simulations. New procedures are proposed for each formulation using the frameworks of Sequential Elimination and Optimal Computing Budget Allocation, with theoretical guarantees provided accordingly (e.g., upper bound on the expected running time and finite-sample bound on the probability of false selection). Numerical results demonstrate the effectiveness of our procedures through a multi-stage production-inventory problem

Finite Simulation Budget Allocation for Ranking and Selection

We consider a simulation-based ranking and selection (R&S) problem under a fixed budget setting. Existing budget allocation procedures focus either on asymptotic optimality or on one-step-ahead allocation efficiency. Neither of them depends on the fixed simulation budget, the ignorance of which could lead to an inefficient allocation, especially when the simulation budget is finite. In light of this, we develop a finite-budget allocation rule that is adaptive to the simulation budget. Theoretical results show that the budget allocation strategies are distinctively different between a finite budget and a sufficiently large budget. Our proposed allocation rule can dynamically determine the ratio of budget allocated to designs according to different simulation budget and is optimal when the simulation budget goes to infinity, indicating it not only possesses desirable finite-budget properties but also achieves asymptotic optimality. Based on the proposed allocation rule, two efficient finite simulation budget allocation algorithms are developed. In the numerical experiments, we use both synthetic examples and a case study to show the superior efficiency of our proposed allocation rule

Selection of the Most Probable Best

We consider an expected-value ranking and selection problem where all k solutions' simulation outputs depend on a common uncertain input model. Given that the uncertainty of the input model is captured by a probability simplex on a finite support, we define the most probable best (MPB) to be the solution whose probability of being optimal is the largest. To devise an efficient sampling algorithm to find the MPB, we first derive a lower bound to the large deviation rate of the probability of falsely selecting the MPB, then formulate an optimal computing budget allocation (OCBA) problem to find the optimal static sampling ratios for all solution-input model pairs that maximize the lower bound. We devise a series of sequential algorithms that apply interpretable and computationally efficient sampling rules and prove their sampling ratios achieve the optimality conditions for the OCBA problem as the simulation budget increases. The algorithms are benchmarked against a state-of-the-art sequential sampling algorithm designed for contextual ranking and selection problems and demonstrated to have superior empirical performances at finding the MPB