Selecting the best stochastic systems for large scale engineering problems

Selecting a subset of the best solutions among large-scale problems is an important area of research. When the alternative solutions are stochastic in nature, then it puts more burden on the problem. The objective of this paper is to select a set that is likely to contain the actual best solutions with high probability. If the selected set contains all the best solutions, then the selection is denoted as correct selection. We are interested in maximizing the probability of this selection; P(CS). In many cases, the available computation budget for simulating the solution set in order to maximize P(CS) is limited. Therefore, instead of distributing these computational efforts equally likely among the alternatives, the optimal computing budget allocation (OCBA) procedure came to put more effort on the solutions that have more impact on the selected set. In this paper, we derive formulas of how to distribute the available budget asymptotically to find the approximation of P(CS). We then present a procedure that uses OCBA with the ordinal optimization (OO) in order to select the set of best solutions. The properties and performance of the proposed procedure are illustrated through a numerical example. Overall results indicate that the procedure is able to select a subset of the best systems with high probability of correct selection using small number of simulation samples under different parameter settings

Ranking and Selection under Input Uncertainty: Fixed Confidence and Fixed Budget

In stochastic simulation, input uncertainty (IU) is caused by the error in estimating the input distributions using finite real-world data. When it comes to simulation-based Ranking and Selection (R&S), ignoring IU could lead to the failure of many existing selection procedures. In this paper, we study R&S under IU by allowing the possibility of acquiring additional data. Two classical R&S formulations are extended to account for IU: (i) for fixed confidence, we consider when data arrive sequentially so that IU can be reduced over time; (ii) for fixed budget, a joint budget is assumed to be available for both collecting input data and running simulations. New procedures are proposed for each formulation using the frameworks of Sequential Elimination and Optimal Computing Budget Allocation, with theoretical guarantees provided accordingly (e.g., upper bound on the expected running time and finite-sample bound on the probability of false selection). Numerical results demonstrate the effectiveness of our procedures through a multi-stage production-inventory problem

Efficient Sampling Policy for Selecting a Good Enough Subset

The note studies the problem of selecting a good enough subset out of a finite number of alternatives under a fixed simulation budget. Our work aims to maximize the posterior probability of correctly selecting a good subset. We formulate the dynamic sampling decision as a stochastic control problem in a Bayesian setting. In an approximate dynamic programming paradigm, we propose a sequential sampling policy based on value function approximation. We analyze the asymptotic property of the proposed sampling policy. Numerical experiments demonstrate the efficiency of the proposed procedure

Finite Simulation Budget Allocation for Ranking and Selection

We consider a simulation-based ranking and selection (R&S) problem under a fixed budget setting. Existing budget allocation procedures focus either on asymptotic optimality or on one-step-ahead allocation efficiency. Neither of them depends on the fixed simulation budget, the ignorance of which could lead to an inefficient allocation, especially when the simulation budget is finite. In light of this, we develop a finite-budget allocation rule that is adaptive to the simulation budget. Theoretical results show that the budget allocation strategies are distinctively different between a finite budget and a sufficiently large budget. Our proposed allocation rule can dynamically determine the ratio of budget allocated to designs according to different simulation budget and is optimal when the simulation budget goes to infinity, indicating it not only possesses desirable finite-budget properties but also achieves asymptotic optimality. Based on the proposed allocation rule, two efficient finite simulation budget allocation algorithms are developed. In the numerical experiments, we use both synthetic examples and a case study to show the superior efficiency of our proposed allocation rule