32 research outputs found
Adaptive Design in Discrete Stochastic Optimization
We present adaptive assignment rules for the design of the necessary simulations when solving discrete stochastic optimization problems. The rules are constructed in such a way, that the expected size of confidence sets for the optimizer is as small as possible
Selecting the best stochastic systems for large scale engineering problems
Selecting a subset of the best solutions among large-scale problems is an important area of research. When the alternative solutions are stochastic in nature, then it puts more burden on the problem. The objective of this paper is to select a set that is likely to contain the actual best solutions with high probability. If the selected set contains all the best solutions, then the selection is denoted as correct selection. We are interested in maximizing the probability of this selection; P(CS). In many cases, the available computation budget for simulating the solution set in order to maximize P(CS) is limited. Therefore, instead of distributing these computational efforts equally likely among the alternatives, the optimal computing budget allocation (OCBA) procedure came to put more effort on the solutions that have more impact on the selected set. In this paper, we derive formulas of how to distribute the available budget asymptotically to find the approximation of P(CS). We then present a procedure that uses OCBA with the ordinal optimization (OO) in order to select the set of best solutions. The properties and performance of the proposed procedure are illustrated through a numerical example. Overall results indicate that the procedure is able to select a subset of the best systems with high probability of correct selection using small number of simulation samples under different parameter settings
Optimal Simple Regret in Bayesian Best Arm Identification
We consider Bayesian best arm identification in the multi-armed bandit
problem. Assuming certain continuity conditions of the prior, we characterize
the rate of the Bayesian simple regret. Differing from Bayesian regret
minimization (Lai, 1987), the leading factor in Bayesian simple regret derives
from the region where the gap between optimal and sub-optimal arms is smaller
than . We propose a simple and easy-to-compute
algorithm with its leading factor matches with the lower bound up to a constant
factor; simulation results support our theoretical findings
Convergence of Sample Path Optimal Policies for Stochastic Dynamic Programming
We consider the solution of stochastic dynamic programs using sample path estimates. Applying the theory of large deviations, we derive probability error bounds associated with the convergence of the estimated optimal policy to the true optimal policy, for finite horizon problems. These bounds decay at an exponential rate, in contrast with the usual canonical (inverse) square root rate associated with estimation of the value (cost-to-go) function itself. These results have practical implications for Monte Carlo simulation-based solution approaches to stochastic dynamic programming problems where it is impractical to extract the explicit transition probabilities of the underlying system model
Convergence Rate Analysis for Optimal Computing Budget Allocation Algorithms
Ordinal optimization (OO) is a widely-studied technique for optimizing
discrete-event dynamic systems (DEDS). It evaluates the performance of the
system designs in a finite set by sampling and aims to correctly make ordinal
comparison of the designs. A well-known method in OO is the optimal computing
budget allocation (OCBA). It builds the optimality conditions for the number of
samples allocated to each design, and the sample allocation that satisfies the
optimality conditions is shown to asymptotically maximize the probability of
correct selection for the best design. In this paper, we investigate two
popular OCBA algorithms. With known variances for samples of each design, we
characterize their convergence rates with respect to different performance
measures. We first demonstrate that the two OCBA algorithms achieve the optimal
convergence rate under measures of probability of correct selection and
expected opportunity cost. It fills the void of convergence analysis for OCBA
algorithms. Next, we extend our analysis to the measure of cumulative regret, a
main measure studied in the field of machine learning. We show that with minor
modification, the two OCBA algorithms can reach the optimal convergence rate
under cumulative regret. It indicates the potential of broader use of
algorithms designed based on the OCBA optimality conditions
Adaptive Data Depth via Multi-Armed Bandits
Data depth, introduced by Tukey (1975), is an important tool in data science,
robust statistics, and computational geometry. One chief barrier to its broader
practical utility is that many common measures of depth are computationally
intensive, requiring on the order of operations to exactly compute the
depth of a single point within a data set of points in -dimensional
space. Often however, we are not directly interested in the absolute depths of
the points, but rather in their relative ordering. For example, we may want to
find the most central point in a data set (a generalized median), or to
identify and remove all outliers (points on the fringe of the data set with low
depth). With this observation, we develop a novel and instance-adaptive
algorithm for adaptive data depth computation by reducing the problem of
exactly computing depths to an -armed stochastic multi-armed bandit
problem which we can efficiently solve. We focus our exposition on simplicial
depth, developed by Liu (1990), which has emerged as a promising notion of
depth due to its interpretability and asymptotic properties. We provide general
instance-dependent theoretical guarantees for our proposed algorithms, which
readily extend to many other common measures of data depth including majority
depth, Oja depth, and likelihood depth. When specialized to the case where the
gaps in the data follow a power law distribution with parameter , we
show that we can reduce the complexity of identifying the deepest point in the
data set (the simplicial median) from to
, where suppresses logarithmic
factors. We corroborate our theoretical results with numerical experiments on
synthetic data, showing the practical utility of our proposed methods.Comment: Keywords: multi-armed bandits, data depth, adaptivity, large-scale
computation, simplicial dept