Stochastic Localization Methods for Discrete Convex Simulation Optimization

We propose a set of new algorithms based on stochastic localization methods for large-scale discrete simulation optimization problems with convexity structure. All proposed algorithms, with the general idea of "localizing" potential good solutions to an adaptively shrinking subset, are guaranteed with high probability to identify a solution that is close enough to the optimal given any precision level. Specifically, for one-dimensional large-scale problems, we propose an enhanced adaptive algorithm with an expected simulation cost asymptotically independent of the problem scale, which is proved to attain the best achievable performance. For multi-dimensional large-scale problems, we propose statistically guaranteed stochastic cutting-plane algorithms, the simulation costs of which have no dependence on model parameters such as the Lipschitz parameter, as well as low polynomial order of dependence on the problem scale and dimension. Numerical experiments are implemented to support our theoretical findings. The theory results, joint the numerical experiments, provide insights and recommendations on which algorithm to use in different real application settings

Simultaneous Perturbation Methods for Adaptive Labor Staffing in Service Systems

Service systems are labor intensive due to the large variation in the tasks required to address service requests from multiple customers. Aligning the staffing levels to the forecasted workloads adaptively in such systems is nontrivial because of a large number of parameters and operational variations leading to a huge search space. A challenging problem here is to optimize the staffing while maintaining the system in steady-state and compliant to aggregate service level agreement (SLA) constraints. Further, because these parameters change on a weekly basis, the optimization should not take longer than a few hours. We formulate this problem as a constrained Markov cost process parameterized by the (discrete) staffing levels. We propose novel simultaneous perturbation stochastic approximation (SPSA) based SASOC (Staff Allocation using Stochastic Optimization with Constraints) algorithms for solving the above problem. The algorithms include both first order as well as second order methods and incorporate SPSA based gradient estimates in the primal, with dual ascent for the Lagrange multipliers. Both the algorithms that we propose are online, incremental and easy to implement. Further, they involve a certain generalized smooth projection operator, which is essential to project the continuous-valued worker parameter tuned by SASOC algorithms onto the discrete set. We validated our algorithms on five real-life service systems and compared them with a state-of-the-art optimization tool-kit OptQuest. Being 25 times faster than OptQuest, our algorithms are particularly suitable for adaptive labor staffing. Also, we observe that our algorithms guarantee convergence and find better solutions than OptQuest in many cases

Simulation optimization: A review of algorithms and applications

Simulation Optimization (SO) refers to the optimization of an objective function subject to constraints, both of which can be evaluated through a stochastic simulation. To address specific features of a particular simulation---discrete or continuous decisions, expensive or cheap simulations, single or multiple outputs, homogeneous or heterogeneous noise---various algorithms have been proposed in the literature. As one can imagine, there exist several competing algorithms for each of these classes of problems. This document emphasizes the difficulties in simulation optimization as compared to mathematical programming, makes reference to state-of-the-art algorithms in the field, examines and contrasts the different approaches used, reviews some of the diverse applications that have been tackled by these methods, and speculates on future directions in the field

Reinforcement Learning: Stochastic Approximation Algorithms for Markov Decision Processes

This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov decision processes

Adaptive Search Algorithms for Discrete Stochastic Optimization: A Smooth Best-Response Approach

This paper considers simulation-based optimization of the performance of a regime-switching stochastic system over a finite set of feasible configurations. Inspired by the stochastic fictitious play learning rules in game theory, we propose an adaptive simulation-based search algorithm that uses a smooth best-response sampling strategy and tracks the set of global optima, yet distributes the search so that most of the effort is spent on simulating the system performance at the global optima. The algorithm converges weakly to the set of global optima even when the observation data is correlated (as long as a weak law of large numbers holds). Numerical examples show that the proposed scheme yields a faster convergence for finite sample lengths compared with several existing random search and pure exploration methods in the literature.Comment: 13 pages, 1 table, 4 figures, submitted to IEEE Transactions on Automatic Contro

Equation-free optimal switching policies for bistable reacting systems using coarse time-steppers

We present a computer-assisted approach to locating approximate coarse optimal switching policies between stationary states of chemically reacting systems described by microscopic/stochastic evolution rules. The ``coarse time-stepper" constitutes a bridge between the underlying kinetic Monte Carlo simulation and traditional, continuum numerical optimization techniques formulated in discrete time. The approach is illustrated through two simple catalytic surface reaction models, implemented through kinetic Monte Carlo: NO reduction on Pt, and CO oxidation on Pt. The objective sought in both cases is to switch between two coexisting stable stationary states by minimal manipulation of a macroscopic system parameter.Comment: 31 pages, 6 tables, 11 figure

A comprehensive literature classification of simulation optimisation methods

Simulation Optimization (SO) provides a structured approach to the system design and configuration when analytical expressions for input/output relationships are unavailable. Several excellent surveys have been written on this topic. Each survey concentrates on only few classification criteria. This paper presents a literature survey with all classification criteria on techniques for SO according to the problem of characteristics such as shape of the response surface (global as compared to local optimization), objective functions (single or multiple objectives) and parameter spaces (discrete or continuous parameters). The survey focuses specifically on the SO problem that involves single per-formance measureSimulation Optimization, classification methods, literature survey

Discrete Convexity and Stochastic Approximation for Cross-layer On-off Transmission Control

This paper considers the discrete convexity of a cross-layer on-off transmission control problem in wireless communications. In this system, a scheduler decides whether or not to transmit in order to optimize the long-term quality of service (QoS) incurred by the queueing effects in the data link layer and the transmission power consumption in the physical (PHY) layer simultaneously. Using a Markov decision process (MDP) formulation, we show that the optimal policy can be determined by solving a minimization problem over a set of queue thresholds if the dynamic programming (DP) is submodular. We prove that this minimization problem is discrete convex. In order to search the minimizer, we consider two discrete stochastic approximation (DSA) algorithms: discrete simultaneous perturbation stochastic approximation (DSPSA) and L-natural-convex stochastic approximation (L-natural-convex SA). Through numerical studies, we show that the two DSA algorithms converge significantly faster than the existing continuous simultaneous perturbation stochastic approximation (CSPSA) algorithm in multi-user systems. Finally, we compare the convergence results and complexity of two DSA and CSPSA algorithms where we show that DSPSA achieves the best trade-off between complexity and accuracy in multi-user systems.Comment: 29 pages, 8 figures, submitted to IEEE Transactions on Wireless Communication

Approximate IPA: Trading Unbiasedness for Simplicity

When Perturbation Analysis (PA) yields unbiased sensitivity estimators for expected-value performance functions in discrete event dynamic systems, it can be used for performance optimization of those functions. However, when PA is known to be unbiased, the complexity of its estimators often does not scale with the system's size. The purpose of this paper is to suggest an alternative approach to optimization which balances precision with computing efforts by trading off complicated, unbiased PA estimators for simple, biased approximate estimators. Furthermore, we provide guidelines for developing such estimators, that are largely based on the Stochastic Flow Modeling framework. We suggest that if the relative error (or bias) is not too large, then optimization algorithms such as stochastic approximation converge to a (local) minimum just like in the case where no approximation is used. We apply this approach to an example of balancing loss with buffer-cost in a finite-buffer queue, and prove a crucial upper bound on the relative error. This paper presents the initial study of the proposed approach, and we believe that if the idea gains traction then it may lead to a significant expansion of the scope of PA in optimization of discrete event systems.Comment: 8 pages, 8 figure