Bayesian optimization allowing for common random numbers

We consider the problem of stochastic simulation optimization with common random numbers over a numerical search domain. We propose the Knowledge Gradient for Common Random Numbers (KG-CRN) sequential sampling algorithm, a simple elegant modification to the Knowledge Gradient that incorporates the use of correlated noise in simulation outputs with Gaussian Process meta-models. We compare this method against the standard Knowledge Gradient and a more recently proposed variation that allows for pairwise sampling. Our method significantly outperforms both baselines under identical laboratory conditions while greatly reducing computational cost compared to pairwise sampling

Bayesian optimisation vs. input uncertainty reduction

Simulators often require calibration inputs estimated from real world data and the estimate can significantly affect simulation output. Particularly when performing simulation optimisation to find an optimal solution, the uncertainty in the inputs significantly affects the quality of the found solution. One remedy is to search for the solution that has the best performance on average over the uncertain range of inputs yielding an optimal compromise solution. We consider the more general setting where a user may choose between either running simulations or instead querying an external data source, improving the input estimate and enabling the search for a more targeted, less compromised solution. We explicitly examine the trade-off between simulation and real data collection in order to find the optimal solution of the simulator with the true inputs. Using a value of information procedure, we propose a novel unified simulation optimisation procedure called Bayesian Information Collection and Optimisation (BICO) that, in each iteration, automatically determines which of the two actions (running simulations or data collection) is more beneficial. We theoretically prove convergence in the infinite budget limit and perform numerical experiments demonstrating that the proposed algorithm is able to automatically determine an appropriate balance between optimisation and data collection