Online stochastic optimization without distributions

This paper considers online stochastic scheduling problems where time constraints severely limit the number of optimizations which can be performed at decision time and/or in between decisions. Prior research has demonstrated that, whenever a distribution of the inputs is available for sampling, online stochatic algorithms may produce significant improvements in solution quality over oblivious approaches. However, the availability of an input distribution, although reasonable in many contexts, is too strong a requirement in a variety of applications. This paper broadens the applicability of online stochastic algorithms by relaxing this requirement and using machine learning techniques or historical data instead. In particular, it shows that machine learning techniques can be engineered to learn the distribution online, when its underlying structure is not available. Moreover, the paper presents the idea of historical sampling which provides a simple and effective way to leverage historical data in continuous and periodic online optimization. Experimental results on packet scheduling and vehicle routing indicate the potential of machine learning and historical sampling for online scheduling