The behavior of simulated annealing in stochastic optimization

In this thesis we examine the performance of simulated annealing (SA) on various response surfaces. The main goals of the study are to evaluate the effectiveness of SA for stochastic optimization, develop modifications to SA in an attempt to improve its performance, and to evaluate whether artificially adding noise to a deterministic response surface might improve the performance of SA. SA is applied to several different response surfaces with different levels of complexity. We first experiment with two basic approaches of computing the performance measure for stochastic surfaces, constant sample size and variable sample size. We found that the constant sample size performed best. At the same time we also show that artificially adding noise may improve the performance of SA on more complex deterministic response surfaces. We develop a hybrid version of SA in which the genetic algorithm is embedded within SA. The effectiveness of the hybrid approach is not conclusive and needs further investigation. Finally, we conclude with a brief discussion on the strengths and weaknesses of the proposed method and an outline of future directions

Estimation of Noisy Cost Functions by Conventional and Adjusted Simulated Annealing Techniques

L'algorithme de recuit simulé est largement utilisé dans la communauté d'optimisation pour résoudre divers types de problèmes, discrets et continus. L'objectif de cette thèse est d'analyser le recuit simulé dans des environnements déterministes et stochastiques pour des problèmes discrets. Les objectifs précis sont de classer des problèmes clés, d'offrir des suggestions et des recommandations à suivre en utilisant l'algorithme de recuit simulé et de recuit simulé sous bruit. Plus spécifiquement, des problèmes apparaissent en optimisation en présence de bruit, et sur la manière de le contrôler. Nous proposons la méthode de recuit simulé bruité (NSA: Noisy Simulated Annealing), basée sur la modification de l'algorithme de Metropolis-Hastings présentée par Ceperlay and Dewing, qui surpasse les techniques de recuit simulé analogues, délivrant des solutions numériques similaires, à coût réduit. Nous considérons les principales approches qui traitent le bruit dans le cadre du recuit simulé afin d'en extraire leurs attributs distinctifs et de produire une comparaison plus pertinente. Nous évaluons ensuite les performances numériques de l'approche sur des instances du problème du voyageur de commerce. Les résultats obtenus montrent un clair avantage pour le recuit simulé bruité, en présence de bruit.The Simulated Annealing (SA) algorithm is extensively used in the optimization community for solving various kinds of problems, discrete and continuous. This thesis aims to analyze SA in both deterministic and stochastic environments for discrete problems. Precise objectives are to classify key problems, offer suggestions and recommendations to be undertaken by using SA and Simulated Annealing Under Noise (SAUN). More specifically, problems appear in optimization due to the existence of noise when evaluating the objective function, and how to control this noise. We propose a method, called Noisy Simulated Annealing (NSA), based on the Metropolis-Hasting algorithm modification presented by Ceperlay and Dewing, that outperforms analogous SA techniques, delivering similar numerical solutions, at a reduced cost. We consider the main approaches in the SA setting that handle noise in order to extract their distinctive attributes and make the comparison more relevant. We next assess the numerical performance of the approach on traveling salesman problem instances. The outcomes of our tests show a clear advantage for NSA when solving different problems to get high-quality solutions in presence of noise

How to calculate the barycenter of a weighted graph

Discrete structures like graphs make it possible to naturally and exibly model complex phenomena. Since graphs that represent various types of information are increasingly available today, their analysis has become a popular subject of research. The graphs studied in the field of data science at this time generally have a large number of nodes that are not fairly weighted and connected to each other, translating a structural specification of the data. Yet, even an algorithm for locating the average position in graphs is lacking although this knowledge would be of primary interest for statistical or representation problems. In this work, we develop a stochastic algorithm for finding the Fréchet mean of weighted undirected metric graphs. This method relies on a noisy simulated annealing algorithm dealt with using homogenization. We then illustrate our algorithm with two examples (subgraphs of a social network and of a collaboration and citation network)

How to calculate the barycenter of a weighted graph

Discrete structures like graphs make it possible to naturally and exibly model complex phenomena. Since graphs that represent various types of information are increasingly available today, their analysis has become a popular subject of research. The graphs studied in the field of data science at this time generally have a large number of nodes that are not fairly weighted and connected to each other, translating a structural specification of the data. Yet, even an algorithm for locating the average position in graphs is lacking although this knowledge would be of primary interest for statistical or representation problems. In this work, we develop a stochastic algorithm for finding the Fréchet mean of weighted undirected metric graphs. This method relies on a noisy simulated annealing algorithm dealt with using homogenization. We then illustrate our algorithm with two examples (subgraphs of a social network and of a collaboration and citation network)

Solving the vehicle routing problem with stochastic demands using the cross-entropy method.

Abstract An alternate formulation of the classical vehicle routing problem with stochastic demands (VRPSD) is considered. We propose a new heuristic method to solve the problem. The algorithm is a modified version of the so-called Cross-Entropy method, which has been proposed in the literature as a heuristic for deterministic combinatorial optimization problems based upon concepts of rare-event simulation. In our version of the method, the objective function is computed using Monte-Carlo simulations at each point in the domain and the modified CrossEntropy heuristic is applied. A framework is also developed for obtaining exact solutions and tight lower bounds for the problem under various conditions, which include specific families of demand distributions. This is used to assess the heuristic's performance. Finally, numerical results are presented for various problem instances to illustrate the ideas

A survey on metaheuristics for stochastic combinatorial optimization

Metaheuristics are general algorithmic frameworks, often nature-inspired, designed to solve complex optimization problems, and they are a growing research area since a few decades. In recent years, metaheuristics are emerging as successful alternatives to more classical approaches also for solving optimization problems that include in their mathematical formulation uncertain, stochastic, and dynamic information. In this paper metaheuristics such as Ant Colony Optimization, Evolutionary Computation, Simulated Annealing, Tabu Search and others are introduced, and their applications to the class of Stochastic Combinatorial Optimization Problems (SCOPs) is thoroughly reviewed. Issues common to all metaheuristics, open problems, and possible directions of research are proposed and discussed. In this survey, the reader familiar to metaheuristics finds also pointers to classical algorithmic approaches to optimization under uncertainty, and useful informations to start working on this problem domain, while the reader new to metaheuristics should find a good tutorial in those metaheuristics that are currently being applied to optimization under uncertainty, and motivations for interest in this fiel

Optimizing Order Consolidation with Simulation Optimization

In this work we study the order consolidation process of an e-retailing warehouse using real data. Order consolidation is a process that follows the picking operation in a warehouse. After stock keeping units (SKUs) of an order are picked into possibly multiple totes, they are sorted and the SKUs are consolidated back into orders to be packed and shipped to customers. There are two main operations in order consolidation: wave sortation and putting. Wave sortation involves tote sequencing which is a major driver of how smooth the system runs. First, totes should be sequenced so that the work load is balanced. Second, totes that carry SKUs of the same order should be close in sequence in order to reduce order processing times, and to reduce resource utilization. At the same time, orders should be assigned to operators for putting in order to balance operator workload and to avoid recirculation. The assignment and sequencing of totes, when assuming known and constant tote induction times and when the goal is total completion time, is a problem similar to parallel machine scheduling. However, the goal from order consolidation is minimizing order, as opposed to tote, consolidation times. Order consolidation also depends on the putting operation. Moreover, processing times are uncertain and may vary because of different sizes and weights of SKUs and because of variations in manual and automatic processes. In order to reflect a realistic system performance accurately, we need to take randomness into account when evaluating measures like order completion time, resource utilization, and congestion. In this thesis, we first build a simulation model for the consolidation process using a given tote assignment and sequencing. We use the empirical distributions derived from the data to run the simulation. Second, we develop a simulated annealing heuristic and apply it to solve the deterministic tote sequencing problem with the goal of minimizing order completion time or order processing time. Third, we develop a stochastic simulated annealing algorithm to optimize the whole consolidation process. The algorithm decides on tote assignment and sequencing and evaluates the sequences using the simulation model. We experimented with optimizing order completion times and SKU wait times for putting. Our approach is then a simulation optimization to optimize the consolidation process using different metrics. We report on numerical tests using real instances