The stochastic vehicle routing problem : a literature review, part II : solution methods

Building on the work of Gendreau et al. (Oper Res 44(3):469–477, 1996), and complementing the first part of this survey, we review the solution methods used for the past 20 years in the scientific literature on stochastic vehicle routing problems (SVRP). We describe the methods and indicate how they are used when dealing with stochastic vehicle routing problems. Keywords: vehicle routing (VRP), stochastic programmingm, SVRPpublishedVersio

Reinforcement learning based local search for grouping problems: A case study on graph coloring

Grouping problems aim to partition a set of items into multiple mutually disjoint subsets according to some specific criterion and constraints. Grouping problems cover a large class of important combinatorial optimization problems that are generally computationally difficult. In this paper, we propose a general solution approach for grouping problems, i.e., reinforcement learning based local search (RLS), which combines reinforcement learning techniques with descent-based local search. The viability of the proposed approach is verified on a well-known representative grouping problem (graph coloring) where a very simple descent-based coloring algorithm is applied. Experimental studies on popular DIMACS and COLOR02 benchmark graphs indicate that RLS achieves competitive performances compared to a number of well-known coloring algorithms

Primal-dual variable neighborhood search for the simple plant-location problem

Copyright @ 2007 INFORMSThe variable neighborhood search metaheuristic is applied to the primal simple plant-location problem and to a reduced dual obtained by exploiting the complementary slackness conditions. This leads to (i) heuristic resolution of (metric) instances with uniform fixed costs, up to n = 15,000 users, and m = n potential locations for facilities with an error not exceeding 0.04%; (ii) exact solution of such instances with up to m = n = 7,000; and (iii) exact solutions of instances with variable fixed costs and up to m = n = 15, 000.This work is supported by NSERC Grant 105574-02; NSERC Grant OGP205041; and partly by the Serbian Ministry of Science, Project 1583

Matheuristics: using mathematics for heuristic design

Matheuristics are heuristic algorithms based on mathematical tools such as the ones provided by mathematical programming, that are structurally general enough to be applied to different problems with little adaptations to their abstract structure. The result can be metaheuristic hybrids having components derived from the mathematical model of the problems of interest, but the mathematical techniques themselves can define general heuristic solution frameworks. In this paper, we focus our attention on mathematical programming and its contributions to developing effective heuristics. We briefly describe the mathematical tools available and then some matheuristic approaches, reporting some representative examples from the literature. We also take the opportunity to provide some ideas for possible future development

Branch-and-Price for Prescriptive Contagion Analytics

Predictive contagion models are ubiquitous in epidemiology, social sciences, engineering, and management. This paper formulates a prescriptive contagion analytics model where a decision-maker allocates shared resources across multiple segments of a population, each governed by continuous-time dynamics. We define four real-world problems under this umbrella: vaccine distribution, vaccination centers deployment, content promotion, and congestion mitigation. These problems feature a large-scale mixed-integer non-convex optimization structure with constraints governed by ordinary differential equations, combining the challenges of discrete optimization, non-linear optimization, and continuous-time system dynamics. This paper develops a branch-and-price methodology for prescriptive contagion analytics based on: (i) a set partitioning reformulation; (ii) a column generation decomposition; (iii) a state-clustering algorithm for discrete-decision continuous-state dynamic programming; and (iv) a tri-partite branching scheme to circumvent non-linearities. Extensive experiments show that the algorithm scales to very large and otherwise-intractable instances, outperforming state-of-the-art benchmarks. Our methodology provides practical benefits in contagion systems; in particular, it can increase the effectiveness of a vaccination campaign by an estimated 12-70%, resulting in 7,000 to 12,000 extra saved lives over a three-month horizon mirroring the COVID-19 pandemic. We provide an open-source implementation of the methodology in an online repository to enable replication

Multi-Modal Mean-Fields via Cardinality-Based Clamping

Mean Field inference is central to statistical physics. It has attracted much interest in the Computer Vision community to efficiently solve problems expressible in terms of large Conditional Random Fields. However, since it models the posterior probability distribution as a product of marginal probabilities, it may fail to properly account for important dependencies between variables. We therefore replace the fully factorized distribution of Mean Field by a weighted mixture of such distributions, that similarly minimizes the KL-Divergence to the true posterior. By introducing two new ideas, namely, conditioning on groups of variables instead of single ones and using a parameter of the conditional random field potentials, that we identify to the temperature in the sense of statistical physics to select such groups, we can perform this minimization efficiently. Our extension of the clamping method proposed in previous works allows us to both produce a more descriptive approximation of the true posterior and, inspired by the diverse MAP paradigms, fit a mixture of Mean Field approximations. We demonstrate that this positively impacts real-world algorithms that initially relied on mean fields.Comment: Submitted for review to CVPR 201

New variants of variable neighbourhood search for 0-1 mixed integer programming and clustering

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Many real-world optimisation problems are discrete in nature. Although recent rapid developments in computer technologies are steadily increasing the speed of computations, the size of an instance of a hard discrete optimisation problem solvable in prescribed time does not increase linearly with the computer speed. This calls for the development of new solution methodologies for solving larger instances in shorter time. Furthermore, large instances of discrete optimisation problems are normally impossible to solve to optimality within a reasonable computational time/space and can only be tackled with a heuristic approach. In this thesis the development of so called matheuristics, the heuristics which are based on the mathematical formulation of the problem, is studied and employed within the variable neighbourhood search framework. Some new variants of the variable neighbourhood searchmetaheuristic itself are suggested, which naturally emerge from exploiting the information from the mathematical programming formulation of the problem. However, those variants may also be applied to problems described by the combinatorial formulation. A unifying perspective on modern advances in local search-based metaheuristics, a so called hyper-reactive approach, is also proposed. Two NP-hard discrete optimisation problems are considered: 0-1 mixed integer programming and clustering with application to colour image quantisation. Several new heuristics for 0-1 mixed integer programming problem are developed, based on the principle of variable neighbourhood search. One set of proposed heuristics consists of improvement heuristics, which attempt to find high-quality near-optimal solutions starting from a given feasible solution. Another set consists of constructive heuristics, which attempt to find initial feasible solutions for 0-1 mixed integer programs. Finally, some variable neighbourhood search based clustering techniques are applied for solving the colour image quantisation problem. All new methods presented are compared to other algorithms recommended in literature and a comprehensive performance analysis is provided. Computational results show that the methods proposed either outperform the existing state-of-the-art methods for the problems observed, or provide comparable results. The theory and algorithms presented in this thesis indicate that hybridisation of the CPLEX MIP solver and the VNS metaheuristic can be very effective for solving large instances of the 0-1 mixed integer programming problem. More generally, the results presented in this thesis suggest that hybridisation of exact (commercial) integer programming solvers and some metaheuristic methods is of high interest and such combinations deserve further practical and theoretical investigation. Results also show that VNS can be successfully applied to solving a colour image quantisation problem.Support from the Mathematical Institute, Serbian Academy of Sciences and Arts, are acknowledged for this research

Algorithme du simplexe en nombres entiers avec décomposition

RÉSUMÉ : L’objectif général de cette thèse est de développer un algorithme efficace pour la résolution du problème de partitionnement d’ensemble (SPP : Set Partitioning Problem). Le SPP est un problème très connu de la programmation linéaire en nombres entiers. Il consiste à partitionner un ensemble de tâches (ex : vols d’avion, segments de trajet d’autobus, ...) en sous-ensembles (routes de véhicules ou suites de tâches effectuées par une personne) de sorte que les sous-ensembles sélectionnés aient un coût total minimum tout en couvrant chaque tâche exactement une et une seule fois. Le SPP est en général résolu par la méthode branch-and-price. Cette méthode peut être excessivement lente dans le cas de problèmes difficiles de très grande taille. Le paradigme derrière la méthode est “dual” ou “tout ou rien” dans le sens où une solution entière du problème est en général obtenue très tard ou à la fin de la résolution pour les problèmes pratiques. Avoir une solution entière rapidement est très apprécié en pratique. En plus, il est très fréquent, en pratique, de vouloir optimiser un problème pour lequel on connaît déjà une solution avec une bonne information primale que l’on veut, au moins, améliorer. La méthode branch-and-price n’est pas adaptée pour tirer avantage d’une telle situation. Une approche “primale” est mieux appropriée pour la résolution du SPP (ex : planification d’horaires de chauffeurs d’autobus). L’approche, en question, s’appelle l’algorithme du simplexe en nombres entiers et consiste à commencer d’une solution initiale connue et effectuer une série de petites améliorations de façon à produire une suite de solutions présentant des coûts décroissants et convergeant vers une solution optimale. Plusieurs auteurs ont proposé par le passé des algorithmes pour résoudre le SPP d’une façon primale. Malheureusement, aucun de ces algorithmes n’est assez efficace pour être utilisé en pratique. Le principal facteur derrière cela est la nature fortement dégénérée du SPP. Pour chaque solution, il y a un très grand nombre de bases permettant d’identifier des mouvements vers des solutions voisines. Le phénomène de la dégénérescence implique qu’il est difficile, et même combinatoire, de passer d’une solution entière à une autre ; mais ces algorithmes ne proposent pas de techniques efficaces pour pallier ce phénomène. Donc, plus précisément, l’objectif de cette thèse est de proposer une implémentation de l’algorithme du simplexe en nombres entiers pratique et efficace contre la dégénérescence. C’est-à-dire que l’implémentation recherchée doit être capable de résoudre des SPPs de grande taille ; et elle doit aussi être en mesure d’exploiter une solution initiale donnée et produire, itérativement et dans des temps raisonnablement courts, des solutions améliorées. Pour ce faire, nous commençons, dans un premier travail, par l’exploitation des idées d’un algorithme appelé simplexe primal amélioré (IPS : Improved Primal Simplex). Cet algorithme cerne efficacement le phénomène de la dégénérescence lors de la résolution d’un programme linéaire quelconque. Ainsi, nous proposons un algorithme inspiré par IPS et adapté au contexte du SPP (nombres entiers). L’algorithme, baptisé simplexe en nombres entiers avec décomposition, commence à partir d’une solution initiale avec une bonne information primale. Comme dans IPS, il améliore itérativement la solution courante en décomposant le problème original en deux sous-problèmes : un premier sous-problème, appelé problème réduit, qui est un petit SPP, permet d’améliorer la solution en ne considérant que les colonnes dites compatibles avec la solution courante. Un deuxième sous-problème, appelé problème complémentaire, ne considérant que les colonnes incompatibles avec la solution courante, permet de trouver une direction de descente combinant plusieurs variables qui garantit d’avoir une meilleure solution, mais pas nécessairement entière. Le domaine réalisable du problème complémentaire, relaxé de toute contrainte d’intégralité, représente un cône des directions réalisables. Une contrainte supplémentaire, appelée contrainte de normalisation, lui est ajoutée pour assurer qu’il soit borné. Les directions qu’il trouve ont la caractéristique d’être minimales dans le sens où elles ne contiennent aucune sous-direction réalisable. Cette caractéristique, accompagnée d’une technique de pricing partiel (partial pricing) appelée multi-phase, fait que, dans la majorité des itérations, le problème complémentaire trouve directement des directions qui mènent vers des solutions entières. Dans le restant des cas, où les directions trouvées mènent vers des solutions fractionnaires, un branchement en profondeur permet souvent d’aboutir rapidement à une solution entière. Nous avons testé ce nouvel algorithme sur des instances d’horaires de chauffeurs d’autobus ayant 1600 contraintes et 570000 variables. L’algorithme atteint la solution optimale, ou une solution assez proche, pour la majorité de ces instances ; et ceci dans un temps qui représente une fraction de ce qu’aurait demandé un solveur commercial tel que CPLEX; sachant que ce dernier n’arrive même pas à trouver une première solution réalisable après une durée de plus de 10 heures d’exécution sur certaines instances. L’algorithme, dans sa première version, représente à notre avis une première implémentation de l’algorithme du simplexe en nombres entiers à être capable de résoudre des instances de SPP de grande taille dans des temps acceptables en pratique. Toutefois, il souffre encore de quelques limitations telles que la nécessité de développer un branchement complexe pour pouvoir améliorer la qualité des solutions trouvées. Cela est dû au fait que le problème complémentaire présente une structure difficilement exploitable par CPLEX. Une autre limitation de cette implémentation est qu’elle ne permet pas de supporter les contraintes supplémentaires qui ne sont pas de type partitionnement. Dans un deuxième travail, nous améliorons notre algorithme en généralisant certains aspects de son concept. Notre objectif dans cette étape est d’éviter d’implémenter un branchement complexe et exhaustif tout en permettant à notre algorithme de pouvoir considérer des contraintes supplémentaires. Nous revoyons donc la façon avec laquelle l’algorithme décompose le problème et nous proposons une méthode de décomposition dynamique où l’intégralité de la solution est contrôlée au niveau du problème réduit au lieu du problème complémentaire. Ainsi, le problème complémentaire n’est plus responsable de trouver une direction menant à une solution entière mais plutôt une direction de descente quelconque ; et c’est le problème réduit qui s’occupe de chercher une solution entière autour de cette direction de descente en déléguant le branchement au solveur commercial. Avec cette décomposition dynamique, l’algorithme atteint une solution optimale, ou presque optimale, pour toutes les instances, tout en maintenant le même ordre de grandeur des temps d’exécution de la version précédente. Dans un troisième travail, nous nous donnons l’objectif d’améliorer la performance de l’algorithme. Nous visons de rendre les temps d’exécution de l’algorithme plus rapides sans perdre tous les avantages introduits par le deuxième travail. Nous constatons, alors, que la minimalité des directions de descente exigée par le problème complémentaire est un facteur qui favorise l’intégralité des solutions subséquentes, mais représente, aussi, un élément de ralentissement puisqu’il force l’algorithme à faire plusieurs petits pas, vers des solutions adjacentes uniquement, en direction de sa solution finale. Nous changeons, alors, le modèle du problème complémentaire pour lui permettre de trouver des directions de descente non minimales. Le nouveau modèle arrive, ainsi, à aller directement vers des solutions entières non adjacentes présentant des améliorations considérables dans le coût ; et ceci en un nombre d’itérations très réduit qui ne dépasse pas deux itérations pour les instances de grande taille dans nos tests. Une solution optimale est toujours atteinte et le temps global d’exécution est réduit par au moins un facteur de cinq sur toutes les instances. Ce facteur est de l’ordre de dix pour les instances de grande taille. Avec ces trois travaux, nous pensons avoir proposé un algorithme du simplexe en nombres entiers efficace qui produit des solutions de qualité en des temps courts.----------ABSTRACT : The general objective of this thesis is to develop an efficient algorithm for solving the Set Partitioning Problem (SPP). SPP is a well known problem of integer programming. Its goal is to partition a set of tasks (e.g. plane flights, bus trip segments, ...) into subsets (vehicle routes or set of tasks performed by a person) such that the selected subsets have a minimum total cost while covering each task exactly once. SPP is usually solved by the method of branch-and-price. This method can be excessively slow when solving difficult problems of large size. The paradigm behind the method is “dual” or “all or nothing” in the sense that an integer solution of the problem is generally obtained very late or at the end of the solution process for large instances. In practice, having an integer solution quickly is very appreciated. Also, it is very common in practice to solve a problem for which a solution having good primal information is already known. We want to, at least, improve that solution. The branch-and-price method is not suitable to take advantage of such a situation. A “primal” approach fits better for the solution of the SPP (e.g. bus driver scheduling). The approach is called the Integral Simplex algorithm. It consists of starting from a known initial solution and performing a series of small improvements so as to produce a series of solutions with decreasing costs and converging towards an optimal solution. Several authors have, in the past, proposed algorithms for solving the SPP in using a primal paradigm. Unfortunately, none of these algorithms is effective enough to be used in practice. The main factor behind this is the highly degenerate nature of the SPP. For each solution, there is a very large number of bases that permit to identify transitions to neighbor solutions. The degeneracy implies that it is difficult, and even combinatorial, to go from an integer solution to another; but these algorithms do not offer effective techniques to overcome degeneracy. So, specifically, the aim of this thesis is to introduce an implementation of the Integral Simplex that is effective against degeneracy in practice. This means that the intended implementation must be able to solve SPPs of large size; and it must also be able to benefit from a given initial solution and produce, iteratively and in reasonably short time, improved solutions. To do this, we first use ideas from an algorithm called Improved Primal Simplex (IPS) algorithm. This algorithm helps the primal simplex algorithm in effectively coping with degeneracy when solving linear programs. Thus, we propose an algorithm inspired by IPS and adapted to the context of the SPP. The algorithm, called Integral Simplex Using Decomposition, starts from an initial solution with good primal information. As in IPS, it iteratively improves the current solution by decomposing the original problem into two sub-problems: a first sub-problem, called reduced problem, which is a small completely non-degenerate SPP that improves the solution by considering only the columns that are said to be compatible with the current solution. A second sub-problem, called complementary problem, considers only the columns that are incompatible with the current solution. The complementary problem finds a descent direction, combining several variables, that guarantees to have a better solution; but not necessarily integer. The feasible domain of the complementary problem, where all the integrality constraints are relaxed, is a cone of feasible directions. An additional constraint, called normalization constraint, is added to ensure that the problem is bounded. The directions found are minimal in the sense that they do not contain any feasible sub-direction. This minimality feature, combined with a partial pricing technique called multi-phase, helps the complementary problem in finding directions that directly lead to integer solutions in the majority of iterations. In the remaining cases, where the directions lead to fractional solutions, a quick deep branching often lead to an integer solution. We tested the new algorithm on bus driver scheduling problems having 1600 rows and 570000 columns. The algorithm reaches an optimal, or near optimal, solution for the majority of these problems; solution times represent a fraction of what would have taken a commercial solver such as CPLEX. The latter does not even find a first feasible solution within a 10 hour runtime period for some of those problems. We think that the algorithm, under its first version, is a first implementation of the integral simplex method that was able to solve large SPP problems within acceptable times in practice. However, it still has some limitations such as the need to develop a complex branching to improve the quality of the solutions found. This is due to the fact that the complementary problem presents a structure that is not suitable to handle. Another limitation of this implementation is the fact that it does not consider supplementary non partitioning constraints. In a second paper, we improve our algorithm generalizing certain aspects of its concept. Our goal in this step is to avoid implementing a complex and exhaustive branching while allowing our algorithm to consider supplementary constraints. We review, therefore, the way in which the algorithm decomposes the problem and propose a method of dynamic decomposition where the integrality of the solution is controlled within the reduced problem instead of the complementary problem. Thus, the complementary problem is no longer responsible for finding a direction leading to an integer solution but only a descent direction; and the reduced problem handles the integrality of the solution, while searching around this descent direction, by delegating the branching to the commercial solver. With this dynamic decomposition, the algorithm reaches an optimal or near optimal solution for all instances; while maintaining execution times comparable to the ones from the previous version. In a third paper, we target the objective of improving the performance of the algorithm. We aim to make the algorithm run faster without losing the benefits introduced by the second paper. We observe, then, that the minimality of descent directions, required by the complementary problem, forces the algorithm to make small steps towards adjacent solutions. We then change the model of the complementary problem to let it find non-minimal descent directions. The new model is, thus, able to go directly to non-adjacent integer solutions with significant improvements in the cost, in a very limited number of iterations that does not exceed two iterations for large problems in our tests. An optimal solution is always reached and the execution time is reduced by at least a factor of five on all instances. This factor is about ten for large instances. With these three papers, we believe we have introduced an effective integral simplex algorithm that produces quality solutions in short times