Combinatorial optimization and metaheuristics

Today, combinatorial optimization is one of the youngest and most active areas of discrete mathematics. It is a branch of optimization in applied mathematics and computer science, related to operational research, algorithm theory and computational complexity theory. It sits at the intersection of several fields, including artificial intelligence, mathematics and software engineering. Its increasing interest arises for the fact that a large number of scientific and industrial problems can be formulated as abstract combinatorial optimization problems, through graphs and/or (integer) linear programs. Some of these problems have polynomial-time (“efficient”) algorithms, while most of them are NP-hard, i.e. it is not proved that they can be solved in polynomial-time. Mainly, it means that it is not possible to guarantee that an exact solution to the problem can be found and one has to settle for an approximate solution with known performance guarantees. Indeed, the goal of approximate methods is to find “quickly” (reasonable run-times), with “high” probability, provable “good” solutions (low error from the real optimal solution). In the last 20 years, a new kind of algorithm commonly called metaheuristics have emerged in this class, which basically try to combine heuristics in high level frameworks aimed at efficiently and effectively exploring the search space. This report briefly outlines the components, concepts, advantages and disadvantages of different metaheuristic approaches from a conceptual point of view, in order to analyze their similarities and differences. The two very significant forces of intensification and diversification, that mainly determine the behavior of a metaheuristic, will be pointed out. The report concludes by exploring the importance of hybridization and integration methods

Analyse et conception de recherches locales génériques pour l'optimisation combinatoire à un ou plusieurs objectifs

Lorsque l’on cherche à résoudre des problèmes d’optimisation combinatoire difficiles, trouver une solution optimale par les méthodes complètes peut s’avérer impraticable. Dans un tel contexte, on peut déterminer des solutions approchées grâce à l’utilisation d’heuristiques. Parmi elles, les métaheuristiques sont une forme générique d’algorithmes approchés facilement applicables à une large gamme de problèmes.Nos travaux de recherche sur les métaheuristiques cherchent à s’abstraire au maximum des spécificités des problèmes d’optimisation en les modélisant notamment sous forme de paysages de recherche à explorer. Cette abstraction, proposée en 1932 en biologie pour modéliser la relation entre génotype des individus et chances de reproduction, a été reprise plus récemment en optimisation combinatoire afin de mettre en relation la qualité des solutions avec les valeurs prises par les variables de décision. Dans ce contexte, nous avons étudié principalement la résolution de problèmes d’optimisation par les algorithmes de recherche locale.En optimisation mono-objectif, l’étude des climbers classiques de la littérature combinée avec l’analyse de la structure des paysages de recherche nous a permis d’obtenir des résultats parfois à contre-courant de ce qui est fait usuellement dans la communauté. Grâce à ces observations, nous nous sommes proposé d’étudier des stratégies originales pour ces algorithmes de recherche. Les résultats obtenus permettent d’entrevoir des perspectives de recherche importantes dans ce domaine.En optimisation multiobjectif, nous avons proposé des algorithmes de type recherche locale basés sur la notion d’indicateur de qualité. Ces algorithmes, en plus d’être génériques, se sont montrés efficaces sur divers types de problèmes tout en étant peu sensibles au paramétrage. L’utilisation d’indicateurs de qualité permet de surcroît de reformuler les problèmes d’optimisation multiobjectif sous forme de problèmes d’optimisation mono-objectif sur des ensembles et ainsi ouvrir diverses perspectives de recherche

Metaheuristics for NP-hard combinatorial optimization problems

Ph.DDOCTOR OF PHILOSOPH

Design of Heuristic Algorithms for Hard Optimization

This open access book demonstrates all the steps required to design heuristic algorithms for difficult optimization. The classic problem of the travelling salesman is used as a common thread to illustrate all the techniques discussed. This problem is ideal for introducing readers to the subject because it is very intuitive and its solutions can be graphically represented. The book features a wealth of illustrations that allow the concepts to be understood at a glance. The book approaches the main metaheuristics from a new angle, deconstructing them into a few key concepts presented in separate chapters: construction, improvement, decomposition, randomization and learning methods. Each metaheuristic can then be presented in simplified form as a combination of these concepts. This approach avoids giving the impression that metaheuristics is a non-formal discipline, a kind of cloud sculpture. Moreover, it provides concrete applications of the travelling salesman problem, which illustrate in just a few lines of code how to design a new heuristic and remove all ambiguities left by a general framework. Two chapters reviewing the basics of combinatorial optimization and complexity theory make the book self-contained. As such, even readers with a very limited background in the field will be able to follow all the content

New variants of variable neighbourhood search for 0-1 mixed integer programming and clustering

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Many real-world optimisation problems are discrete in nature. Although recent rapid developments in computer technologies are steadily increasing the speed of computations, the size of an instance of a hard discrete optimisation problem solvable in prescribed time does not increase linearly with the computer speed. This calls for the development of new solution methodologies for solving larger instances in shorter time. Furthermore, large instances of discrete optimisation problems are normally impossible to solve to optimality within a reasonable computational time/space and can only be tackled with a heuristic approach. In this thesis the development of so called matheuristics, the heuristics which are based on the mathematical formulation of the problem, is studied and employed within the variable neighbourhood search framework. Some new variants of the variable neighbourhood searchmetaheuristic itself are suggested, which naturally emerge from exploiting the information from the mathematical programming formulation of the problem. However, those variants may also be applied to problems described by the combinatorial formulation. A unifying perspective on modern advances in local search-based metaheuristics, a so called hyper-reactive approach, is also proposed. Two NP-hard discrete optimisation problems are considered: 0-1 mixed integer programming and clustering with application to colour image quantisation. Several new heuristics for 0-1 mixed integer programming problem are developed, based on the principle of variable neighbourhood search. One set of proposed heuristics consists of improvement heuristics, which attempt to find high-quality near-optimal solutions starting from a given feasible solution. Another set consists of constructive heuristics, which attempt to find initial feasible solutions for 0-1 mixed integer programs. Finally, some variable neighbourhood search based clustering techniques are applied for solving the colour image quantisation problem. All new methods presented are compared to other algorithms recommended in literature and a comprehensive performance analysis is provided. Computational results show that the methods proposed either outperform the existing state-of-the-art methods for the problems observed, or provide comparable results. The theory and algorithms presented in this thesis indicate that hybridisation of the CPLEX MIP solver and the VNS metaheuristic can be very effective for solving large instances of the 0-1 mixed integer programming problem. More generally, the results presented in this thesis suggest that hybridisation of exact (commercial) integer programming solvers and some metaheuristic methods is of high interest and such combinations deserve further practical and theoretical investigation. Results also show that VNS can be successfully applied to solving a colour image quantisation problem.Support from the Mathematical Institute, Serbian Academy of Sciences and Arts, are acknowledged for this research

Mixed-integer linear programming based approaches for the resource constrained project scheduling problem.

Programa de P?s-Gradua??o em Ci?ncia da Computa??o. Departamento de Ci?ncia da Computa??o, Instituto de Ci?ncias Exatas e Biol?gicas, Universidade Federal de Ouro Preto.Resource Constrained Project Scheduling Problems (RCPSPs) without preemption are well-known NP-hard combinatorial optimization problems. A feasible RCPSP solution consists of a time-ordered schedule of jobs with corresponding execution modes, respecting precedence and resources constraints. First, in this thesis, we provide improved upper bounds for many hard instances from the literature by using methods based on Stochastic Local Search (SLS). As the most contribution part of this work, we propose a cutting plane algorithm to separate five different cut families, as well as a new preprocessing routine to strengthen resource-related constraints. New lifted versions of the well-known precedence and cover inequalities are employed. At each iteration, a dense conict graph is built considering feasibility and optimality conditions to separate cliques, odd-holes and strengthened Chv?tal-Gomory cuts. The proposed strategies considerably improve the linear relaxation bounds, allowing a state-of-the-art mixed-integer linear programming solver to nd provably optimal solutions for 754 previously open instances of different variants of the RCPSPs, which was not possible using the original linear programming formulations

Development and application of hyperheuristics to personnel scheduling

This thesis is concerned with the investigation of hyperheuristic techniques. Hyperheuristics are heuristics which choose heuristics in order to solve a given optimisation problem. In this thesis we investigate and develop a number of hyperheuristic techniques including a hyperheuristic which uses a choice function in order to select which low-level heuristic to apply at each decision point. We demonstrate the effectiveness of our hyperheuristics by means of three personnel scheduling problems taken from the real world. For each application problem, we apply our hyperheuristics to several instances and compare our results with those of other heuristic methods. For all problems, the choice function hyperheuristic appears to be superior to other hyperheuristics considered. It also produces results competitive with those obtained using other sophisticated means. It is hoped that - hyperheuristics can produce solutions of good quality, often competitive with those of modern heuristic techniques, within a short amount of implementation and development time, using only simple and easy-to-implement low-level heuristics. - hyperheuristics are easily re-usable methods as opposed to some metaheuristic methods which tend to use extensive problem-specific information in order to arrive at good solutions. These two latter points constitute the main contributions of this thesis