Matheuristics:survey and synthesis

In integer programming and combinatorial optimisation, people use the term matheuristics to refer to methods that are heuristic in nature, but draw on concepts from the literature on exact methods. We survey the literature on this topic, with a particular emphasis on matheuristics that yield both primal and dual bounds (i.e., upper and lower bounds in the case of a minimisation problem). We also make some comments about possible future developments

Developing New Multidimensional Knapsack Heuristics Based on Empirical Analysis of Legacy Heuristics

The multidimensional knapsack problem (MKP) has been used to model a variety of practical optimization and decision-making applications. Due to its combinatorial nature, heuristics are often employed to quickly find good solutions to MKPs. While there have been a variety of heuristics proposed for the MKP, and a plethora of empirical studies comparing the performance of these heuristics, little has been done to garner a deeper understanding of heuristic performance as a function of problem structure. This dissertation presents a research methodology, empirical and theoretical results explicitly aimed at gaining a deeper understanding of heuristic procedural performance as a function of test problem characteristics. This work first employs an available, robust set of two-dimensional knapsack problems in an empirical study to garner performance insights. These performance insights are tested against a larger set of problems, five-dimensional knapsack problems specifically generated for empirical testing purposes. The performance insights are found to hold in the higher dimensions. These insights are used to formulate and test a suite of three new greedy heuristics for the MKP, each improving upon its successor. These heuristics are found to outperform available legacy heuristics across a complete spectrum of test problems. Problem reduction heuristics are examined and the subsequent performance insights garnered are used to derive a new problem reduction heuristic, which is then further extended to employ a local improvement phase. These problem reduction heuristics are also found to outperform currently available approaches. Available problem test sets are shown lacking along multiple dimensions of importance for viable empirical testing. A new problem generation methodology is developed and shown to overcome the current limitations in available problem test sets. This problem generation methodology is used to generate a new set of empirical test problems specifically designed for competitive computational tests. This new test set is shown to stress existing heuristics; not only does the computational time required by these legacy heuristics increase with problem size, but solution quality is found to decrease with problem size. However, the solution quality obtained by the suite of heuristics developed in this dissertation are shown to be unaffected by problem size thereby providing a level of robust solution quality not previously seen in heuristic development for the MKP. This research demonstrates that the test problems can have a profound, and sometimes misleading, impact on the general insights gained via empirical testing, provides six new quality heuristics, and two new robust sets of test problems, one focused on empirical testing, the other focused on competitive testing

Proceedings of the XIII Global Optimization Workshop: GOW'16

[Excerpt] Preface: Past Global Optimization Workshop shave been held in Sopron (1985 and 1990), Szeged (WGO, 1995), Florence (GO’99, 1999), Hanmer Springs (Let’s GO, 2001), Santorini (Frontiers in GO, 2003), San José (Go’05, 2005), Mykonos (AGO’07, 2007), Skukuza (SAGO’08, 2008), Toulouse (TOGO’10, 2010), Natal (NAGO’12, 2012) and Málaga (MAGO’14, 2014) with the aim of stimulating discussion between senior and junior researchers on the topic of Global Optimization. In 2016, the XIII Global Optimization Workshop (GOW’16) takes place in Braga and is organized by three researchers from the University of Minho. Two of them belong to the Systems Engineering and Operational Research Group from the Algoritmi Research Centre and the other to the Statistics, Applied Probability and Operational Research Group from the Centre of Mathematics. The event received more than 50 submissions from 15 countries from Europe, South America and North America. We want to express our gratitude to the invited speaker Panos Pardalos for accepting the invitation and sharing his expertise, helping us to meet the workshop objectives. GOW’16 would not have been possible without the valuable contribution from the authors and the International Scientific Committee members. We thank you all. This proceedings book intends to present an overview of the topics that will be addressed in the workshop with the goal of contributing to interesting and fruitful discussions between the authors and participants. After the event, high quality papers can be submitted to a special issue of the Journal of Global Optimization dedicated to the workshop. [...

Revisiting the Evolution and Application of Assignment Problem: A Brief Overview

The assignment problem (AP) is incredibly challenging that can model many real-life problems. This paper provides a limited review of the recent developments that have appeared in the literature, meaning of assignment problem as well as solving techniques and will provide a review on   a lot of research studies on different types of assignment problem taking place in present day real life situation in order to capture the variations in different types of assignment techniques. Keywords: Assignment problem, Quadratic Assignment, Vehicle Routing, Exact Algorithm, Bound, Heuristic etc

The Plant Propagation Algorithm for Discrete Optimisation

The thesis is concerned with novel Nature-Inspired heuristics for the so called NP-hard problems of optimisation. A particular algorithm which has been recently introduced and shown to be effective in continuous optimisation is the Plant Propagation Algorithm or PPA. Here, we intend to extend it to cope with combinatorial optimisation. In order to show that our extension is viable and effective, we consider three types of problems which are good representatives of the whole topic. These are the Travelling Salesman Problem or TSP, the Knapsack Problem or KP and the scheduling problem of Berth Allocation as arises in container ports or BAP. Because PPA is a population-based search heuristic, we devote a chapter to the important issue of generating good and yet computationally relatively light initial populations of solutions to kick start the search process. In the case of the TSP we revisit and extend the Strip Algorithm (SA). We introduce the 2-Part SA and show that it is better than the classical SA. We also introduce new variants such as the Adaptive SA and the Spiral SA which cope with clustered cities and instances with cities concentrated around the center of the unit square, respectively. In the case of KP we adapt the Roulette Wheel selection approach to generate solutions to start with PPA. And in the case of BAP, we introduce a number of simple heuristics which consider a schedule as a flat box with one side being the processing time and the other the position of vessels on the wharf. The heuristics try to generate schedules by avoiding overlap as much as possible. All approaches and algorithms are implemented and tested against well established algorithms. The results are recorded and discussed extensively. The thesis ends with a conclusion and ideas for further research

Co-evolutionary Hybrid Bi-level Optimization

Multi-level optimization stems from the need to tackle complex problems involving multiple decision makers. Two-level optimization, referred as ``Bi-level optimization'', occurs when two decision makers only control part of the decision variables but impact each other (e.g., objective value, feasibility). Bi-level problems are sequential by nature and can be represented as nested optimization problems in which one problem (the ``upper-level'') is constrained by another one (the ``lower-level''). The nested structure is a real obstacle that can be highly time consuming when the lower-level is $\mathcal{NP}-hard$. Consequently, classical nested optimization should be avoided. Some surrogate-based approaches have been proposed to approximate the lower-level objective value function (or variables) to reduce the number of times the lower-level is globally optimized. Unfortunately, such a methodology is not applicable for large-scale and combinatorial bi-level problems. After a deep study of theoretical properties and a survey of the existing applications being bi-level by nature, problems which can benefit from a bi-level reformulation are investigated. A first contribution of this work has been to propose a novel bi-level clustering approach. Extending the well-know ``uncapacitated k-median problem'', it has been shown that clustering can be easily modeled as a two-level optimization problem using decomposition techniques. The resulting two-level problem is then turned into a bi-level problem offering the possibility to combine distance metrics in a hierarchical manner. The novel bi-level clustering problem has a very interesting property that enable us to tackle it with classical nested approaches. Indeed, its lower-level problem can be solved in polynomial time. In cooperation with the Luxembourg Centre for Systems Biomedicine (LCSB), this new clustering model has been applied on real datasets such as disease maps (e.g. Parkinson, Alzheimer). Using a novel hybrid and parallel genetic algorithm as optimization approach, the results obtained after a campaign of experiments have the ability to produce new knowledge compared to classical clustering techniques combining distance metrics in a classical manner. The previous bi-level clustering model has the advantage that the lower-level can be solved in polynomial time although the global problem is by definition $\mathcal{NP}$-hard. Therefore, next investigations have been undertaken to tackle more general bi-level problems in which the lower-level problem does not present any specific advantageous properties. Since the lower-level problem can be very expensive to solve, the focus has been turned to surrogate-based approaches and hyper-parameter optimization techniques with the aim of approximating the lower-level problem and reduce the number of global lower-level optimizations. Adapting the well-know bayesian optimization algorithm to solve general bi-level problems, the expensive lower-level optimizations have been dramatically reduced while obtaining very accurate solutions. The resulting solutions and the number of spared lower-level optimizations have been compared to the bi-level evolutionary algorithm based on quadratic approximations (BLEAQ) results after a campaign of experiments on official bi-level benchmarks. Although both approaches are very accurate, the bi-level bayesian version required less lower-level objective function calls. Surrogate-based approaches are restricted to small-scale and continuous bi-level problems although many real applications are combinatorial by nature. As for continuous problems, a study has been performed to apply some machine learning strategies. Instead of approximating the lower-level solution value, new approximation algorithms for the discrete/combinatorial case have been designed. Using the principle employed in GP hyper-heuristics, heuristics are trained in order to tackle efficiently the $\mathcal{NP}-hard$ lower-level of bi-level problems. This automatic generation of heuristics permits to break the nested structure into two separated phases: \emph{training lower-level heuristics} and \emph{solving the upper-level problem with the new heuristics}. At this occasion, a second modeling contribution has been introduced through a novel large-scale and mixed-integer bi-level problem dealing with pricing in the cloud, i.e., the Bi-level Cloud Pricing Optimization Problem (BCPOP). After a series of experiments that consisted in training heuristics on various lower-level instances of the BCPOP and using them to tackle the bi-level problem itself, the obtained results are compared to the ``cooperative coevolutionary algorithm for bi-level optimization'' (COBRA). Although training heuristics enables to \emph{break the nested structure}, a two phase optimization is still required. Therefore, the emphasis has been put on training heuristics while optimizing the upper-level problem using competitive co-evolution. Instead of adopting the classical decomposition scheme as done by COBRA which suffers from the strong epistatic links between lower-level and upper-level variables, co-evolving the solution and the mean to get to it can cope with these epistatic link issues. The ``CARBON'' algorithm developed in this thesis is a competitive and hybrid co-evolutionary algorithm designed for this purpose. In order to validate the potential of CARBON, numerical experiments have been designed and results have been compared to state-of-the-art algorithms. These results demonstrate that ``CARBON'' makes possible to address nested optimization efficiently

The assignment problem with dependent costs.

Assigning workers, each with their own skill set, to tasks which demand different skills in an efficient manner is a challenging problem that often requires workers to receive additional training. The training of workers is very costly with Training Magazine’s Annual Industry Report stating 58.5 billion dollars were spent in 2007 on employee training in the United States. Therefore assigning workers to tasks in such a way as to minimize the overall training costs is an important problem in many organizations. In this research, the assignment problem with dependent cost is considered, i.e. the training cost associated with assigning a worker to a particular task depends on the training the worker receives for their other assigned tasks. Once a worker is trained in a skill that training will available for any additional tasks that may be assigned. The problem is formulated mathematically as an integer linear program. Based on past research, high quality solutions to large-size problems are difficult to obtain. This research develops and upper bound approach and three heuristic solution methodologies. The basic idea of the heuristics is to form groups of tasks which require similar skills, then assign a worker to the task group. The Shortest Augmenting Path (SAP) algorithm of Jonker and Volgenant is known to quickly find the optimal assignment of N workers to N tasks. This SAP algorithm will be used in this research after grouping the tasks into N groups which can then be assigned to the N workers. The task grouping heuristic methods developed in this research were tested for several randomly generated large-sized data sets. Results showed an average 7.34% improvement compared to previous solution methods. Additionally to consider workers’ preferences, a multiple-objective model is presented for the skills management problem to maximize workers’ preferences and aggregate training while minimizing training cost. The model is demonstrated for randomly generated data sets

Exact and heuristic approaches for multi-component optimisation problems

Modern real world applications are commonly complex, consisting of multiple subsystems that may interact with or depend on each other. Our case-study about wave energy converters (WEC) for the renewable energy industry shows that in such a multi-component system, optimising each individual component cannot yield global optimality for the entire system, owing to the influence of their interactions or the dependence on one another. Moreover, modelling a multi-component problem is rarely easy due to the complexity of the issues, which leads to a desire for existent models on which to base, and against which to test, calculations. Recently, the travelling thief problem (TTP) has attracted significant attention in the Evolutionary Computation community. It is intended to offer a better model for multicomponent systems, where researchers can push forward their understanding of the optimisation of such systems, especially for understanding of the interconnections between the components. The TTP interconnects with two classic NP-hard problems, namely the travelling salesman problem and the 0-1 knapsack problem, via the transportation cost that non-linearly depends on the accumulated weight of items. This non-linear setting introduces additional complexity. We study this nonlinearity through a simplified version of the TTP - the packing while travelling (PWT) problem, which aims to maximise the total reward for a given travelling tour. Our theoretical and experimental investigations demonstrate that the difficulty of a given problem instance is significantly influenced by adjusting a single parameter, the renting rate, which prompted our method of creating relatively hard instances using simple evolutionary algorithms. Our further investigations into the PWT problem yield a dynamic programming (DP) approach that can solve the problem in pseudo polynomial time and a corresponding approximation scheme. The experimental investigations show that the new approaches outperform the state-of-the-art ones. We furthermore propose three exact algorithms for the TTP, based on the DP of the PWT problem. By employing the exact DP for the underlying PWT problem as a subroutine, we create a novel indicator-based hybrid evolutionary approach for a new bi-criteria formulation of the TTP. This hybrid design takes advantage of the DP approach, along with a number of novel indicators and selection mechanisms to achieve better solutions. The results of computational experiments show that the approach is capable to outperform the state-of-the-art results.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 201