New variants of variable neighbourhood search for 0-1 mixed integer programming and clustering

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Many real-world optimisation problems are discrete in nature. Although recent rapid developments in computer technologies are steadily increasing the speed of computations, the size of an instance of a hard discrete optimisation problem solvable in prescribed time does not increase linearly with the computer speed. This calls for the development of new solution methodologies for solving larger instances in shorter time. Furthermore, large instances of discrete optimisation problems are normally impossible to solve to optimality within a reasonable computational time/space and can only be tackled with a heuristic approach. In this thesis the development of so called matheuristics, the heuristics which are based on the mathematical formulation of the problem, is studied and employed within the variable neighbourhood search framework. Some new variants of the variable neighbourhood searchmetaheuristic itself are suggested, which naturally emerge from exploiting the information from the mathematical programming formulation of the problem. However, those variants may also be applied to problems described by the combinatorial formulation. A unifying perspective on modern advances in local search-based metaheuristics, a so called hyper-reactive approach, is also proposed. Two NP-hard discrete optimisation problems are considered: 0-1 mixed integer programming and clustering with application to colour image quantisation. Several new heuristics for 0-1 mixed integer programming problem are developed, based on the principle of variable neighbourhood search. One set of proposed heuristics consists of improvement heuristics, which attempt to find high-quality near-optimal solutions starting from a given feasible solution. Another set consists of constructive heuristics, which attempt to find initial feasible solutions for 0-1 mixed integer programs. Finally, some variable neighbourhood search based clustering techniques are applied for solving the colour image quantisation problem. All new methods presented are compared to other algorithms recommended in literature and a comprehensive performance analysis is provided. Computational results show that the methods proposed either outperform the existing state-of-the-art methods for the problems observed, or provide comparable results. The theory and algorithms presented in this thesis indicate that hybridisation of the CPLEX MIP solver and the VNS metaheuristic can be very effective for solving large instances of the 0-1 mixed integer programming problem. More generally, the results presented in this thesis suggest that hybridisation of exact (commercial) integer programming solvers and some metaheuristic methods is of high interest and such combinations deserve further practical and theoretical investigation. Results also show that VNS can be successfully applied to solving a colour image quantisation problem.Support from the Mathematical Institute, Serbian Academy of Sciences and Arts, are acknowledged for this research

Incorporating Memory and Learning Mechanisms Into Meta-RaPS

Due to the rapid increase of dimensions and complexity of real life problems, it has become more difficult to find optimal solutions using only exact mathematical methods. The need to find near-optimal solutions in an acceptable amount of time is a challenge when developing more sophisticated approaches. A proper answer to this challenge can be through the implementation of metaheuristic approaches. However, a more powerful answer might be reached by incorporating intelligence into metaheuristics. Meta-RaPS (Metaheuristic for Randomized Priority Search) is a metaheuristic that creates high quality solutions for discrete optimization problems. It is proposed that incorporating memory and learning mechanisms into Meta-RaPS, which is currently classified as a memoryless metaheuristic, can help the algorithm produce higher quality results. The proposed Meta-RaPS versions were created by taking different perspectives of learning. The first approach taken is Estimation of Distribution Algorithms (EDA), a stochastic learning technique that creates a probability distribution for each decision variable to generate new solutions. The second Meta-RaPS version was developed by utilizing a machine learning algorithm, Q Learning, which has been successfully applied to optimization problems whose output is a sequence of actions. In the third Meta-RaPS version, Path Relinking (PR) was implemented as a post-optimization method in which the new algorithm learns the good attributes by memorizing best solutions, and follows them to reach better solutions. The fourth proposed version of Meta-RaPS presented another form of learning with its ability to adaptively tune parameters. The efficiency of these approaches motivated us to redesign Meta-RaPS by removing the improvement phase and adding a more sophisticated Path Relinking method. The new Meta-RaPS could solve even the largest problems in much less time while keeping up the quality of its solutions. To evaluate their performance, all introduced versions were tested using the 0-1 Multidimensional Knapsack Problem (MKP). After comparing the proposed algorithms, Meta-RaPS PR and Meta-RaPS Q Learning appeared to be the algorithms with the best and worst performance, respectively. On the other hand, they could all show superior performance than other approaches to the 0-1 MKP in the literature

New variants of variable neighbourhood search for 0-1 mixed integer programming and clustering

Many real-world optimisation problems are discrete in nature. Although recent rapid developments in computer technologies are steadily increasing the speed of computations, the size of an instance of a hard discrete optimisation problem solvable in prescribed time does not increase linearly with the computer speed. This calls for the development of new solution methodologies for solving larger instances in shorter time. Furthermore, large instances of discrete optimisation problems are normally impossible to solve to optimality within a reasonable computational time/space and can only be tackled with a heuristic approach. In this thesis the development of so called matheuristics, the heuristics which are based on the mathematical formulation of the problem, is studied and employed within the variable neighbourhood search framework. Some new variants of the variable neighbourhood searchmetaheuristic itself are suggested, which naturally emerge from exploiting the information from the mathematical programming formulation of the problem. However, those variants may also be applied to problems described by the combinatorial formulation. A unifying perspective on modern advances in local search-based metaheuristics, a so called hyper-reactive approach, is also proposed. Two NP-hard discrete optimisation problems are considered: 0-1 mixed integer programming and clustering with application to colour image quantisation. Several new heuristics for 0-1 mixed integer programming problem are developed, based on the principle of variable neighbourhood search. One set of proposed heuristics consists of improvement heuristics, which attempt to find high-quality near-optimal solutions starting from a given feasible solution. Another set consists of constructive heuristics, which attempt to find initial feasible solutions for 0-1 mixed integer programs. Finally, some variable neighbourhood search based clustering techniques are applied for solving the colour image quantisation problem. All new methods presented are compared to other algorithms recommended in literature and a comprehensive performance analysis is provided. Computational results show that the methods proposed either outperform the existing state-of-the-art methods for the problems observed, or provide comparable results. The theory and algorithms presented in this thesis indicate that hybridisation of the CPLEX MIP solver and the VNS metaheuristic can be very effective for solving large instances of the 0-1 mixed integer programming problem. More generally, the results presented in this thesis suggest that hybridisation of exact (commercial) integer programming solvers and some metaheuristic methods is of high interest and such combinations deserve further practical and theoretical investigation. Results also show that VNS can be successfully applied to solving a colour image quantisation problem.EThOS - Electronic Theses Online ServiceMathematical Institute, Serbian Academy of Sciences and ArtsGBUnited Kingdo

Evolvable Neuronal Paths: A Novel Basis for Information and Search in the Brain

We propose a previously unrecognized kind of informational entity in the brain that is capable of acting as the basis for unlimited hereditary variation in neuronal networks. This unit is a path of activity through a network of neurons, analogous to a path taken through a hidden Markov model. To prove in principle the capabilities of this new kind of informational substrate, we show how a population of paths can be used as the hereditary material for a neuronally implemented genetic algorithm, (the swiss-army knife of black-box optimization techniques) which we have proposed elsewhere could operate at somatic timescales in the brain. We compare this to the same genetic algorithm that uses a standard ‘genetic’ informational substrate, i.e. non-overlapping discrete genotypes, on a range of optimization problems. A path evolution algorithm (PEA) is defined as any algorithm that implements natural selection of paths in a network substrate. A PEA is a previously unrecognized type of natural selection that is well suited for implementation by biological neuronal networks with structural plasticity. The important similarities and differences between a standard genetic algorithm and a PEA are considered. Whilst most experiments are conducted on an abstract network model, at the conclusion of the paper a slightly more realistic neuronal implementation of a PEA is outlined based on Izhikevich spiking neurons. Finally, experimental predictions are made for the identification of such informational paths in the brain

A Comprehensive Survey on Particle Swarm Optimization Algorithm and Its Applications

Particle swarm optimization (PSO) is a heuristic global optimization method, proposed originally by Kennedy and Eberhart in 1995. It is now one of the most commonly used optimization techniques. This survey presented a comprehensive investigation of PSO. On one hand, we provided advances with PSO, including its modifications (including quantum-behaved PSO, bare-bones PSO, chaotic PSO, and fuzzy PSO), population topology (as fully connected, von Neumann, ring, star, random, etc.), hybridization (with genetic algorithm, simulated annealing, Tabu search, artificial immune system, ant colony algorithm, artificial bee colony, differential evolution, harmonic search, and biogeography-based optimization), extensions (to multiobjective, constrained, discrete, and binary optimization), theoretical analysis (parameter selection and tuning, and convergence analysis), and parallel implementation (in multicore, multiprocessor, GPU, and cloud computing forms). On the other hand, we offered a survey on applications of PSO to the following eight fields: electrical and electronic engineering, automation control systems, communication theory, operations research, mechanical engineering, fuel and energy, medicine, chemistry, and biology. It is hoped that this survey would be beneficial for the researchers studying PSO algorithms

Evolutionary Computation

This book presents several recent advances on Evolutionary Computation, specially evolution-based optimization methods and hybrid algorithms for several applications, from optimization and learning to pattern recognition and bioinformatics. This book also presents new algorithms based on several analogies and metafores, where one of them is based on philosophy, specifically on the philosophy of praxis and dialectics. In this book it is also presented interesting applications on bioinformatics, specially the use of particle swarms to discover gene expression patterns in DNA microarrays. Therefore, this book features representative work on the field of evolutionary computation and applied sciences. The intended audience is graduate, undergraduate, researchers, and anyone who wishes to become familiar with the latest research work on this field

Meta-Heuristics for the Multiple Trip Vehicle Routing Problem with Backhauls

With the growing and more accessible computational power, the demand for robust and sophisticated computerised optimisation is increasing for logistical problems. By making good use of computational technologies, the research in this thesis concentrates on efficient fleet management by studying a class of vehicle routing problems and developing efficient solution algorithms. The literature review in this thesis looks at VRPs from various development angles. The search reveals that from the problem modelling side clear efforts are made to bring the classical VRP models closer to reality by developing various variants. However, apart from the real VRP applications (termed as 'rich' VRPs), it is also noticeable that these classical VRP based variants address merely one or two additional characteristics from the real routing problem issues, concentrating on either operational (fleet management) or tactical (fleet acquisition) aspects. This thesis certainly hopes to add to one of those good efforts which have helped in bringing the VRPs closer to reality through addressing both the operational as well as the tactical aspects. On the solution methodologies development side, the proposed research noted some considerable and impressive developments. Although, it is well established that the VRPs belong to the NP-hard combinatorial class of problems, there are considerable efforts on the development of exact methods. However the literature is full of a variety of heuristic methodologies including the classical and the most modern hybrid approaches. Among the hybrid approaches, the most recent one noted is mat-heuristics that combine heuristics and mathematical programming techniques to solve combinatorial optimisation problems. The mat-heuristics approaches appear to be comparatively in its infant age at this point in time. However this is an exciting area of research which seeks more attention in the literature. Hence, a good part of this research is devoted to the development of a hybrid approach that combines heuristics and mathematical programming techniques. When reviewing the specific literature on the VRP problems focused in this thesis, the vehicle routing problem with backhauls (VRPB) and the multiple trip vehicle routing problem (MT-VRP), there is not sufficient development on the problem modelling side in terms of bringing these two problems closer to the reality. Hence, to fill the gap this thesis introduces and investigates a new variant, the multiple trip vehicle routing problem with backhauls (MT-VRPB) that combines the above two variants of the VRP. The problem is first described thoroughly and a new ILP (Integer Linear Programming) mathematical formulation of the MT-VRPB along with its possible variations is presented. The MT-VRPB is then solved optimally by using CPLEX along with providing an illustrative example showing the validation of the mathematical formulation. As part of the contribution, a large set of MT-VRPB data instances is created which is made available for future benchmarking. The CPLEX implementation produced optimal solutions for a good number of small and medium size data instances of the MT-VRPB and generated lower bounds for all instances. The CPLEX success may be considered as modest, but the produced results proved very important for the validation of the heuristic results produced in the thesis. To solve the larger instances of the MT-VRPB, a two level VNS algorithm called 'Two-Level VNS' is developed. It was noticed from the literature that the choice of using VNS for the VRPs has increased in recent literature due to its simplicity and speed. However our initial experiments with the classical VNS indicated that the algorithm is more inclined towards the intensification side. Hence, the Two-Level VNS is designed to obtain a maximum balance of the diversification and the intensification during the search process. It is achieved by incorporating a sub-set of neighbourhood structures and a sus-set of local search refinement routines and hence, a full set of neighbourhood structures and a full set of local search refinement routines at two levels of the algorithm respectively. The algorithm found very encouraging results when compared with the solutions found by CPLEX. These findings in this thesis demonstrate the power of VNS yet again in terms of its speed, simplicity and efficiency. To investigate this new variant further, we developed an algorithm belonging to the new class of the hybrid methodologies, i.e., mat-heuristics. A hybrid collaborative sequential mat-heuristic approach called the CSMH to solve the MT-VRPB is developed. The exact method approach produced in Chapter 4 is then hybridised with the Two-Level VNS algorithm developed in Chapter 5. The overall performance of the CSMH remained very encouraging in terms of the solution quality and the time taken on average compared with the CPLEX and the Two-Level VNS meta-heuristic. To demonstrate the power and effectiveness of our methodologies, we tested the designed algorithms on the two special versions of the VRP (i.e., VRPB and MT-VRP) to assess whether they are efficient and dynamic enough to solve a range of VRP variants. Hence the Two-Level VNS and the CSMH algorithms developed to solve the MT-VRPB are adapted accordingly and implemented to solve the two above variants separately. The algorithms produced very competitive results for the benchmark data sets when compared to the best known solutions from the literature. The successful implementations of these algorithms on the three VRP models with only minor amendments prove their generalizability and their robustness. The results in this research show that significant cost savings could be obtained by choosing the right fleet size and better vehicle utilisations with multiple trips and backhauling. Hence, the research proved the justification of studying this interesting combination. Moreover, the problem modelling, efficient algorithm design and implementation, and the research results reveal some vital information and implications from the managerial point of view in terms of making the tactical (fleet acquisition) and the operational (fleet management) decisions in a more informative manner

La métaheuristique CAT pour le design de réseaux logistiques déterministes et stochastiques

De nos jours, les entreprises d’ici et d’ailleurs sont confrontées à une concurrence mondiale sans cesse plus féroce. Afin de survivre et de développer des avantages concurrentiels, elles doivent s’approvisionner et vendre leurs produits sur les marchés mondiaux. Elles doivent aussi offrir simultanément à leurs clients des produits d’excellente qualité à prix concurrentiels et assortis d’un service impeccable. Ainsi, les activités d’approvisionnement, de production et de marketing ne peuvent plus être planifiées et gérées indépendamment. Dans ce contexte, les grandes entreprises manufacturières se doivent de réorganiser et reconfigurer sans cesse leur réseau logistique pour faire face aux pressions financières et environnementales ainsi qu’aux exigences de leurs clients. Tout doit être révisé et planifié de façon intégrée : sélection des fournisseurs, choix d’investissements, planification du transport et préparation d’une proposition de valeur incluant souvent produits et services au fournisseur. Au niveau stratégique, ce problème est fréquemment désigné par le vocable « design de réseau logistique ». Une approche intéressante pour résoudre ces problématiques décisionnelles complexes consiste à formuler et résoudre un modèle mathématique en nombres entiers représentant la problématique. Plusieurs modèles ont ainsi été récemment proposés pour traiter différentes catégories de décision en matière de design de réseau logistique. Cependant, ces modèles sont très complexes et difficiles à résoudre, et même les solveurs les plus performants échouent parfois à fournir une solution de qualité. Les travaux développés dans cette thèse proposent plusieurs contributions. Tout d’abord, un modèle de design de réseau logistique incorporant plusieurs innovations proposées récemment dans la littérature a été développé; celui-ci intègre les dimensions du choix des fournisseurs, la localisation, la configuration et l’assignation de mission aux installations (usines, entrepôts, etc.) de l’entreprise, la planification stratégique du transport et la sélection de politiques de marketing et d’offre de valeur au consommateur. Des innovations sont proposées au niveau de la modélisation des inventaires ainsi que de la sélection des options de transport. En deuxième lieu, une méthode de résolution distribuée inspirée du paradigme des systèmes multi-agents a été développée afin de résoudre des problèmes d’optimisation de grande taille incorporant plusieurs catégories de décisions. Cette approche, appelée CAT (pour collaborative agent teams), consiste à diviser le problème en un ensemble de sous-problèmes, et assigner chacun de ces sous-problèmes à un agent qui devra le résoudre. Par la suite, les solutions à chacun de ces sous-problèmes sont combinées par d’autres agents afin d’obtenir une solution de qualité au problème initial. Des mécanismes efficaces sont conçus pour la division du problème, pour la résolution des sous-problèmes et pour l’intégration des solutions. L’approche CAT ainsi développée est utilisée pour résoudre le problème de design de réseaux logistiques en univers certain (déterministe). Finalement, des adaptations sont proposées à CAT permettant de résoudre des problèmes de design de réseaux logistiques en univers incertain (stochastique)

Evaluating human-centered approaches for geovisualization

Working with two small group of domain experts I evaluate human-centered approaches to application development which are applicable to geovisualization, following an ISO13407 taxonomy that covers context of use, eliciting requirements, and design. These approaches include field studies and contextual analysis of subjects' context; establishing requirements using a template, via a lecture to communicate geovisualization to subjects and by communicating subjects' context to geovisualization experts with a scenario; autoethnography to understand the geovisualization design process; wireframe, paper and digital interactive prototyping with alternative protocols; and a decision making process for prioritising application improvement. I find that the acquisition and use of real user data is key; that a template approach and teaching subjects about visualization tools and interactions both fail to elicit useful requirements for a visualization application. Consulting geovisualization experts with a scenario of user context and samples of user data does yield suggestions for tools and interactions of use to a visualization designer. The complex and composite natures of both visualization and human-centered domains, incorporating learning from both domains, with user context, makes design challenging. Wireframe, paper and digital interactive prototypes mediate between the user and visualization domains successfully, eliciting exploratory behaviour and suggestions to improve prototypes. Paper prototypes are particularly successful at eliciting suggestions and especially novel visualization improvements. Decision-making techniques prove useful for prioritising different possible improvements, although domain subjects select data-related features over more novel alternative and rank these more inconsistently. The research concludes that understanding subject context of use and data is important and occurs throughout the process of engagement with domain experts, and that standard requirements elicitation techniques are unsuccessful for geovisualization. Engagement with subjects at an early stage with simple prototypes incorporating real subject data and moving to successively more complex prototypes holds the best promise for creating successful geovisualization applications