Combinatorial optimization and metaheuristics

Today, combinatorial optimization is one of the youngest and most active areas of discrete mathematics. It is a branch of optimization in applied mathematics and computer science, related to operational research, algorithm theory and computational complexity theory. It sits at the intersection of several fields, including artificial intelligence, mathematics and software engineering. Its increasing interest arises for the fact that a large number of scientific and industrial problems can be formulated as abstract combinatorial optimization problems, through graphs and/or (integer) linear programs. Some of these problems have polynomial-time (“efficient”) algorithms, while most of them are NP-hard, i.e. it is not proved that they can be solved in polynomial-time. Mainly, it means that it is not possible to guarantee that an exact solution to the problem can be found and one has to settle for an approximate solution with known performance guarantees. Indeed, the goal of approximate methods is to find “quickly” (reasonable run-times), with “high” probability, provable “good” solutions (low error from the real optimal solution). In the last 20 years, a new kind of algorithm commonly called metaheuristics have emerged in this class, which basically try to combine heuristics in high level frameworks aimed at efficiently and effectively exploring the search space. This report briefly outlines the components, concepts, advantages and disadvantages of different metaheuristic approaches from a conceptual point of view, in order to analyze their similarities and differences. The two very significant forces of intensification and diversification, that mainly determine the behavior of a metaheuristic, will be pointed out. The report concludes by exploring the importance of hybridization and integration methods

A Comparison of GAs Penalizing Infeasible Solutions and Repairing Infeasible Solutions on the 0-1 Knapsack Problem

Constraints exist in almost every optimization problem. Different constraint handling techniques have been incorporated with genetic algorithms (GAs), however most of current studies are based on computer experiments. An example is Michalewicz\u27s comparison among GAs using different constraint handling techniques on the 0-1 knapsack problem. The following phenomena are observed in experiments: 1) the penalty method needs more generations to find a feasible solution to the restrictive capacity knapsack than the repair method; 2) the penalty method can find better solutions to the average capacity knapsack. Such observations need a theoretical explanation. This paper aims at providing a theoretical analysis of Michalewicz\u27s experiments. The main result of the paper is that GAs using the repair method are more efficient than GAs using the penalty method on both restrictive capacity and average capacity knapsack problems. This result of the average capacity is a little different from Michalewicz\u27s experimental results. So a supplemental experiment is implemented to support the theoretical claim. The results confirm the general principle pointed out by Coello: a better constraint-handling approach should tend to exploit specific domain knowledge

Neuroevolution for solving multiobjective knapsack problems

The multiobjective knapsack problem (MOKP) is an important combinatorial problem that arises in various applications, including resource allocation, computer science and finance. When tackling this problem by evolutionary multiobjective optimization algorithms (EMOAs), it has been demonstrated that traditional recombination operators acting on binary solution representations are susceptible to a loss of diversity and poor scalability. To address those issues, we propose to use artificial neural networks for generating solutions by performing a binary classification of items using the information about their profits and weights. As gradient-based learning cannot be used when target values are unknown, neuroevolution is adapted to adjust the neural network parameters. The main contribution of this study resides in developing a solution encoding and genotype-phenotype mapping for EMOAs to solve MOKPs. The proposal is implemented within a state-of-the-art EMOA and benchmarked against traditional variation operators based on binary crossovers. The obtained experimental results indicate a superior performance of the proposed approach. Furthermore, it is advantageous in terms of scalability and can be readily incorporated into different EMOAs.Portuguese “Fundação para a Ciência e Tecnologia” under grant PEst-C/CTM/LA0025/2013 (Projecto Estratégico - LA 25 - 2013-2014 - Strategic Project - LA 25 - 2013-2014

Genetic algorithms with implicit memory

This thesis investigates the workings of genetic algorithms in dynamic optimisation problems where fitness landscapes materialise that are identical to, or resemble in some way, landscapes previously encountered. The objective is to appraise the performances of the various approaches offered by the GAs. Approaches specifically tailored for different kinds of dynamic environment lie outside the remit of the thesis. The main topics that are explored are: genetic redundancy, modularity, neutral evolution, explicit memory, and implicit memory. It is in the matter of implicit memory that the thesis makes the majority of its novel contributions. It is demonstrated via experimental analysis that the pre-existing techniques are deficient, and a new algorithm – the pointer genetic algorithm (pGA) – is expounded and assessed in an attempt to offer an improvement. It is shown that though it outperforms its rivals, it cannot attain the performance levels of an explicit memory algorithm (that is, an algorithm using an external memory bank). The main claims of the thesis are that with regard to memory, the pre-existing implicit-memory algorithms are deficient, the new pointer GA is superior, and that because all of the implicit approaches are inferior to explicit approaches, it is explicit approaches that should be used in real-world problem solving

Incorporating Memory and Learning Mechanisms Into Meta-RaPS

Due to the rapid increase of dimensions and complexity of real life problems, it has become more difficult to find optimal solutions using only exact mathematical methods. The need to find near-optimal solutions in an acceptable amount of time is a challenge when developing more sophisticated approaches. A proper answer to this challenge can be through the implementation of metaheuristic approaches. However, a more powerful answer might be reached by incorporating intelligence into metaheuristics. Meta-RaPS (Metaheuristic for Randomized Priority Search) is a metaheuristic that creates high quality solutions for discrete optimization problems. It is proposed that incorporating memory and learning mechanisms into Meta-RaPS, which is currently classified as a memoryless metaheuristic, can help the algorithm produce higher quality results. The proposed Meta-RaPS versions were created by taking different perspectives of learning. The first approach taken is Estimation of Distribution Algorithms (EDA), a stochastic learning technique that creates a probability distribution for each decision variable to generate new solutions. The second Meta-RaPS version was developed by utilizing a machine learning algorithm, Q Learning, which has been successfully applied to optimization problems whose output is a sequence of actions. In the third Meta-RaPS version, Path Relinking (PR) was implemented as a post-optimization method in which the new algorithm learns the good attributes by memorizing best solutions, and follows them to reach better solutions. The fourth proposed version of Meta-RaPS presented another form of learning with its ability to adaptively tune parameters. The efficiency of these approaches motivated us to redesign Meta-RaPS by removing the improvement phase and adding a more sophisticated Path Relinking method. The new Meta-RaPS could solve even the largest problems in much less time while keeping up the quality of its solutions. To evaluate their performance, all introduced versions were tested using the 0-1 Multidimensional Knapsack Problem (MKP). After comparing the proposed algorithms, Meta-RaPS PR and Meta-RaPS Q Learning appeared to be the algorithms with the best and worst performance, respectively. On the other hand, they could all show superior performance than other approaches to the 0-1 MKP in the literature

Model based test suite minimization using metaheuristics

Software testing is one of the most widely used methods for quality assurance and fault detection purposes. However, it is one of the most expensive, tedious and time consuming activities in software development life cycle. Code-based and specification-based testing has been going on for almost four decades. Model-based testing (MBT) is a relatively new approach to software testing where the software models as opposed to other artifacts (i.e. source code) are used as primary source of test cases. Models are simplified representation of a software system and are cheaper to execute than the original or deployed system. The main objective of the research presented in this thesis is the development of a framework for improving the efficiency and effectiveness of test suites generated from UML models. It focuses on three activities: transformation of Activity Diagram (AD) model into Colored Petri Net (CPN) model, generation and evaluation of AD based test suite and optimization of AD based test suite. Unified Modeling Language (UML) is a de facto standard for software system analysis and design. UML models can be categorized into structural and behavioral models. AD is a behavioral type of UML model and since major revision in UML version 2.x it has a new Petri Nets like semantics. It has wide application scope including embedded, workflow and web-service systems. For this reason this thesis concentrates on AD models. Informal semantics of UML generally and AD specially is a major challenge in the development of UML based verification and validation tools. One solution to this challenge is transforming a UML model into an executable formal model. In the thesis, a three step transformation methodology is proposed for resolving ambiguities in an AD model and then transforming it into a CPN representation which is a well known formal language with extensive tool support. Test case generation is one of the most critical and labor intensive activities in testing processes. The flow oriented semantic of AD suits modeling both sequential and concurrent systems. The thesis presented a novel technique to generate test cases from AD using a stochastic algorithm. In order to determine if the generated test suite is adequate, two test suite adequacy analysis techniques based on structural coverage and mutation have been proposed. In terms of structural coverage, two separate coverage criteria are also proposed to evaluate the adequacy of the test suite from both perspectives, sequential and concurrent. Mutation analysis is a fault-based technique to determine if the test suite is adequate for detecting particular types of faults. Four categories of mutation operators are defined to seed specific faults into the mutant model. Another focus of thesis is to improve the test suite efficiency without compromising its effectiveness. One way of achieving this is identifying and removing the redundant test cases. It has been shown that the test suite minimization by removing redundant test cases is a combinatorial optimization problem. An evolutionary computation based test suite minimization technique is developed to address the test suite minimization problem and its performance is empirically compared with other well known heuristic algorithms. Additionally, statistical analysis is performed to characterize the fitness landscape of test suite minimization problems. The proposed test suite minimization solution is extended to include multi-objective minimization. As the redundancy is contextual, different criteria and their combination can significantly change the solution test suite. Therefore, the last part of the thesis describes an investigation into multi-objective test suite minimization and optimization algorithms. The proposed framework is demonstrated and evaluated using prototype tools and case study models. Empirical results have shown that the techniques developed within the framework are effective in model based test suite generation and optimizatio

Applied (Meta)-Heuristic in Intelligent Systems

Engineering and business problems are becoming increasingly difficult to solve due to the new economics triggered by big data, artificial intelligence, and the internet of things. Exact algorithms and heuristics are insufficient for solving such large and unstructured problems; instead, metaheuristic algorithms have emerged as the prevailing methods. A generic metaheuristic framework guides the course of search trajectories beyond local optimality, thus overcoming the limitations of traditional computation methods. The application of modern metaheuristics ranges from unmanned aerial and ground surface vehicles, unmanned factories, resource-constrained production, and humanoids to green logistics, renewable energy, circular economy, agricultural technology, environmental protection, finance technology, and the entertainment industry. This Special Issue presents high-quality papers proposing modern metaheuristics in intelligent systems

How to exploit fitness landscape properties of timetabling problem: A newoperator for quantum evolutionary algorithm

© 2020 Elsevier Ltd. All rights reserved. This is the accepted manuscript version of an article which has been published in final form at https://doi.org/10.1016/j.eswa.2020.114211The fitness landscape of the timetabling problems is analyzed in this paper to provide some insight into theproperties of the problem. The analyses suggest that the good solutions are clustered in the search space andthere is a correlation between the fitness of a local optimum and its distance to the best solution. Inspiredby these findings, a new operator for Quantum Evolutionary Algorithms is proposed which, during the searchprocess, collects information about the fitness landscape and tried to capture the backbone structure of thelandscape. The knowledge it has collected is used to guide the search process towards a better region in thesearch space. The proposed algorithm consists of two phases. The first phase uses a tabu mechanism to collectinformation about the fitness landscape. In the second phase, the collected data are processed to guide thealgorithm towards better regions in the search space. The algorithm clusters the good solutions it has foundin its previous search process. Then when the population is converged and trapped in a local optimum, itis divided into sub-populations and each sub-population is designated to a cluster. The information in thedatabase is then used to reinitialize the q-individuals, so they represent better regions in the search space.This way the population maintains diversity and by capturing the fitness landscape structure, the algorithmis guided towards better regions in the search space. The algorithm is compared with some state-of-the-artalgorithms from PATAT competition conferences and experimental results are presented.Peer reviewe

Evolutionary Algorithms In Dynamic Environments: Managing Changes Within Generations

Tez (Yüksek Lisans) -- İstanbul Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2007Thesis (M.Sc.) -- İstanbul Technical University, Institute of Science and Technology, 2007Evrimsel Algoritmalar, evrim teorisinin kavramına dayalı olarak statik ve dinamik problemleri çözmek adına geliştirilmiş hesaplama yöntemleridir. Evrimsel Algoritmaların doğadan esinlenen yapısı itibarı ile günümüzdeki değişken problemlere en çok uyum sağlayan algoritmalardır. Dinamik problemlere Evrimsel algoritmalar ile çözüm bulabilmek için değişen ortamın ve değişimin çok detaylı analiz edilmesi gerekmektedir. Yani ortam değiştikten sonra değişim çeşidine ve probleme bağlı olarak en uygun yol seçildiğinde Evrimsel Algoritmaların daha etkin bir şekilde çözüm bulunması sağlanabilir. Günümüzdeki gerçek problemlerde var olan değişimin rastgele olmasına rağmen, bu alandaki çalışmaların hemen hepsi değişimlerin sistematik bir şekilde, yani nesiller arası gerçekleştiğini varsayılarak yapılmıştır. Bu varsayım yeterli olmuş olsa da geliştirilmesi ve araştırılması gereken bir konu olarak karşımıza çıkmaktadır. Bu tezin asıl amacı ortamın nesil içi değiştiğini varsayarak, aşağıdaki yöntemlerden hangisinin daha etkin bir yol olduğunu deneysel sonuçlarla kanıtlamak. Bunlar: - Ortam değiştikten sonra neslin geri kalanını yeni ortama göre hesaplamak ve yeni bireyleri eski bireylerle değerlendirmek - Ortam değiştikten sonra neslin yaşamını durdurmak ve bir sonraki nesli eski nesilden türeterek devam etmek - Ortam değiştikten sonra neslin şimdiye kadar hesaplanan bireylerini tekrar yeni ortamda değerlendirmek - Ortam değiştikten sonra neslin geri kalan bireylerini eski ortama göre hesaplamaya devam etmek ve değişen ortamı bir sonraki nesle uygulamak Yukarıda sıralanan yöntemlerin hangisinin daha etkin bir yol olduğunun bilinmesi gerçek problemler üzerinde, yani değişimin neslin ortasında olduğu durumlarda daha etkin çalışan, zamandan ve kaynaktan tasarruf eden uygun Evrimsel Algoritma geliştirmemize yardımcı olacaktır.Evolutionary algorithms (EAs), based on natural evolutionary theory, are computational methods for static and dynamic problems. Since EAs are naturally inspired, they seem to be more suitable to be applied to dynamic optimization problems. In order to apply EAs to the continuously changing real world problems, changes have to be analyzed in a detailed way. According to the type of change and problem instance, if method of managing changes can be predicted it will improve ability of problem solving of EAs. In nature changes are not happening in an organized way, although almost all researchers’ approach to dynamic changing environment was in that way, thus assuming that changes are happening between generations The main goal of the thesis is to examine and show the ability of methods on changing environments within generations by empirical way. The methods are: - Use the changed fitness function for all subsequent individuals, but keep the evaluations of the offspring already evaluated - Temporarily reduce the population size. The generation is terminated, and the offspring generated so far serve as basis to generate the next. - Re-evaluate all offspring already generated; ignore the change and continue to work with the old fitness function to the end of that generation - Ignore the change and continue to work with the old fitness function until all offspring of that generation have been evaluated. As a result, in real world problems where changes happen at any time, as in design of EAs within generation, knowing the ability of above methods will help us in organizing more effective EAs which are not time consuming as well as resources.Yüksek LisansM.Sc