On Algorithmic Descriptions and Software Implementations for Multi-objective Optimisation: A Comparative Study

Multi-objective optimisation is a prominent subfield of optimisa-tion with high relevance in real-world problems, such as engineering design. Over the past two decades a multitude of heuristic algorithms for multi-objective optimisation have been introduced and some of them have become extremely popular. Some of the most promising and versatile algorithms are have been implemented in software platforms. This article experimentally investigates the process of interpreting and implementing algorithms by examining multiple popular implementations of three well-known algorithms for multi-objective optimisation. We observed that official and broadly employed software platforms interpreted and thus implemented the same heuristic search algorithm differently. These different interpretations affect the algorithmic structure as well as the software implementation. Numerical results show that these differences cause statistically significant differences in performance

Scalarized Preferences in Multi-objective Optimization

Multikriterielle Optimierungsprobleme verfügen über keine Lösung, die optimal in jeder Zielfunktion ist. Die Schwierigkeit solcher Probleme liegt darin eine Kompromisslösung zu finden, die den Präferenzen des Entscheiders genügen, der den Kompromiss implementiert. Skalarisierung – die Abbildung des Vektors der Zielfunktionswerte auf eine reelle Zahl – identifiziert eine einzige Lösung als globales Präferenzenoptimum um diese Probleme zu lösen. Allerdings generieren Skalarisierungsmethoden keine zusätzlichen Informationen über andere Kompromisslösungen, die die Präferenzen des Entscheiders bezüglich des globalen Optimums verändern könnten. Um dieses Problem anzugehen stellt diese Dissertation eine theoretische und algorithmische Analyse skalarisierter Präferenzen bereit. Die theoretische Analyse besteht aus der Entwicklung eines Ordnungsrahmens, der Präferenzen als Problemtransformationen charakterisiert, die präferierte Untermengen der Paretofront definieren. Skalarisierung wird als Transformation der Zielmenge in diesem Ordnungsrahmen dargestellt. Des Weiteren werden Axiome vorgeschlagen, die wünschenswerte Eigenschaften von Skalarisierungsfunktionen darstellen. Es wird gezeigt unter welchen Bedingungen existierende Skalarisierungsfunktionen diese Axiome erfüllen. Die algorithmische Analyse kennzeichnet Präferenzen anhand des Resultats, das ein Optimierungsalgorithmus generiert. Zwei neue Paradigmen werden innerhalb dieser Analyse identifiziert. Für beide Paradigmen werden Algorithmen entworfen, die skalarisierte Präferenzeninformationen verwenden: Präferenzen-verzerrte Paretofrontapproximationen verteilen Punkte über die gesamte Paretofront, fokussieren aber mehr Punkte in Regionen mit besseren Skalarisierungswerten; multimodale Präferenzenoptima sind Punkte, die lokale Skalarisierungsoptima im Zielraum darstellen. Ein Drei-Stufen-Algorith\-mus wird entwickelt, der lokale Skalarisierungsoptima approximiert und verschiedene Methoden werden für die unterschiedlichen Stufen evaluiert. Zwei Realweltprobleme werden vorgestellt, die die Nützlichkeit der beiden Algorithmen illustrieren. Das erste Problem besteht darin Fahrpläne für ein Blockheizkraftwerk zu finden, die die erzeugte Elektrizität und Wärme maximieren und den Kraftstoffverbrauch minimiert. Präferenzen-verzerrte Approximationen generieren mehr Energie-effiziente Lösungen, unter denen der Entscheider seine favorisierte Lösung auswählen kann, indem er die Konflikte zwischen den drei Zielen abwägt. Das zweite Problem beschäftigt sich mit der Erstellung von Fahrplänen für Geräte in einem Wohngebäude, so dass Energiekosten, Kohlenstoffdioxidemissionen und thermisches Unbehagen minimiert werden. Es wird gezeigt, dass lokale Skalarisierungsoptima Fahrpläne darstellen, die eine gute Balance zwischen den drei Zielen bieten. Die Analyse und die Experimente, die in dieser Arbeit vorgestellt werden, ermöglichen es Entscheidern bessere Entscheidungen zu treffen indem Methoden angewendet werden, die mehr Optionen generieren, die mit den Präferenzen der Entscheider übereinstimmen

Dynamic multi-objective optimization using evolutionary algorithms

Dynamic Multi-objective Optimization Problems (DMOPs) offer an opportunity to examine and solve challenging real world scenarios where trade-off solutions between conflicting objectives change over time. Definition of benchmark problems allows modelling of industry scenarios across transport, power and communications networks, manufacturing and logistics. Recently, significant progress has been made in the variety and complexity of DMOP benchmarks and the incorporation of realistic dynamic characteristics. However, significant gaps still exist in standardised methodology for DMOPs, specific problem domain examples and in the understanding of the impacts and explanations of dynamic characteristics. This thesis provides major contributions on these three topics within evolutionary dynamic multi-objective optimization. Firstly, experimental protocols for DMOPs are varied. This limits the applicability and relevance of results produced and conclusions made in the field. A major source of the inconsistency lies in the parameters used to define specific problem instances being examined. The uninformed selection of these has historically held back understanding of their impacts and standardisation in experimental approach to these parameters in the multi-objective problem domain. Using the frequency and severity (or magnitude) of change events, a more informed approach to DMOP experimentation is conceptualized, implemented and evaluated. Establishment of a baseline performance expectation across a comprehensive range of dynamic instances for well-studied DMOP benchmarks is analyzed. To maximize relevance, these profiles are composed from the performance of evolutionary algorithms commonly used for baseline comparisons and those with simple dynamic responses. Comparison and contrast with the coverage of parameter combinations in the sampled literature highlights the importance of these contributions. Secondly, the provision of useful and realistic DMOPs in the combinatorial domain is limited in previous literature. A novel dynamic benchmark problem is presented by the extension of the Travelling Thief Problem (TTP) to include a variety of realistic and contextually justified dynamic changes. Investigation of problem information exploitation and it's potential application as a dynamic response is a key output of these results and context is provided through comparison to results obtained by adapting existing TTP heuristics. Observation driven iterative development prompted the investigation of multi-population island model strategies, together with improvements in the approaches to accurately describe and compare the performance of algorithm models for DMOPs, a contribution which is applicable beyond the dynamic TTP. Thirdly, the purpose of DMOPs is to reconstruct realistic scenarios, or features from them, to allow for experimentation and development of better optimization algorithms. However, numerous important characteristics from real systems still require implementation and will drive research and development of algorithms and mechanisms to handle these industrially relevant problem classes. The novel challenges associated with these implementations are significant and diverse, even for a simple development such as consideration of DMOPs with multiple time dependencies. Real world systems with dynamics are likely to contain multiple temporally changing aspects, particularly in energy and transport domains. Problems with more than one dynamic problem component allow for asynchronous changes and a differing severity between components that leads to an explosion in the size of the possible dynamic instance space. Both continuous and combinatorial problem domains require structured investigation into the best practices for experimental design, algorithm application and performance measurement, comparison and visualization. Highlighting the challenges, the key requirements for effective progress and recommendations on experimentation are explored here

Effective and Efficient Evolutionary Many-Objective Optimization

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonMany-objective optimization is core to both artificial intelligence and data analytics as real-world problems commonly involve multiple objectives which are required to be optimized simultaneously. A large number of evolutionary algorithms have been developed to search for a set of Pareto optimal solutions for many-objective optimization problems. It is very rare that a many-objective evolutionary algorithm performs well in terms of both effectiveness and efficiency, two key evaluation criteria. Some algorithms may struggle to guide the solutions towards the Pareto front, e.g., Pareto-based algorithms, while other algorithms may have difficulty in diversifying the solutions evenly over the front on certain problems, e.g., decomposition-based algorithms. Furthermore, some effective algorithms may become very computationally expensive as the number of objectives increases, e.g., indicator-based algorithms. The aim of this thesis is to investigate how to make evolutionary algorithms perform well in terms of effectiveness and efficiency in many-objective optimization. After conducting a review of key concepts and the state of the art in the evolutionary many-objective optimization, this thesis shows how to improve the effectiveness of conventional Pareto-based algorithms on a challenging real-world problem in software engineering. This thesis then explores how to further enhance the effectiveness of leading many-objective evolutionary algorithms in general by extending the capability of a very popular and widely cited bi-goal evolution method. Last but not least, this thesis investigates how to strike a balance between effectiveness and efficiency of evolutionary algorithms when solving many-objective optimization problems. The work reported is based on either real-world or recognized synthetic datasets, and the proposed algorithms are compared and evaluated against leading algorithms in the field. The work does not only demonstrate ways of improving the effectiveness and efficiency of many-objective optimization algorithms but also led to promising areas for future research

Multiple Objective Evolutionary Algorithms for Independent, Computationally Expensive Objectives

This research augments current Multiple Objective Evolutionary Algorithms with methods that dramatically reduce the time required to evolve toward a region of interest in objective space. Multiple Objective Evolutionary Algorithms (MOEAs) are superior to other optimization techniques when the search space is of high dimension and contains many local minima and maxima. Likewise, MOEAs are most interesting when applied to non-intuitive complex systems. But, these systems are often computationally expensive to calculate. When these systems require independent computations to evaluate each objective, the computational expense grows with each additional objective. This method has developed methods that reduces the time required for evolution by reducing the number of objective evaluations, while still evolving solutions that are Pareto optimal. To date, all other Multiple Objective Evolutionary Algorithms (MOEAs) require the evaluation of all objectives before a fitness value can be assigned to an individual. The original contributions of this thesis are: 1. Development of a hierarchical search space description that allows association of crossover and mutation settings with elements of the genotypic description. 2. Development of a method for parallel evaluation of individuals that removes the need for delays for synchronization. 3. Dynamical evolution of thresholds for objectives to allow partial evaluation of objectives for individuals. 4. Dynamic objective orderings to minimize the time required for unnecessary objective evaluations. 5. Application of MOEAs to the computationally expensive flare pattern design domain. 6. Application of MOEAs to the optimization of fielded missile warning receiver algorithms. 7. Development of a new method of using MOEAs for automatic design of pattern recognition systems.Ph.D.Committee Chair: Dr. Mark A. Clements; Committee Member: Dr, Mark A. Richards; Committee Member: Dr. Darrell R. Lamm; Committee Member: Dr. Ellis Johnson; Committee Member: Dr. James H. McClellan; Committee Member: Dr. James O. Hamble

Heuristics and metaheuristics in the design of sound-absorbing porous materials

Inexact optimisation techniques such as heuristics and metaheuristics that quickly find near-optimal solutions are widely used to solve hard problems. While metaheuristics are well studied on specific problem domains such as travelling salesman, timetabling, vehicle routing etc., their extension to engineering domains is largely unexplored due to the requirement of domain expertise. In this thesis, we address a specific engineering domain: the design of sound-absorbing porous materials. Porous materials are foams, fibrous materials, woven and non-woven textiles, etc., that are widely used in automotive, aerospace and household applications to isolate and absorb noise to prevent equipment damage, protect hearing or ensure comfort. These materials constitute a significant amount of dead weight in aircraft and space applications, and choosing sub-optimal designs would lead to inefficiency and increased costs. By carefully choosing the material properties and shapes of these materials, favourable resonances can be created making it possible to improve absorption while also reducing weight. The optimisation problem structure is yet to be well-explored and not many comparison studies are available in this domain. This thesis aims to address the knowledge gap by analysing the performance of existing and novel heuristic and metaheuristic methods. Initially, the problem structure is explored by considering a one-dimensional layered sound package problem. Then, the challenging two-dimensional foam shape and topology optimisation is addressed. Topology optimisation involves optimally distributing a given volume of material in a design region such that a performance measure is maximised. Although extensive studies exist for the compliance minimisation problem domain, studies and comparisons on porous material problems are relatively rare. Firstly, a single objective absorption maximisation problem with a constraint on the weight is considered. Then a multi-objective problem of simultaneously maximising absorption and minimising weight is considered. The unique nature of the topology optimisation problem allows it to be solved using combinatorial or continuous, gradient or non-gradient methods. In this work, several optimisation methods are studied, including solid isotropic material with penalisation (SIMP), hill climbing, constructive heuristics, genetic algorithms, tabu search, co-variance matrix adaptation evolution strategy (CMA-ES), differential evolution, non-dominated sorting genetic algorithm (NSGA-II) and hybrid strategies. These approaches are tested on a benchmark of seven acoustics problem instances. The results are used to extract domain-specific insights. The findings highlight that the problem domain is rich with unique varieties of solutions, and by using domain-specific insights, one can design hybrid gradient and non-gradient methods that consistently outperform state-of-the-art ones