Multiple Objective Evolutionary Algorithms for Independent, Computationally Expensive Objectives

This research augments current Multiple Objective Evolutionary Algorithms with methods that dramatically reduce the time required to evolve toward a region of interest in objective space. Multiple Objective Evolutionary Algorithms (MOEAs) are superior to other optimization techniques when the search space is of high dimension and contains many local minima and maxima. Likewise, MOEAs are most interesting when applied to non-intuitive complex systems. But, these systems are often computationally expensive to calculate. When these systems require independent computations to evaluate each objective, the computational expense grows with each additional objective. This method has developed methods that reduces the time required for evolution by reducing the number of objective evaluations, while still evolving solutions that are Pareto optimal. To date, all other Multiple Objective Evolutionary Algorithms (MOEAs) require the evaluation of all objectives before a fitness value can be assigned to an individual. The original contributions of this thesis are: 1. Development of a hierarchical search space description that allows association of crossover and mutation settings with elements of the genotypic description. 2. Development of a method for parallel evaluation of individuals that removes the need for delays for synchronization. 3. Dynamical evolution of thresholds for objectives to allow partial evaluation of objectives for individuals. 4. Dynamic objective orderings to minimize the time required for unnecessary objective evaluations. 5. Application of MOEAs to the computationally expensive flare pattern design domain. 6. Application of MOEAs to the optimization of fielded missile warning receiver algorithms. 7. Development of a new method of using MOEAs for automatic design of pattern recognition systems.Ph.D.Committee Chair: Dr. Mark A. Clements; Committee Member: Dr, Mark A. Richards; Committee Member: Dr. Darrell R. Lamm; Committee Member: Dr. Ellis Johnson; Committee Member: Dr. James H. McClellan; Committee Member: Dr. James O. Hamble

Scalarized Preferences in Multi-objective Optimization

Multikriterielle Optimierungsprobleme verfügen über keine Lösung, die optimal in jeder Zielfunktion ist. Die Schwierigkeit solcher Probleme liegt darin eine Kompromisslösung zu finden, die den Präferenzen des Entscheiders genügen, der den Kompromiss implementiert. Skalarisierung – die Abbildung des Vektors der Zielfunktionswerte auf eine reelle Zahl – identifiziert eine einzige Lösung als globales Präferenzenoptimum um diese Probleme zu lösen. Allerdings generieren Skalarisierungsmethoden keine zusätzlichen Informationen über andere Kompromisslösungen, die die Präferenzen des Entscheiders bezüglich des globalen Optimums verändern könnten. Um dieses Problem anzugehen stellt diese Dissertation eine theoretische und algorithmische Analyse skalarisierter Präferenzen bereit. Die theoretische Analyse besteht aus der Entwicklung eines Ordnungsrahmens, der Präferenzen als Problemtransformationen charakterisiert, die präferierte Untermengen der Paretofront definieren. Skalarisierung wird als Transformation der Zielmenge in diesem Ordnungsrahmen dargestellt. Des Weiteren werden Axiome vorgeschlagen, die wünschenswerte Eigenschaften von Skalarisierungsfunktionen darstellen. Es wird gezeigt unter welchen Bedingungen existierende Skalarisierungsfunktionen diese Axiome erfüllen. Die algorithmische Analyse kennzeichnet Präferenzen anhand des Resultats, das ein Optimierungsalgorithmus generiert. Zwei neue Paradigmen werden innerhalb dieser Analyse identifiziert. Für beide Paradigmen werden Algorithmen entworfen, die skalarisierte Präferenzeninformationen verwenden: Präferenzen-verzerrte Paretofrontapproximationen verteilen Punkte über die gesamte Paretofront, fokussieren aber mehr Punkte in Regionen mit besseren Skalarisierungswerten; multimodale Präferenzenoptima sind Punkte, die lokale Skalarisierungsoptima im Zielraum darstellen. Ein Drei-Stufen-Algorith\-mus wird entwickelt, der lokale Skalarisierungsoptima approximiert und verschiedene Methoden werden für die unterschiedlichen Stufen evaluiert. Zwei Realweltprobleme werden vorgestellt, die die Nützlichkeit der beiden Algorithmen illustrieren. Das erste Problem besteht darin Fahrpläne für ein Blockheizkraftwerk zu finden, die die erzeugte Elektrizität und Wärme maximieren und den Kraftstoffverbrauch minimiert. Präferenzen-verzerrte Approximationen generieren mehr Energie-effiziente Lösungen, unter denen der Entscheider seine favorisierte Lösung auswählen kann, indem er die Konflikte zwischen den drei Zielen abwägt. Das zweite Problem beschäftigt sich mit der Erstellung von Fahrplänen für Geräte in einem Wohngebäude, so dass Energiekosten, Kohlenstoffdioxidemissionen und thermisches Unbehagen minimiert werden. Es wird gezeigt, dass lokale Skalarisierungsoptima Fahrpläne darstellen, die eine gute Balance zwischen den drei Zielen bieten. Die Analyse und die Experimente, die in dieser Arbeit vorgestellt werden, ermöglichen es Entscheidern bessere Entscheidungen zu treffen indem Methoden angewendet werden, die mehr Optionen generieren, die mit den Präferenzen der Entscheider übereinstimmen

Heuristics and metaheuristics in the design of sound-absorbing porous materials

Inexact optimisation techniques such as heuristics and metaheuristics that quickly find near-optimal solutions are widely used to solve hard problems. While metaheuristics are well studied on specific problem domains such as travelling salesman, timetabling, vehicle routing etc., their extension to engineering domains is largely unexplored due to the requirement of domain expertise. In this thesis, we address a specific engineering domain: the design of sound-absorbing porous materials. Porous materials are foams, fibrous materials, woven and non-woven textiles, etc., that are widely used in automotive, aerospace and household applications to isolate and absorb noise to prevent equipment damage, protect hearing or ensure comfort. These materials constitute a significant amount of dead weight in aircraft and space applications, and choosing sub-optimal designs would lead to inefficiency and increased costs. By carefully choosing the material properties and shapes of these materials, favourable resonances can be created making it possible to improve absorption while also reducing weight. The optimisation problem structure is yet to be well-explored and not many comparison studies are available in this domain. This thesis aims to address the knowledge gap by analysing the performance of existing and novel heuristic and metaheuristic methods. Initially, the problem structure is explored by considering a one-dimensional layered sound package problem. Then, the challenging two-dimensional foam shape and topology optimisation is addressed. Topology optimisation involves optimally distributing a given volume of material in a design region such that a performance measure is maximised. Although extensive studies exist for the compliance minimisation problem domain, studies and comparisons on porous material problems are relatively rare. Firstly, a single objective absorption maximisation problem with a constraint on the weight is considered. Then a multi-objective problem of simultaneously maximising absorption and minimising weight is considered. The unique nature of the topology optimisation problem allows it to be solved using combinatorial or continuous, gradient or non-gradient methods. In this work, several optimisation methods are studied, including solid isotropic material with penalisation (SIMP), hill climbing, constructive heuristics, genetic algorithms, tabu search, co-variance matrix adaptation evolution strategy (CMA-ES), differential evolution, non-dominated sorting genetic algorithm (NSGA-II) and hybrid strategies. These approaches are tested on a benchmark of seven acoustics problem instances. The results are used to extract domain-specific insights. The findings highlight that the problem domain is rich with unique varieties of solutions, and by using domain-specific insights, one can design hybrid gradient and non-gradient methods that consistently outperform state-of-the-art ones

Development of a multi-objective optimization algorithm based on lichtenberg figures

This doctoral dissertation presents the most important concepts of multi-objective optimization and a systematic review of the most cited articles in the last years of this subject in mechanical engineering. The State of the Art shows a trend towards the use of metaheuristics and the use of a posteriori decision-making techniques to solve engineering problems. This fact increases the demand for algorithms, which compete to deliver the most accurate answers at the lowest possible computational cost. In this context, a new hybrid multi-objective metaheuristic inspired by lightning and Linchtenberg Figures is proposed. The Multi-objective Lichtenberg Algorithm (MOLA) is tested using complex test functions and explicit contrainted engineering problems and compared with other metaheuristics. MOLA outperformed the most used algorithms in the literature: NSGA-II, MOPSO, MOEA/D, MOGWO, and MOGOA. After initial validation, it was applied to two complex and impossible to be analytically evaluated problems. The first was a design case: the multi-objective optimization of CFRP isogrid tubes using the finite element method. The optimizations were made considering two methodologies: i) using a metamodel, and ii) the finite element updating. The last proved to be the best methodology, finding solutions that reduced at least 45.69% of the mass, 18.4% of the instability coefficient, 61.76% of the Tsai-Wu failure index and increased by at least 52.57% the natural frequency. In the second application, MOLA was internally modified and associated with feature selection techniques to become the Multi-objective Sensor Selection and Placement Optimization based on the Lichtenberg Algorithm (MOSSPOLA), an unprecedented Sensor Placement Optimization (SPO) algorithm that maximizes the acquired modal response and minimizes the number of sensors for any structure. Although this is a structural health monitoring principle, it has never been done before. MOSSPOLA was applied to a real helicopter’s main rotor blade using the 7 best-known metrics in SPO. Pareto fronts and sensor configurations were unprecedentedly generated and compared. Better sensor distributions were associated with higher hypervolume and the algorithm found a sensor configuration for each sensor number and metric, including one with 100% accuracy in identifying delamination considering triaxial modal displacements, minimum number of sensors, and noise for all blade sections.Esta tese de doutorado traz os conceitos mais importantes de otimização multi-objetivo e uma revisão sistemática dos artigos mais citados nos últimos anos deste tema em engenharia mecânica. O estado da arte mostra uma tendência no uso de meta-heurísticas e de técnicas de tomada de decisão a posteriori para resolver problemas de engenharia. Este fato aumenta a demanda sobre os algoritmos, que competem para entregar respostas mais precisas com o menor custo computacional possível. Nesse contexto, é proposta uma nova meta-heurística híbrida multi-objetivo inspirada em raios e Figuras de Lichtenberg. O Algoritmo de Lichtenberg Multi-objetivo (MOLA) é testado e comparado com outras metaheurísticas usando funções de teste complexas e problemas restritos e explícitos de engenharia. Ele superou os algoritmos mais utilizados na literatura: NSGA-II, MOPSO, MOEA/D, MOGWO e MOGOA. Após validação, foi aplicado em dois problemas complexos e impossíveis de serem analiticamente otimizados. O primeiro foi um caso de projeto: otimização multi-objetivo de tubos isogrid CFRP usando o método dos elementos finitos. As otimizações foram feitas considerando duas metodologias: i) usando um meta-modelo, e ii) atualização por elementos finitos. A última provou ser a melhor metodologia, encontrando soluções que reduziram pelo menos 45,69% da massa, 18,4% do coeficiente de instabilidade, 61,76% do TW e aumentaram em pelo menos 52,57% a frequência natural. Na segunda aplicação, MOLA foi modificado internamente e associado a técnicas de feature selection para se tornar o Seleção e Alocação ótima de Sensores Multi-objetivo baseado no Algoritmo de Lichtenberg (MOSSPOLA), um algoritmo inédito de Otimização de Posicionamento de Sensores (SPO) que maximiza a resposta modal adquirida e minimiza o número de sensores para qualquer estrutura. Embora isto seja um princípio de Monitoramento da Saúde Estrutural, nunca foi feito antes. O MOSSPOLA foi aplicado na pá do rotor principal de um helicóptero real usando as 7 métricas mais conhecidas em SPO. Frentes de Pareto e configurações de sensores foram ineditamente geradas e comparadas. Melhores distribuições de sensores foram associadas a um alto hipervolume e o algoritmo encontrou uma configuração de sensor para cada número de sensores e métrica, incluindo uma com 100% de precisão na identificação de delaminação considerando deslocamentos modais triaxiais, número mínimo de sensores e ruído para todas as seções da lâmina

Heuristics and metaheuristics in the design of sound-absorbing porous materials

Inexact optimisation techniques such as heuristics and metaheuristics that quickly find near-optimal solutions are widely used to solve hard problems. While metaheuristics are well studied on specific problem domains such as travelling salesman, timetabling, vehicle routing etc., their extension to engineering domains is largely unexplored due to the requirement of domain expertise. In this thesis, we address a specific engineering domain: the design of sound-absorbing porous materials. Porous materials are foams, fibrous materials, woven and non-woven textiles, etc., that are widely used in automotive, aerospace and household applications to isolate and absorb noise to prevent equipment damage, protect hearing or ensure comfort. These materials constitute a significant amount of dead weight in aircraft and space applications, and choosing sub-optimal designs would lead to inefficiency and increased costs. By carefully choosing the material properties and shapes of these materials, favourable resonances can be created making it possible to improve absorption while also reducing weight. The optimisation problem structure is yet to be well-explored and not many comparison studies are available in this domain. This thesis aims to address the knowledge gap by analysing the performance of existing and novel heuristic and metaheuristic methods. Initially, the problem structure is explored by considering a one-dimensional layered sound package problem. Then, the challenging two-dimensional foam shape and topology optimisation is addressed. Topology optimisation involves optimally distributing a given volume of material in a design region such that a performance measure is maximised. Although extensive studies exist for the compliance minimisation problem domain, studies and comparisons on porous material problems are relatively rare. Firstly, a single objective absorption maximisation problem with a constraint on the weight is considered. Then a multi-objective problem of simultaneously maximising absorption and minimising weight is considered. The unique nature of the topology optimisation problem allows it to be solved using combinatorial or continuous, gradient or non-gradient methods. In this work, several optimisation methods are studied, including solid isotropic material with penalisation (SIMP), hill climbing, constructive heuristics, genetic algorithms, tabu search, co-variance matrix adaptation evolution strategy (CMA-ES), differential evolution, non-dominated sorting genetic algorithm (NSGA-II) and hybrid strategies. These approaches are tested on a benchmark of seven acoustics problem instances. The results are used to extract domain-specific insights. The findings highlight that the problem domain is rich with unique varieties of solutions, and by using domain-specific insights, one can design hybrid gradient and non-gradient methods that consistently outperform state-of-the-art ones

Dynamic multi-objective optimization using evolutionary algorithms

Dynamic Multi-objective Optimization Problems (DMOPs) offer an opportunity to examine and solve challenging real world scenarios where trade-off solutions between conflicting objectives change over time. Definition of benchmark problems allows modelling of industry scenarios across transport, power and communications networks, manufacturing and logistics. Recently, significant progress has been made in the variety and complexity of DMOP benchmarks and the incorporation of realistic dynamic characteristics. However, significant gaps still exist in standardised methodology for DMOPs, specific problem domain examples and in the understanding of the impacts and explanations of dynamic characteristics. This thesis provides major contributions on these three topics within evolutionary dynamic multi-objective optimization. Firstly, experimental protocols for DMOPs are varied. This limits the applicability and relevance of results produced and conclusions made in the field. A major source of the inconsistency lies in the parameters used to define specific problem instances being examined. The uninformed selection of these has historically held back understanding of their impacts and standardisation in experimental approach to these parameters in the multi-objective problem domain. Using the frequency and severity (or magnitude) of change events, a more informed approach to DMOP experimentation is conceptualized, implemented and evaluated. Establishment of a baseline performance expectation across a comprehensive range of dynamic instances for well-studied DMOP benchmarks is analyzed. To maximize relevance, these profiles are composed from the performance of evolutionary algorithms commonly used for baseline comparisons and those with simple dynamic responses. Comparison and contrast with the coverage of parameter combinations in the sampled literature highlights the importance of these contributions. Secondly, the provision of useful and realistic DMOPs in the combinatorial domain is limited in previous literature. A novel dynamic benchmark problem is presented by the extension of the Travelling Thief Problem (TTP) to include a variety of realistic and contextually justified dynamic changes. Investigation of problem information exploitation and it's potential application as a dynamic response is a key output of these results and context is provided through comparison to results obtained by adapting existing TTP heuristics. Observation driven iterative development prompted the investigation of multi-population island model strategies, together with improvements in the approaches to accurately describe and compare the performance of algorithm models for DMOPs, a contribution which is applicable beyond the dynamic TTP. Thirdly, the purpose of DMOPs is to reconstruct realistic scenarios, or features from them, to allow for experimentation and development of better optimization algorithms. However, numerous important characteristics from real systems still require implementation and will drive research and development of algorithms and mechanisms to handle these industrially relevant problem classes. The novel challenges associated with these implementations are significant and diverse, even for a simple development such as consideration of DMOPs with multiple time dependencies. Real world systems with dynamics are likely to contain multiple temporally changing aspects, particularly in energy and transport domains. Problems with more than one dynamic problem component allow for asynchronous changes and a differing severity between components that leads to an explosion in the size of the possible dynamic instance space. Both continuous and combinatorial problem domains require structured investigation into the best practices for experimental design, algorithm application and performance measurement, comparison and visualization. Highlighting the challenges, the key requirements for effective progress and recommendations on experimentation are explored here

Regularized model learning in EDAs for continuous and multi-objective optimization

Probabilistic modeling is the de�ning characteristic of estimation of distribution algorithms (EDAs) which determines their behavior and performance in optimization. Regularization is a well-known statistical technique used for obtaining an improved model by reducing the generalization error of estimation, especially in high-dimensional problems. `1-regularization is a type of this technique with the appealing variable selection property which results in sparse model estimations. In this thesis, we study the use of regularization techniques for model learning in EDAs. Several methods for regularized model estimation in continuous domains based on a Gaussian distribution assumption are presented, and analyzed from di�erent aspects when used for optimization in a high-dimensional setting, where the population size of EDA has a logarithmic scale with respect to the number of variables. The optimization results obtained for a number of continuous problems with an increasing number of variables show that the proposed EDA based on regularized model estimation performs a more robust optimization, and is able to achieve signi�cantly better results for larger dimensions than other Gaussian-based EDAs. We also propose a method for learning a marginally factorized Gaussian Markov random �eld model using regularization techniques and a clustering algorithm. The experimental results show notable optimization performance on continuous additively decomposable problems when using this model estimation method. Our study also covers multi-objective optimization and we propose joint probabilistic modeling of variables and objectives in EDAs based on Bayesian networks, speci�cally models inspired from multi-dimensional Bayesian network classi�ers. It is shown that with this approach to modeling, two new types of relationships are encoded in the estimated models in addition to the variable relationships captured in other EDAs: objectivevariable and objective-objective relationships. An extensive experimental study shows the e�ectiveness of this approach for multi- and many-objective optimization. With the proposed joint variable-objective modeling, in addition to the Pareto set approximation, the algorithm is also able to obtain an estimation of the multi-objective problem structure. Finally, the study of multi-objective optimization based on joint probabilistic modeling is extended to noisy domains, where the noise in objective values is represented by intervals. A new version of the Pareto dominance relation for ordering the solutions in these problems, namely �-degree Pareto dominance, is introduced and its properties are analyzed. We show that the ranking methods based on this dominance relation can result in competitive performance of EDAs with respect to the quality of the approximated Pareto sets. This dominance relation is then used together with a method for joint probabilistic modeling based on `1-regularization for multi-objective feature subset selection in classi�cation, where six di�erent measures of accuracy are considered as objectives with interval values. The individual assessment of the proposed joint probabilistic modeling and solution ranking methods on datasets with small-medium dimensionality, when using two di�erent Bayesian classi�ers, shows that comparable or better Pareto sets of feature subsets are approximated in comparison to standard methods

Evolutionary approaches for portfolio optimization

Portfolio optimization involves the optimal assignment of limited capital to different available financial assets to achieve a reasonable trade-off between profit and risk objectives. Markowitz’s mean variance (MV) model is widely regarded as the foundation of modern portfolio theory and provides a quantitative framework for portfolio optimization problems. In real market, investors commonly face real-world trading restrictions and it requires that the constructed portfolios have to meet trading constraints. When additional constraints are added to the basic MV model, the problem thus becomes more complex and the exact optimization approaches run into difficulties to deliver solutions within reasonable time for large problem size. By introducing the cardinality constraint alone already transformed the classic quadratic optimization model into a mixed-integer quadratic programming problem which is an NP-hard problem. Evolutionary algorithms, a class of metaheuristics, are one of the known alternatives for optimization problems that are too complex to be solved using deterministic techniques. This thesis focuses on single-period portfolio optimization problems with practical trading constraints and two different risk measures. Four hybrid evolutionary algorithms are presented to efficiently solve these problems with gradually more complex real world constraints. In the first part of the thesis, the mean variance portfolio model is investigated by taking into account real-world constraints. A hybrid evolutionary algorithm (PBILDE) for portfolio optimization with cardinality and quantity constraints is presented. The proposed PBILDE is able to achieve a strong synergetic effect through hybridization of PBIL and DE. A partially guided mutation and an elitist update strategy are proposed in order to promote the efficient convergence of PBILDE. Its effectiveness is evaluated and compared with other existing algorithms over a number of datasets. A multi-objective scatter search with archive (MOSSwA) algorithm for portfolio optimization with cardinality, quantity and pre-assignment constraints is then presented. New subset generations and solution combination methods are proposed to generate efficient and diverse portfolios. A learning-guided multi-objective evolutionary (MODEwAwL) algorithm for the portfolio optimization problems with cardinality, quantity, pre-assignment and round lot constraints is presented. A learning mechanism is introduced in order to extract important features from the set of elite solutions. Problem-specific selection heuristics are introduced in order to identify high-quality solutions with a reduced computational cost. An efficient and effective candidate generation scheme utilizing a learning mechanism, problem specific heuristics and effective direction-based search methods is proposed to guide the search towards the promising regions of the search space. In the second part of the thesis, an alternative risk measure, VaR, is considered. A non-parametric mean-VaR model with six practical trading constraints is investigated. A multi-objective evolutionary algorithm with guided learning (MODE-GL) is presented for the mean-VaR model. Two different variants of DE mutation schemes in the solution generation scheme are proposed in order to promote the exploration of the search towards the least crowded region of the solution space. Experimental results using historical daily financial market data from S &P 100 and S & P 500 indices are presented. When the cardinality constraints are considered, incorporating a learning mechanism significantly promotes the efficient convergence of the search

Evolutionary approaches for portfolio optimization

Portfolio optimization involves the optimal assignment of limited capital to different available financial assets to achieve a reasonable trade-off between profit and risk objectives. Markowitz’s mean variance (MV) model is widely regarded as the foundation of modern portfolio theory and provides a quantitative framework for portfolio optimization problems. In real market, investors commonly face real-world trading restrictions and it requires that the constructed portfolios have to meet trading constraints. When additional constraints are added to the basic MV model, the problem thus becomes more complex and the exact optimization approaches run into difficulties to deliver solutions within reasonable time for large problem size. By introducing the cardinality constraint alone already transformed the classic quadratic optimization model into a mixed-integer quadratic programming problem which is an NP-hard problem. Evolutionary algorithms, a class of metaheuristics, are one of the known alternatives for optimization problems that are too complex to be solved using deterministic techniques. This thesis focuses on single-period portfolio optimization problems with practical trading constraints and two different risk measures. Four hybrid evolutionary algorithms are presented to efficiently solve these problems with gradually more complex real world constraints. In the first part of the thesis, the mean variance portfolio model is investigated by taking into account real-world constraints. A hybrid evolutionary algorithm (PBILDE) for portfolio optimization with cardinality and quantity constraints is presented. The proposed PBILDE is able to achieve a strong synergetic effect through hybridization of PBIL and DE. A partially guided mutation and an elitist update strategy are proposed in order to promote the efficient convergence of PBILDE. Its effectiveness is evaluated and compared with other existing algorithms over a number of datasets. A multi-objective scatter search with archive (MOSSwA) algorithm for portfolio optimization with cardinality, quantity and pre-assignment constraints is then presented. New subset generations and solution combination methods are proposed to generate efficient and diverse portfolios. A learning-guided multi-objective evolutionary (MODEwAwL) algorithm for the portfolio optimization problems with cardinality, quantity, pre-assignment and round lot constraints is presented. A learning mechanism is introduced in order to extract important features from the set of elite solutions. Problem-specific selection heuristics are introduced in order to identify high-quality solutions with a reduced computational cost. An efficient and effective candidate generation scheme utilizing a learning mechanism, problem specific heuristics and effective direction-based search methods is proposed to guide the search towards the promising regions of the search space. In the second part of the thesis, an alternative risk measure, VaR, is considered. A non-parametric mean-VaR model with six practical trading constraints is investigated. A multi-objective evolutionary algorithm with guided learning (MODE-GL) is presented for the mean-VaR model. Two different variants of DE mutation schemes in the solution generation scheme are proposed in order to promote the exploration of the search towards the least crowded region of the solution space. Experimental results using historical daily financial market data from S &P 100 and S & P 500 indices are presented. When the cardinality constraints are considered, incorporating a learning mechanism significantly promotes the efficient convergence of the search