Using Optimality Theory and Reference Points to Improve the Diversity and Convergence of a Fuzzy-Adaptive Multi-Objective Particle Swarm Optimizer

Particle Swarm Optimization (PSO) has received increasing attention from the evolutionary optimization research community in the last twenty years. PSO is a metaheuristic approach based on collective intelligence obtained by emulating the swarming behavior of bees. A number of multi-objective variants of the original PSO algorithm that extend its applicability to optimization problems with conflicting objectives have also been developed; these multi-objective PSO (MOPSO) algorithms demonstrate comparable performance to other state-of-the-art metaheuristics. The existence of multiple optimal solutions (Pareto-optimal set) in optimization problems with conflicting objectives is not the only challenge posed to an optimizer, as the latter needs to be able to identify and preserve a well-distributed set of solutions during the search of the decision variable space. Recent attempts by evolutionary optimization researchers to incorporate mathematical convergence conditions into genetic algorithm optimizers have led to the derivation of a point-wise proximity measure, which is based on the solution of the achievement scalarizing function (ASF) optimization problem with a complementary slackness condition that quantifies the violation of the Karush-Kuhn-Tucker necessary conditions of optimality. In this work, the aforementioned KKT proximity measure is incorporated into the original Adaptive Coevolutionary Multi-Objective Swarm Optimizer (ACMOPSO) in order to monitor the convergence of the sub-swarms towards the Pareto-optimal front and provide feedback to Mamdani-type fuzzy logic controllers (FLCs) that are utilized for online adaptation of the algorithmic parameters. The proposed Fuzzy-Adaptive Multi-Objective Optimization Algorithm with the KKT proximity measure (FAMOPSOkkt) utilizes a set of reference points to cluster the computed nondominated solutions. These clusters interact with their corresponding sub-swarms to provide the swarm leaders and are also utilized to manage the external archive of nondominated solutions. The performance of the proposed algorithm is evaluated on benchmark problems chosen from the multi-objective optimization literature and compared to the performance of state-of-the-art multi-objective optimization algorithms with similar features

Multi-objective routing optimization using evolutionary algorithms

Wireless ad hoc networks suffer from several limitations, such as routing failures, potentially excessive bandwidth requirements, computational constraints and limited storage capability. Their routing strategy plays a significant role in determining the overall performance of the multi-hop network. However, in conventional network design only one of the desired routing-related objectives is optimized, while other objectives are typically assumed to be the constraints imposed on the problem. In this paper, we invoke the Non-dominated Sorting based Genetic Algorithm-II (NSGA-II) and the MultiObjective Differential Evolution (MODE) algorithm for finding optimal routes from a given source to a given destination in the face of conflicting design objectives, such as the dissipated energy and the end-to-end delay in a fully-connected arbitrary multi-hop network. Our simulation results show that both the NSGA-II and MODE algorithms are efficient in solving these routing problems and are capable of finding the Pareto-optimal solutions at lower complexity than the ’brute-force’ exhaustive search, when the number of nodes is higher than or equal to 10. Additionally, we demonstrate that at the same complexity, the MODE algorithm is capable of finding solutions closer to the Pareto front and typically, converges faster than the NSGA-II algorithm

A Hierachical Evolutionary Algorithm for Multiobjective Optimization in IMRT

Purpose: Current inverse planning methods for IMRT are limited because they are not designed to explore the trade-offs between the competing objectives between the tumor and normal tissues. Our goal was to develop an efficient multiobjective optimization algorithm that was flexible enough to handle any form of objective function and that resulted in a set of Pareto optimal plans. Methods: We developed a hierarchical evolutionary multiobjective algorithm designed to quickly generate a diverse Pareto optimal set of IMRT plans that meet all clinical constraints and reflect the trade-offs in the plans. The top level of the hierarchical algorithm is a multiobjective evolutionary algorithm (MOEA). The genes of the individuals generated in the MOEA are the parameters that define the penalty function minimized during an accelerated deterministic IMRT optimization that represents the bottom level of the hierarchy. The MOEA incorporates clinical criteria to restrict the search space through protocol objectives and then uses Pareto optimality among the fitness objectives to select individuals. Results: Acceleration techniques implemented on both levels of the hierarchical algorithm resulted in short, practical runtimes for optimizations. The MOEA improvements were evaluated for example prostate cases with one target and two OARs. The modified MOEA dominated 11.3% of plans using a standard genetic algorithm package. By implementing domination advantage and protocol objectives, small diverse populations of clinically acceptable plans that were only dominated 0.2% by the Pareto front could be generated in a fraction of an hour. Conclusions: Our MOEA produces a diverse Pareto optimal set of plans that meet all dosimetric protocol criteria in a feasible amount of time. It optimizes not only beamlet intensities but also objective function parameters on a patient-specific basis

Proximity measures based on KKT points for constrained multi-objective optimization

An important aspect of optimization algorithms, for instance evolutionary algorithms, are termination criteria that measure the proximity of the found solution to the optimal solution set. A frequently used approach is the numerical verification of necessary optimality conditions such as the Karush-Kuhn-Tucker (KKT) conditions. In this paper, we present a proximity measure which characterizes the violation of the KKT conditions. It can be computed easily and is continuous in every efficient solution. Hence, it can be used as an indicator for the proximity of a certain point to the set of efficient (Edgeworth-Pareto-minimal) solutions and is well suited for algorithmic use due to its continuity properties. This is especially useful within evolutionary algorithms for candidate selection and termination, which we also illustrate numerically for some test problems

Bundle methods in nonsmooth DC optimization

Due to the complexity of many practical applications, we encounter optimization problems with nonsmooth functions, that is, functions which are not continuously differentiable everywhere. Classical gradient-based methods are not applicable to solve such problems, since they may fail in the nonsmooth setting. Therefore, it is imperative to develop numerical methods specifically designed for nonsmooth optimization. To date, bundle methods are considered to be the most efficient and reliable general purpose solvers for this type of problems. The idea in bundle methods is to approximate the subdifferential of the objective function by a bundle of subgradients. This information is then used to build a model for the objective. However, this model is typically convex and, due to this, it may be inaccurate and unable to adequately reflect the behaviour of the objective function in the nonconvex case. These circumstances motivate to design new bundle methods based on nonconvex models of the objective function. In this dissertation, the main focus is on nonsmooth DC optimization that constitutes an important and broad subclass of nonconvex optimization problems. A DC function can be presented as a difference of two convex functions. Thus, we can obtain a model that utilizes explicitly both the convexity and concavity of the objective by approximating separately the convex and concave parts. This way we end up with a nonconvex DC model describing the problem more accurately than the convex one. Based on the new DC model we introduce three different bundle methods. Two of them are designed for unconstrained DC optimization and the third one is capable of solving also multiobjective and constrained DC problems. The finite convergence is proved for each method. The numerical results demonstrate the efficiency of the methods and show the benefits obtained from the utilization of the DC decomposition. Even though the usage of the DC decomposition can improve the performance of the bundle methods, it is not always available or possible to construct. Thus, we present another bundle method for a general objective function implicitly collecting information about the DC structure. This method is developed for large-scale nonsmooth optimization and its convergence is proved for semismooth functions. The efficiency of the method is shown with numerical results. As an application of the developed methods, we consider the clusterwise linear regression (CLR) problems. By applying the support vector machines (SVM) approach a new model for these problems is proposed. The objective in the new formulation of the CLR problem is expressed as a DC function and a method based on one of the presented bundle methods is designed to solve it. Numerical results demonstrate robustness of the new approach to outliers.Monissa käytännön sovelluksissa tarkastelun kohteena oleva ongelma on monimutkainen ja joudutaan näin ollen mallintamaan epäsileillä funktioilla, jotka eivät välttämättä ole jatkuvasti differentioituvia kaikkialla. Klassisia gradienttiin perustuvia optimointimenetelmiä ei voida käyttää epäsileisiin tehtäviin, sillä epäsileillä funktioilla ei ole olemassa klassista gradienttia kaikkialla. Näin ollen epäsileään optimointiin on välttämätöntä kehittää omia numeerisia ratkaisumenetelmiä. Näistä kimppumenetelmiä pidetään tällä hetkellä kaikista tehokkaimpina ja luotettavimpina yleismenetelminä kyseisten tehtävien ratkaisemiseksi. Ideana kimppumenetelmissä on approksimoida kohdefunktion alidifferentiaalia kimpulla, joka on muodostettu keräämällä kohdefunktion aligradientteja edellisiltä iteraatiokierroksilta. Tätä tietoa hyödyntämällä voidaan muodostaa kohdefunktiolle malli, joka on alkuperäistä tehtävää helpompi ratkaista. Käytetty malli on tyypillisesti konveksi ja näin ollen se voi olla epätarkka ja kykenemätön esittämään alkuperäisen tehtävän rakennetta epäkonveksissa tapauksessa. Tästä syystä väitöskirjassa keskitytään kehittämään uusia kimppumenetelmiä, jotka mallinnusvaiheessa muodostavat kohdefunktiolle epäkonveksin mallin. Pääpaino väitöskirjassa on epäsileissä optimointitehtävissä, joissa funktiot voidaan esittää kahden konveksin funktion erotuksena (difference of two convex functions). Kyseisiä funktioita kutsutaan DC-funktioiksi ja ne muodostavat tärkeän ja laajan epäkonveksien funktioiden osajoukon. Tämä valinta mahdollistaa kohdefunktion konveksisuuden ja konkaavisuuden eksplisiittisen hyödyntämisen, sillä uusi malli kohdefunktiolle muodostetaan yhdistämällä erilliset konveksille ja konkaaville osalle rakennetut mallit. Tällä tavalla päädytään epäkonveksiin DC-malliin, joka pystyy kuvaamaan ratkaistavaa tehtävää tarkemmin kuin konveksi arvio. Väitöskirjassa esitetään kolme erilaista uuden DC-mallin pohjalta kehitettyä kimppumenetelmää sekä todistetaan menetelmien konvergenssit. Kaksi näistä menetelmistä on suunniteltu rajoitteettomaan DC-optimointiin ja kolmannella voidaan ratkaista myös monitavoitteisia ja rajoitteellisia DC-optimointitehtäviä. Numeeriset tulokset havainnollistavat menetelmien tehokkuutta sekä DC-hajotelman käytöstä saatuja etuja. Vaikka DC-hajotelman käyttö voi parantaa kimppumenetelmien suoritusta, sitä ei aina ole saatavilla tai mahdollista muodostaa. Tästä syystä väitöskirjassa esitetään myös neljäs kimppumenetelmä konvergenssitodistuksineen yleiselle kohdefunktiolle, jossa kerätään implisiittisesti tietoa kohdefunktion DC-rakenteesta. Menetelmä on kehitetty erityisesti suurille epäsileille optimointitehtäville ja sen tehokkuus osoitetaan numeerisella testauksella Sovelluksena väitöskirjassa tarkastellaan datalle klustereittain tehtävää lineaarista regressiota (clusterwise linear regression). Kyseiselle sovellukselle muodostetaan uusi malli hyödyntäen koneoppimisessa käytettyä SVM-lähestymistapaa (support vector machines approach) ja saatu kohdefunktio esitetään DC-funktiona. Näin ollen yhtä kehitetyistä kimppumenetelmistä sovelletaan tehtävän ratkaisemiseen. Numeeriset tulokset havainnollistavat uuden lähestymistavan robustisuutta ja tehokkuutta

The proximal point method for locally lipschitz functions in multiobjective optimization with application to the compromise problem

This paper studies the constrained multiobjective optimization problem of finding Pareto critical points of vector-valued functions. The proximal point method considered by Bonnel, Iusem, and Svaiter [SIAM J. Optim., 15 (2005), pp. 953–970] is extended to locally Lipschitz functions in the finite dimensional multiobjective setting. To this end, a new (scalarization-free) approach for convergence analysis of the method is proposed where the first-order optimality condition of the scalarized problem is replaced by a necessary condition for weak Pareto points of a multiobjective problem. As a consequence, this has allowed us to consider the method without any assumption of convexity over the constraint sets that determine the vectorial improvement steps. This is very important for applications; for example, to extend to a dynamic setting the famous compromise problem in management sciences and game theory.Fundação de Amparo à Pesquisa do Estado de GoiásConselho Nacional de Desenvolvimento Científico e TecnológicoCoordenação de Aperfeiçoamento de Pessoal de Nivel SuperiorMinisterio de Economía y CompetitividadAgence nationale de la recherch