Development of Methods for Solving Bilevel Optimization Problems

Bilevel optimization, also referred to as bilevel programming, involves solving an upper level problem subject to the optimality of a corresponding lower level problem. The upper and lower level problems are also referred to as the leader and follower problems, respectively. Both levels have their associated objective(s), variable(s) and constraint(s). Such problems model real-life scenarios of cases where the performance of an upper level authority is realizable/sustainable only if the corresponding lower level objective is optimum. A number of practical applications in the field of engineering, logistics, economics and transportation have inherent nested structure that are suited to this type of modelling. The range of applications as well as a rapid increase in the size and complexity of such problems has prompted active interest in the design of efficient algorithms for bilevel optimization. Bilevel optimization problems present a number of unique and interesting challenges to algorithm design. The nested nature of the problem requires optimization of a lower level problem to evaluate each upper level solution, which makes it computationally exorbitant. Theoretically, an upper level solution is considered valid/feasible only if the corresponding lower level variables are the true global optimum of the lower level problem. Global optimality can be reliably asserted in very limited cases, for example convex and linear problems. In deceptive cases, an inaccurate lower level optimum may result in an objective value better than true optimum at the upper level, which poses a severe challenge for ranking/selection strategies used within any optimization technique. In turn, this also makes the performance evaluation very difficult since the performance cannot be judged based on the objective values alone. While the area of bilevel (or more generally, multilevel) programming itself is not very new, most reports in this direction up until about a decade ago considered solving linear or at most quadratic problems at both levels. Correspondingly, the focus on was on development of exact methods to solve such problems. However, such methods typically require assumptions on mathematical properties, which may not always hold in practical applications. With increasing use of computer simulation-based evaluations in a number of disciplines in science and engineering, there is more need than ever to handle problems that are highly nonlinear or even black-box in nature. Metaheuristic algorithms, such as evolutionary algorithms are more suited to this emerging paradigm. The foray of evolutionary algorithms in bilevel programming is relatively recent and there remains scope of substantial development in the field in terms of addressing the aforementioned challenges. The work presented in this thesis is directed towards improving evolutionary techniques to enable them solve generic bilevel problems more accurately using lower number of function evaluations compared to the existing methods. Three key approaches are investigated towards accomplishing this: (a) e active hybridization of global and local search methods during dierent stages of the overall search; (b) use of surrogate models to guide the search using approximations in lieu of true function evaluations, and (c) use of a non-nested re-formulation of the problem. While most of the work is focused on single-objective problems, preliminary studies are also presented on multi-objective bilevel problems. The performance of the proposed approaches is evaluated on a comprehensive suite of mathematical test problems available in the literature, as well as some practical problems. The proposed approaches are observed to achieve a favourable balance between accuracy and computational expense for solving bilevel optimization problems, and thus exhibit suitability for use in real-life applications

Co-evolutionary Hybrid Bi-level Optimization

Multi-level optimization stems from the need to tackle complex problems involving multiple decision makers. Two-level optimization, referred as ``Bi-level optimization'', occurs when two decision makers only control part of the decision variables but impact each other (e.g., objective value, feasibility). Bi-level problems are sequential by nature and can be represented as nested optimization problems in which one problem (the ``upper-level'') is constrained by another one (the ``lower-level''). The nested structure is a real obstacle that can be highly time consuming when the lower-level is $\mathcal{NP}-hard$. Consequently, classical nested optimization should be avoided. Some surrogate-based approaches have been proposed to approximate the lower-level objective value function (or variables) to reduce the number of times the lower-level is globally optimized. Unfortunately, such a methodology is not applicable for large-scale and combinatorial bi-level problems. After a deep study of theoretical properties and a survey of the existing applications being bi-level by nature, problems which can benefit from a bi-level reformulation are investigated. A first contribution of this work has been to propose a novel bi-level clustering approach. Extending the well-know ``uncapacitated k-median problem'', it has been shown that clustering can be easily modeled as a two-level optimization problem using decomposition techniques. The resulting two-level problem is then turned into a bi-level problem offering the possibility to combine distance metrics in a hierarchical manner. The novel bi-level clustering problem has a very interesting property that enable us to tackle it with classical nested approaches. Indeed, its lower-level problem can be solved in polynomial time. In cooperation with the Luxembourg Centre for Systems Biomedicine (LCSB), this new clustering model has been applied on real datasets such as disease maps (e.g. Parkinson, Alzheimer). Using a novel hybrid and parallel genetic algorithm as optimization approach, the results obtained after a campaign of experiments have the ability to produce new knowledge compared to classical clustering techniques combining distance metrics in a classical manner. The previous bi-level clustering model has the advantage that the lower-level can be solved in polynomial time although the global problem is by definition $\mathcal{NP}$-hard. Therefore, next investigations have been undertaken to tackle more general bi-level problems in which the lower-level problem does not present any specific advantageous properties. Since the lower-level problem can be very expensive to solve, the focus has been turned to surrogate-based approaches and hyper-parameter optimization techniques with the aim of approximating the lower-level problem and reduce the number of global lower-level optimizations. Adapting the well-know bayesian optimization algorithm to solve general bi-level problems, the expensive lower-level optimizations have been dramatically reduced while obtaining very accurate solutions. The resulting solutions and the number of spared lower-level optimizations have been compared to the bi-level evolutionary algorithm based on quadratic approximations (BLEAQ) results after a campaign of experiments on official bi-level benchmarks. Although both approaches are very accurate, the bi-level bayesian version required less lower-level objective function calls. Surrogate-based approaches are restricted to small-scale and continuous bi-level problems although many real applications are combinatorial by nature. As for continuous problems, a study has been performed to apply some machine learning strategies. Instead of approximating the lower-level solution value, new approximation algorithms for the discrete/combinatorial case have been designed. Using the principle employed in GP hyper-heuristics, heuristics are trained in order to tackle efficiently the $\mathcal{NP}-hard$ lower-level of bi-level problems. This automatic generation of heuristics permits to break the nested structure into two separated phases: \emph{training lower-level heuristics} and \emph{solving the upper-level problem with the new heuristics}. At this occasion, a second modeling contribution has been introduced through a novel large-scale and mixed-integer bi-level problem dealing with pricing in the cloud, i.e., the Bi-level Cloud Pricing Optimization Problem (BCPOP). After a series of experiments that consisted in training heuristics on various lower-level instances of the BCPOP and using them to tackle the bi-level problem itself, the obtained results are compared to the ``cooperative coevolutionary algorithm for bi-level optimization'' (COBRA). Although training heuristics enables to \emph{break the nested structure}, a two phase optimization is still required. Therefore, the emphasis has been put on training heuristics while optimizing the upper-level problem using competitive co-evolution. Instead of adopting the classical decomposition scheme as done by COBRA which suffers from the strong epistatic links between lower-level and upper-level variables, co-evolving the solution and the mean to get to it can cope with these epistatic link issues. The ``CARBON'' algorithm developed in this thesis is a competitive and hybrid co-evolutionary algorithm designed for this purpose. In order to validate the potential of CARBON, numerical experiments have been designed and results have been compared to state-of-the-art algorithms. These results demonstrate that ``CARBON'' makes possible to address nested optimization efficiently

Efficient Algorithms for Computationally Expensive Multifidelity Optimization Problems

Multifidelity optimization problems refer to a class of problems where one is presented with a physical system or mathematical model that can be represented in different levels of fidelity. The term “fidelity” refers to the accuracy of representation, where higher fidelity estimates are more accurate and expensive, while lower fidelity estimates are inaccurate, albeit cheaper. Most common iterative solvers such as those employed in computational fluid dynamics (CFD), finite element analysis (FEA), computational electromagnetics (CEM) etc. can be run with different fine/course meshes or residual error thresholds to yield estimates in various fidelities. In the event an optimization exercise requires their use, it is possible to invoke analysis in various fidelities for different solutions during the course of search. Multifidelity optimization algorithms are the special class of algorithms that are able to deal with analysis in various levels of fidelity. In this thesis, two novel multifidelity optimization algorithms have been developed. The first is to deal with bilevel optimization problems and the second is to deal with robust optimization problems involving iterative solvers. Bilevel optimization problems are particularly challenging as the optimum of an upper level (UL) problem is sought subject to the optimality of a nested lower level (LL) problem. Due to the inherent nested nature, naive implementations consume very significant number of UL and LL evaluations. The proposed multifidelity approach controls the rigour of LL optimization exercise for any given UL solution during the course of search as opposed to undertaking exhaustive LL optimization for every UL solution. Robust optimization problems are yet another class of problems where numerous solutions need to be assessed since the intent is to identify solutions that have both good performance and is also insensitive to unavoidable perturbations in the variable values. Computing the latter metric requires evaluation of numerous solutions in the vicinity of the given solution and not all solutions are worthy of such computation. The proposed multifidelity approach considers pre-converged simulations as lower fidelity estimates and uses them to reduce the computational overhead. While multi-objective optimization problems have long been in existence, there has been limited attempts in the past to deal with problems where the objectives can be independently computed. For example, the weight of a structure and the maximum stress in the structure are two objectives that can be independently computed. For such classes of problems, an efficient algorithm should ideally evaluate either one or both objectives as opposed of always evaluating both objectives. A novel algorithm is introduced that is capable of selectively evaluating the objectives of the infill solutions. The approach exploits principles of non-dominance and sparse subset selection to facilitate decomposition and through maximization of probabilistic dominance (PD) measure, identifies the infill solutions. Thereafter, for each of these infill solutions, one or more objectives are evaluated based on evaluation status of its closest neighbor and the probability of improvement along each objective. Finally, there has been significant research interest in recent years to develop efficient algorithms to deal with multimodal, multi-objective optimization problems (MMOPs). Such problems are particulatly challenging as there is a need to identify well distributed and well converged solutions in the objective space along with diverse solutions in the variable space. Existing algorithms for MMOPs still require prohibitive number of function evaluations (often in several thousands). The algorithms are typically embedded with sophisticated, customized mechanisms that require additional parameters to manage the diversity and convergence in the variable and the objective spaces. A steady-state evolutionary algorithm is introduced in this thesis for solving MMOPs, with a simple design and no additional user-defined parameters that need tuning. All the developments listed above have been studied using well established benchmarks and real-world examples. The results have been compared with existing state-of-the-art approaches to substantiate the benefits

Bi-level optimisation and machine learning in the management of large service-oriented field workforces.

The tactical planning problem for members of the service industry with large multi-skilled workforces is an important process that is often underlooked. It sits between the operational plan - which involves the actual allocation of members of the workforce to tasks - and the strategic plan where long term visions are set. An accurate tactical plan can have great benefits to service organisations and this is something we demonstrate in this work. Sitting where it does, it is made up of a mix of forecast and actual data, which can make effectively solving the problem difficult. In members of the service industry with large multi-skilled workforces it can often become a very large problem very quickly, as the number of decisions scale quickly with the number of elements within the plan. In this study, we first update and define the tactical planning problem to fit the process currently undertaken manually in practice. We then identify properties within the problem that identify it as a new candidate for the application of bi-level optimisation techniques. The tactical plan is defined in the context of a pair of leader-follower linked sub-models, which we show to be solvable to produce automated solutions to the tactical plan. We further identify the need for the use of machine learning techniques to effectively find solutions in practical applications, where limited detail is available in the data due to its forecast nature. We develop neural network models to solve this issue and show that they provide more accurate results than the current planners. Finally, we utilise them as a surrogate for the follower in the bi-level framework to provide real world applicable solutions to the tactical planning problem. The models developed in this work have already begun to be deployed in practice and are providing significant impact. This is along with identifying a new application area for bi-level modelling techniques

Application of a Modified ACO Algorithm for Optimizing Routes and Externality Effect of Solid Waste Management

To improve solid waste management and maintain its sustainability, it is important to reduce both the solid waste operational cost which includes the monetary value of distances covered and the externality effects of solid waste management. Therefore, this paper presents an application of a modified Ant Colony System algorithm to a bi-objective model for solid waste management in the Shama District in the Western Region of Ghana. The objective is to optimize route lengths and externality effects of solid waste management. Data on route lengths and population of communities along the routes were collected from 20 communities in the Shama Distric. Externality effect was measured by considering the population of the communities along the routes, the cost of treating a common cold subject to the assumption of two percent of the population being affected by the externality effect. The implemented algorithm has demonstrated the bi-objective optimal solution of route length (km) and externality effect (GHS) of (11, 2100) achievable on the path , which respectively represents a path linking the following communities: Aboadze, Abuesi Assorko Essaman, Beposo, Bosomdo and Fawomanye. There is therefore the need to ensure that the communities involved are linked with good roads

Understanding Complexity in Multiobjective Optimization

This report documents the program and outcomes of the Dagstuhl Seminar 15031 Understanding Complexity in Multiobjective Optimization. This seminar carried on the series of four previous Dagstuhl Seminars (04461, 06501, 09041 and 12041) that were focused on Multiobjective Optimization, and strengthening the links between the Evolutionary Multiobjective Optimization (EMO) and Multiple Criteria Decision Making (MCDM) communities. The purpose of the seminar was to bring together researchers from the two communities to take part in a wide-ranging discussion about the different sources and impacts of complexity in multiobjective optimization. The outcome was a clarified viewpoint of complexity in the various facets of multiobjective optimization, leading to several research initiatives with innovative approaches for coping with complexity

Evolutionary Algorithms in Engineering Design Optimization

Evolutionary algorithms (EAs) are population-based global optimizers, which, due to their characteristics, have allowed us to solve, in a straightforward way, many real world optimization problems in the last three decades, particularly in engineering fields. Their main advantages are the following: they do not require any requisite to the objective/fitness evaluation function (continuity, derivability, convexity, etc.); they are not limited by the appearance of discrete and/or mixed variables or by the requirement of uncertainty quantification in the search. Moreover, they can deal with more than one objective function simultaneously through the use of evolutionary multi-objective optimization algorithms. This set of advantages, and the continuously increased computing capability of modern computers, has enhanced their application in research and industry. From the application point of view, in this Special Issue, all engineering fields are welcomed, such as aerospace and aeronautical, biomedical, civil, chemical and materials science, electronic and telecommunications, energy and electrical, manufacturing, logistics and transportation, mechanical, naval architecture, reliability, robotics, structural, etc. Within the EA field, the integration of innovative and improvement aspects in the algorithms for solving real world engineering design problems, in the abovementioned application fields, are welcomed and encouraged, such as the following: parallel EAs, surrogate modelling, hybridization with other optimization techniques, multi-objective and many-objective optimization, etc