Development of Methods for Solving Bilevel Optimization Problems

Bilevel optimization, also referred to as bilevel programming, involves solving an upper level problem subject to the optimality of a corresponding lower level problem. The upper and lower level problems are also referred to as the leader and follower problems, respectively. Both levels have their associated objective(s), variable(s) and constraint(s). Such problems model real-life scenarios of cases where the performance of an upper level authority is realizable/sustainable only if the corresponding lower level objective is optimum. A number of practical applications in the field of engineering, logistics, economics and transportation have inherent nested structure that are suited to this type of modelling. The range of applications as well as a rapid increase in the size and complexity of such problems has prompted active interest in the design of efficient algorithms for bilevel optimization. Bilevel optimization problems present a number of unique and interesting challenges to algorithm design. The nested nature of the problem requires optimization of a lower level problem to evaluate each upper level solution, which makes it computationally exorbitant. Theoretically, an upper level solution is considered valid/feasible only if the corresponding lower level variables are the true global optimum of the lower level problem. Global optimality can be reliably asserted in very limited cases, for example convex and linear problems. In deceptive cases, an inaccurate lower level optimum may result in an objective value better than true optimum at the upper level, which poses a severe challenge for ranking/selection strategies used within any optimization technique. In turn, this also makes the performance evaluation very difficult since the performance cannot be judged based on the objective values alone. While the area of bilevel (or more generally, multilevel) programming itself is not very new, most reports in this direction up until about a decade ago considered solving linear or at most quadratic problems at both levels. Correspondingly, the focus on was on development of exact methods to solve such problems. However, such methods typically require assumptions on mathematical properties, which may not always hold in practical applications. With increasing use of computer simulation-based evaluations in a number of disciplines in science and engineering, there is more need than ever to handle problems that are highly nonlinear or even black-box in nature. Metaheuristic algorithms, such as evolutionary algorithms are more suited to this emerging paradigm. The foray of evolutionary algorithms in bilevel programming is relatively recent and there remains scope of substantial development in the field in terms of addressing the aforementioned challenges. The work presented in this thesis is directed towards improving evolutionary techniques to enable them solve generic bilevel problems more accurately using lower number of function evaluations compared to the existing methods. Three key approaches are investigated towards accomplishing this: (a) e active hybridization of global and local search methods during dierent stages of the overall search; (b) use of surrogate models to guide the search using approximations in lieu of true function evaluations, and (c) use of a non-nested re-formulation of the problem. While most of the work is focused on single-objective problems, preliminary studies are also presented on multi-objective bilevel problems. The performance of the proposed approaches is evaluated on a comprehensive suite of mathematical test problems available in the literature, as well as some practical problems. The proposed approaches are observed to achieve a favourable balance between accuracy and computational expense for solving bilevel optimization problems, and thus exhibit suitability for use in real-life applications

Bi-level optimisation and machine learning in the management of large service-oriented field workforces.

The tactical planning problem for members of the service industry with large multi-skilled workforces is an important process that is often underlooked. It sits between the operational plan - which involves the actual allocation of members of the workforce to tasks - and the strategic plan where long term visions are set. An accurate tactical plan can have great benefits to service organisations and this is something we demonstrate in this work. Sitting where it does, it is made up of a mix of forecast and actual data, which can make effectively solving the problem difficult. In members of the service industry with large multi-skilled workforces it can often become a very large problem very quickly, as the number of decisions scale quickly with the number of elements within the plan. In this study, we first update and define the tactical planning problem to fit the process currently undertaken manually in practice. We then identify properties within the problem that identify it as a new candidate for the application of bi-level optimisation techniques. The tactical plan is defined in the context of a pair of leader-follower linked sub-models, which we show to be solvable to produce automated solutions to the tactical plan. We further identify the need for the use of machine learning techniques to effectively find solutions in practical applications, where limited detail is available in the data due to its forecast nature. We develop neural network models to solve this issue and show that they provide more accurate results than the current planners. Finally, we utilise them as a surrogate for the follower in the bi-level framework to provide real world applicable solutions to the tactical planning problem. The models developed in this work have already begun to be deployed in practice and are providing significant impact. This is along with identifying a new application area for bi-level modelling techniques

Efficient Algorithms for Computationally Expensive Multifidelity Optimization Problems

Multifidelity optimization problems refer to a class of problems where one is presented with a physical system or mathematical model that can be represented in different levels of fidelity. The term “fidelity” refers to the accuracy of representation, where higher fidelity estimates are more accurate and expensive, while lower fidelity estimates are inaccurate, albeit cheaper. Most common iterative solvers such as those employed in computational fluid dynamics (CFD), finite element analysis (FEA), computational electromagnetics (CEM) etc. can be run with different fine/course meshes or residual error thresholds to yield estimates in various fidelities. In the event an optimization exercise requires their use, it is possible to invoke analysis in various fidelities for different solutions during the course of search. Multifidelity optimization algorithms are the special class of algorithms that are able to deal with analysis in various levels of fidelity. In this thesis, two novel multifidelity optimization algorithms have been developed. The first is to deal with bilevel optimization problems and the second is to deal with robust optimization problems involving iterative solvers. Bilevel optimization problems are particularly challenging as the optimum of an upper level (UL) problem is sought subject to the optimality of a nested lower level (LL) problem. Due to the inherent nested nature, naive implementations consume very significant number of UL and LL evaluations. The proposed multifidelity approach controls the rigour of LL optimization exercise for any given UL solution during the course of search as opposed to undertaking exhaustive LL optimization for every UL solution. Robust optimization problems are yet another class of problems where numerous solutions need to be assessed since the intent is to identify solutions that have both good performance and is also insensitive to unavoidable perturbations in the variable values. Computing the latter metric requires evaluation of numerous solutions in the vicinity of the given solution and not all solutions are worthy of such computation. The proposed multifidelity approach considers pre-converged simulations as lower fidelity estimates and uses them to reduce the computational overhead. While multi-objective optimization problems have long been in existence, there has been limited attempts in the past to deal with problems where the objectives can be independently computed. For example, the weight of a structure and the maximum stress in the structure are two objectives that can be independently computed. For such classes of problems, an efficient algorithm should ideally evaluate either one or both objectives as opposed of always evaluating both objectives. A novel algorithm is introduced that is capable of selectively evaluating the objectives of the infill solutions. The approach exploits principles of non-dominance and sparse subset selection to facilitate decomposition and through maximization of probabilistic dominance (PD) measure, identifies the infill solutions. Thereafter, for each of these infill solutions, one or more objectives are evaluated based on evaluation status of its closest neighbor and the probability of improvement along each objective. Finally, there has been significant research interest in recent years to develop efficient algorithms to deal with multimodal, multi-objective optimization problems (MMOPs). Such problems are particulatly challenging as there is a need to identify well distributed and well converged solutions in the objective space along with diverse solutions in the variable space. Existing algorithms for MMOPs still require prohibitive number of function evaluations (often in several thousands). The algorithms are typically embedded with sophisticated, customized mechanisms that require additional parameters to manage the diversity and convergence in the variable and the objective spaces. A steady-state evolutionary algorithm is introduced in this thesis for solving MMOPs, with a simple design and no additional user-defined parameters that need tuning. All the developments listed above have been studied using well established benchmarks and real-world examples. The results have been compared with existing state-of-the-art approaches to substantiate the benefits

Multidimensional Resource Fragmentation-Aware Virtual Network Embedding in MEC Systems Interconnected by Metro Optical Networks

The increasing demand for diverse emerging applications has resulted in the interconnection of multi-access edge computing (MEC) systems via metro optical networks. To cater to these diverse applications, network slicing has become a popular tool for creating specialized virtual networks. However, resource fragmentation caused by uneven utilization of multidimensional resources can lead to reduced utilization of limited edge resources. To tackle this issue, this paper focuses on addressing the multidimensional resource fragmentation problem in virtual network embedding (VNE) in MEC systems with the aim of maximizing the profit of an infrastructure provider (InP). The VNE problem in MEC systems is transformed into a bilevel optimization problem, taking into account the interdependence between virtual node embedding (VNoE) and virtual link embedding (VLiE). To solve this problem, we propose a nested bilevel optimization approach named BiVNE. The VNoE is solved using the ant colony system (ACS) in the upper level, while the VLiE is solved using a combination of a shortest path algorithm and an exact-fit spectrum slot allocation method in the lower level. Evaluation results show that the BiVNE algorithm can effectively enhance the profit of the InP by increasing the acceptance ratio and avoiding resource fragmentation simultaneously

New heuristics for multi-objective worst-case optimization in evidence-based robust design

This paper presents a non-nested algorithm for the solution of multi-objective min-max problems (MOMMP) in worst-case optimization. The algorithm has been devised for evidence-based robust optimization, where the lack of a defined probabilistic behaviour of the uncertain parameters makes it impossible to apply sample-based techniques and forces the designer to identify the worst case over the subdomains of the uncertainty space. In evidence theory, the robustness of the solutions is measured in terms of the Belief in the realization of the value of the design budgets, which acts as a lower bound to the unknown cumulative distribution function of the budget. Thus a means of finding robust solutions in preliminary design consists on applying the minimax model, where the worst-case budget over the uncertainty space is optimized over the control space. The paper proposes a novel heuristic to solve MOMMP and demonstrates its capability to approximate the worst-case Pareto front at a very reduced cost with respect to approaches based on nested optimization

An efficient hybrid differential evolutionary algorithm for zbilevel optimisation problems

Bilevel problems are widely used to describe the decision problems with hierarchical upper–lower-level structures in many economic fields. The bilevel optimisation problem (BLOP) is intrinsically NP-hard when its objectives and constraints are complex and the decision variables are large in scale at both levels. An efficient hybrid differential evolutionary algorithm for BLOP (HDEAB) is proposed where the optimal lower level value function mapping method, the differential evolutionary algorithm, k-near- est neighbours (KNN) and a nested local search are hybridised to improve the computational accuracy and efficiency. To show the performance of the HDEAB, numerical studies were conducted on SMD (Sinha, Maro and Deb) instances and an application example of optimising a venture capital staged-financing contract. The results demonstrate that the HDEAB outperforms the BLEAQ (bile- vel evolutionary algorithm based on quadratic approximations) greatly in solving the BLOPs with different scale