A review of population-based metaheuristics for large-scale black-box global optimization: Part B

This paper is the second part of a two-part survey series on large-scale global optimization. The first part covered two major algorithmic approaches to large-scale optimization, namely decomposition methods and hybridization methods such as memetic algorithms and local search. In this part we focus on sampling and variation operators, approximation and surrogate modeling, initialization methods, and parallelization. We also cover a range of problem areas in relation to large-scale global optimization, such as multi-objective optimization, constraint handling, overlapping components, the component imbalance issue, and benchmarks, and applications. The paper also includes a discussion on pitfalls and challenges of current research and identifies several potential areas of future research

Memetic cooperative coevolution of Elman recurrent neural networks

Cooperative coevolution decomposes an optimi- sation problem into subcomponents and collectively solves them using evolutionary algorithms. Memetic algorithms provides enhancement to evolutionary algorithms with local search. Recently, the incorporation of local search into a memetic cooperative coevolution method has shown to be efficient for training feedforward networks on pattern classification problems. This paper applies the memetic cooperative coevolution method for training recurrent neural networks on grammatical inference problems. The results show that the proposed method achieves better performance in terms of optimisation time and robustness

Towards a more efficient use of computational budget in large-scale black-box optimization

Evolutionary algorithms are general purpose optimizers that have been shown effective in solving a variety of challenging optimization problems. In contrast to mathematical programming models, evolutionary algorithms do not require derivative information and are still effective when the algebraic formula of the given problem is unavailable. Nevertheless, the rapid advances in science and technology have witnessed the emergence of more complex optimization problems than ever, which pose significant challenges to traditional optimization methods. The dimensionality of the search space of an optimization problem when the available computational budget is limited is one of the main contributors to its difficulty and complexity. This so-called curse of dimensionality can significantly affect the efficiency and effectiveness of optimization methods including evolutionary algorithms. This research aims to study two topics related to a more efficient use of computational budget in evolutionary algorithms when solving large-scale black-box optimization problems. More specifically, we study the role of population initializers in saving the computational resource, and computational budget allocation in cooperative coevolutionary algorithms. Consequently, this dissertation consists of two major parts, each of which relates to one of these research directions. In the first part, we review several population initialization techniques that have been used in evolutionary algorithms. Then, we categorize them from different perspectives. The contribution of each category to improving evolutionary algorithms in solving large-scale problems is measured. We also study the mutual effect of population size and initialization technique on the performance of evolutionary techniques when dealing with large-scale problems. Finally, assuming uniformity of initial population as a key contributor in saving a significant part of the computational budget, we investigate whether achieving a high-level of uniformity in high-dimensional spaces is feasible given the practical restriction in computational resources. In the second part of the thesis, we study the large-scale imbalanced problems. In many real world applications, a large problem may consist of subproblems with different degrees of difficulty and importance. In addition, the solution to each subproblem may contribute differently to the overall objective value of the final solution. When the computational budget is restricted, which is the case in many practical problems, investing the same portion of resources in optimizing each of these imbalanced subproblems is not the most efficient strategy. Therefore, we examine several ways to learn the contribution of each subproblem, and then, dynamically allocate the limited computational resources in solving each of them according to its contribution to the overall objective value of the final solution. To demonstrate the effectiveness of the proposed framework, we design a new set of 40 large-scale imbalanced problems and study the performance of some possible instances of the framework

Bio-inspired computation: where we stand and what's next

In recent years, the research community has witnessed an explosion of literature dealing with the adaptation of behavioral patterns and social phenomena observed in nature towards efficiently solving complex computational tasks. This trend has been especially dramatic in what relates to optimization problems, mainly due to the unprecedented complexity of problem instances, arising from a diverse spectrum of domains such as transportation, logistics, energy, climate, social networks, health and industry 4.0, among many others. Notwithstanding this upsurge of activity, research in this vibrant topic should be steered towards certain areas that, despite their eventual value and impact on the field of bio-inspired computation, still remain insufficiently explored to date. The main purpose of this paper is to outline the state of the art and to identify open challenges concerning the most relevant areas within bio-inspired optimization. An analysis and discussion are also carried out over the general trajectory followed in recent years by the community working in this field, thereby highlighting the need for reaching a consensus and joining forces towards achieving valuable insights into the understanding of this family of optimization techniques

Bio-inspired computation: where we stand and what's next

In recent years, the research community has witnessed an explosion of literature dealing with the adaptation of behavioral patterns and social phenomena observed in nature towards efficiently solving complex computational tasks. This trend has been especially dramatic in what relates to optimization problems, mainly due to the unprecedented complexity of problem instances, arising from a diverse spectrum of domains such as transportation, logistics, energy, climate, social networks, health and industry 4.0, among many others. Notwithstanding this upsurge of activity, research in this vibrant topic should be steered towards certain areas that, despite their eventual value and impact on the field of bio-inspired computation, still remain insufficiently explored to date. The main purpose of this paper is to outline the state of the art and to identify open challenges concerning the most relevant areas within bio-inspired optimization. An analysis and discussion are also carried out over the general trajectory followed in recent years by the community working in this field, thereby highlighting the need for reaching a consensus and joining forces towards achieving valuable insights into the understanding of this family of optimization techniques

Problem Decomposition and Adaptation in Cooperative Neuro-Evolution

One way to train neural networks is to use evolutionary algorithms such as cooperative coevolution - a method that decomposes the network's learnable parameters into subsets, called subcomponents. Cooperative coevolution gains advantage over other methods by evolving particular subcomponents independently from the rest of the network. Its success depends strongly on how the problem decomposition is carried out. This thesis suggests new forms of problem decomposition, based on a novel and intuitive choice of modularity, and examines in detail at what stage and to what extent the different decomposition methods should be used. The new methods are evaluated by training feedforward networks to solve pattern classification tasks, and by training recurrent networks to solve grammatical inference problems. Efficient problem decomposition methods group interacting variables into the same subcomponents. We examine the methods from the literature and provide an analysis of the nature of the neural network optimization problem in terms of interacting variables. We then present a novel problem decomposition method that groups interacting variables and that can be generalized to neural networks with more than a single hidden layer. We then incorporate local search into cooperative neuro-evolution. We present a memetic cooperative coevolution method that takes into account the cost of employing local search across several sub-populations. The optimisation process changes during evolution in terms of diversity and interacting variables. To address this, we examine the adaptation of the problem decomposition method during the evolutionary process. The results in this thesis show that the proposed methods improve performance in terms of optimization time, scalability and robustness. As a further test, we apply the problem decomposition and adaptive cooperative coevolution methods for training recurrent neural networks on chaotic time series problems. The proposed methods show better performance in terms of accuracy and robustness

Optimized task scheduling based on hybrid symbiotic organisms search algorithms for cloud computing environment

In Cloud Computing model, users are charged according to the usage of resources and desired Quality of Service (QoS). Task scheduling algorithms are responsible for specifying adequate set of resources to execute user applications in the form of tasks, and schedule decisions of task scheduling algorithms are based on QoS requirements defined by the user. Task scheduling problem is an NP-Complete problem, due to the NP-Complete nature of task scheduling problems and huge search space presented by large scale problem instances, many of the existing solution algorithms incur high computational complexity and cannot effectively obtain global optimum solutions. Recently, Symbiotic Organisms Search (SOS) has been applied to various optimization problems and results obtained were found to be competitive with state-of-the-art metaheuristic algorithms. However, similar to the case other metaheuristic optimization algorithms, the efficiency of SOS algorithm deteriorates as the size of the search space increases. Moreover, SOS suffers from local optima entrapment and its static control parameters cannot maintain a balance between local and global search. In this study, Cooperative Coevolutionary Constrained Multiobjective Symbiotic Organisms Search (CC-CMSOS), Cooperative Coevolutionary Constrained Multi-objective Memetic Symbiotic Organisms Search (CC-CMMSOS), and Cooperative Coevolutionary Constrained Multi-objective Adaptive Benefit Factor Symbiotic Organisms Search (CC-CMABFSOS) algorithms are proposed to solve constrained multi-objective large scale task scheduling optimization problem on IaaS cloud computing environment. To address the issue of scalability, the concept of Cooperative Coevolutionary for enhancing SOS named CC-CMSOS make SOS more efficient for solving large scale task scheduling problems. CC-CMMSOS algorithm further improves the performance of SOS algorithm by hybridizing with Simulated Annealing (SA) to avoid entrapment in local optima for global convergence. Finally, CC-CMABFSOS algorithm adaptively turn SOS control parameters to balance the local and global search procedure for faster convergence speed. The performance of the proposed CC-CMSOS, CC-CMMSOS, and CC-CMABFSOS algorithms are evaluated on CloudSim simulator, using both standard workload traces and synthesized workloads for larger problem instances of up to 5000. Moreover, CC-CMSOS, CC-CMMSOS, and CC-CMABFSOS algorithms are compared with multi-objective optimization algorithms, namely, EMS-C, ECMSMOO, and BOGA. The CC-CMSOS, CC-CMMSOS, and CC-CMABFSOS algorithms obtained significant improved optimal trade-offs between execution time (makespan) and financial cost (cost) while meeting deadline constraints with no computational overhead. The performance improvements obtained by the proposed algorithms in terms of hypervolume ranges from 8.72% to 37.95% across the workloads. Therefore, the proposed algorithms have potentials to improve the performance of QoS delivery