Seeking multiple solutions:an updated survey on niching methods and their applications

Multi-Modal Optimization (MMO) aiming to locate multiple optimal (or near-optimal) solutions in a single simulation run has practical relevance to problem solving across many fields. Population-based meta-heuristics have been shown particularly effective in solving MMO problems, if equipped with specificallydesigned diversity-preserving mechanisms, commonly known as niching methods. This paper provides an updated survey on niching methods. The paper first revisits the fundamental concepts about niching and its most representative schemes, then reviews the most recent development of niching methods, including novel and hybrid methods, performance measures, and benchmarks for their assessment. Furthermore, the paper surveys previous attempts at leveraging the capabilities of niching to facilitate various optimization tasks (e.g., multi-objective and dynamic optimization) and machine learning tasks (e.g., clustering, feature selection, and learning ensembles). A list of successful applications of niching methods to real-world problems is presented to demonstrate the capabilities of niching methods in providing solutions that are difficult for other optimization methods to offer. The significant practical value of niching methods is clearly exemplified through these applications. Finally, the paper poses challenges and research questions on niching that are yet to be appropriately addressed. Providing answers to these questions is crucial before we can bring more fruitful benefits of niching to real-world problem solving

A Stackelberg game theoretic model for optimizing product family architecting with supply chain consideration

Planning of an optimal product family architecture (PFA) plays a critical role in defining an organization's product platforms for product variant configuration while leveraging commonality and variety. The focus of PFA planning has been traditionally limited to the product design stage, yet with limited consideration of the downstream supply chain-related issues. Decisions of supply chain configuration have a profound impact on not only the end cost of product family fulfillment, but also how to design the architecture of module configuration within a product family. It is imperative for product family architecting to be optimized in conjunction with supply chain configuration decisions. This paper formulates joint optimization of PFA planning and supply chain configuration as a Stackelberg game. A nonlinear, mixed integer bilevel programming model is developed to deal with the leader–follower game decisions between product family architecting and supply chain configuration. The PFA decision making is represented as an upper-level optimization problem for optimal selection of the base modules and compound modules. A lower-level optimization problem copes with supply chain decisions in accordance with the upper-level decisions of product variant configuration. Consistent with the bilevel optimization model, a nested genetic algorithm is developed to derive near optimal solutions for PFA and the corresponding supply chain network. A case study of joint PFA and supply chain decisions for power transformers is reported to demonstrate the feasibility and potential of the proposed Stackelberg game theoretic joint optimization of PFA and supply chain decisions

Bi-level optimisation and machine learning in the management of large service-oriented field workforces.

The tactical planning problem for members of the service industry with large multi-skilled workforces is an important process that is often underlooked. It sits between the operational plan - which involves the actual allocation of members of the workforce to tasks - and the strategic plan where long term visions are set. An accurate tactical plan can have great benefits to service organisations and this is something we demonstrate in this work. Sitting where it does, it is made up of a mix of forecast and actual data, which can make effectively solving the problem difficult. In members of the service industry with large multi-skilled workforces it can often become a very large problem very quickly, as the number of decisions scale quickly with the number of elements within the plan. In this study, we first update and define the tactical planning problem to fit the process currently undertaken manually in practice. We then identify properties within the problem that identify it as a new candidate for the application of bi-level optimisation techniques. The tactical plan is defined in the context of a pair of leader-follower linked sub-models, which we show to be solvable to produce automated solutions to the tactical plan. We further identify the need for the use of machine learning techniques to effectively find solutions in practical applications, where limited detail is available in the data due to its forecast nature. We develop neural network models to solve this issue and show that they provide more accurate results than the current planners. Finally, we utilise them as a surrogate for the follower in the bi-level framework to provide real world applicable solutions to the tactical planning problem. The models developed in this work have already begun to be deployed in practice and are providing significant impact. This is along with identifying a new application area for bi-level modelling techniques

A new differential evolution using a bilevel optimization model for solving generalized multi-point dynamic aggregation problems

The multi-point dynamic aggregation problem (MPDAP) comes mainly from real-world applications, which is characterized by dynamic task assignation and routing optimization with limited resources. Due to the dynamic allocation of tasks, more than one optimization objective, limited resources, and other factors involved, the computational complexity of both route programming and resource allocation optimization is a growing problem. In this manuscript, a task scheduling problem of fire-fighting robots is investigated and solved, and serves as a representative multi-point dynamic aggregation problem. First, in terms of two optimized objectives, the cost and completion time, a new bilevel programming model is presented, in which the task cost is taken as the leader's objective. In addition, in order to effectively solve the bilevel model, a differential evolution is developed based on a new matrix coding scheme. Moreover, some percentage of high-quality solutions are applied in mutation and selection operations, which helps to generate potentially better solutions and keep them into the next generation of population. Finally, the experimental results show that the proposed algorithm is feasible and effective in dealing with the multi-point dynamic aggregation problem

Evolutionary Algorithms in Engineering Design Optimization

Evolutionary algorithms (EAs) are population-based global optimizers, which, due to their characteristics, have allowed us to solve, in a straightforward way, many real world optimization problems in the last three decades, particularly in engineering fields. Their main advantages are the following: they do not require any requisite to the objective/fitness evaluation function (continuity, derivability, convexity, etc.); they are not limited by the appearance of discrete and/or mixed variables or by the requirement of uncertainty quantification in the search. Moreover, they can deal with more than one objective function simultaneously through the use of evolutionary multi-objective optimization algorithms. This set of advantages, and the continuously increased computing capability of modern computers, has enhanced their application in research and industry. From the application point of view, in this Special Issue, all engineering fields are welcomed, such as aerospace and aeronautical, biomedical, civil, chemical and materials science, electronic and telecommunications, energy and electrical, manufacturing, logistics and transportation, mechanical, naval architecture, reliability, robotics, structural, etc. Within the EA field, the integration of innovative and improvement aspects in the algorithms for solving real world engineering design problems, in the abovementioned application fields, are welcomed and encouraged, such as the following: parallel EAs, surrogate modelling, hybridization with other optimization techniques, multi-objective and many-objective optimization, etc

Uncertainty-wise software anti-patterns detection: A possibilistic evolutionary machine learning approach

Context: Code smells (a.k.a. anti-patterns) are manifestations of poor design solutions that can deteriorate software maintainability and evolution. Research gap: Existing works did not take into account the issue of uncertain class labels, which is an important inherent characteristic of the smells detection problem. More precisely, two human experts may have different degrees of uncertainty about the smelliness of a particular software class not only for the smell detection task but also for the smell type identification one. Unluckily, existing approaches usually reject and/or ignore uncertain data that correspond to software classes (i.e. dataset instances) with uncertain labels. Throwing away and/or disregarding the uncertainty factor could considerably degrade the detection/identification process effectiveness. From a solution approach viewpoint, there is no work in the literature that proposed a method that is able to detect and/or identify code smells while preserving the uncertainty aspect. Objective: The main goal of our research work is to handle the uncertainty factor, issued from human experts, in detecting and/or identifying code smells by proposing an evolutionary approach that is able to deal with anti-patterns classification with uncertain labels. Method: We suggest Bi-ADIPOK, as an effective search-based tool that is capable to tackle the previously mentioned challenge for both detection and identification cases. The proposed method corresponds to an EA (Evolutionary Algorithm) that optimizes a set of detectors encoded as PK-NNs (Possibilistic K-nearest neighbors) based on a bi-level hierarchy, in which the upper level role consists on finding the optimal PK-NNs parameters, while the lower level one is to generate the PK-NNs. A newly fitness function has been proposed fitness function PomAURPC-OVA_dist (Possibilistic modified Area Under Recall Precision Curve One-Versus-All_distance, abbreviated PAURPC_d in this paper). Bi-ADIPOK is able to deal with label uncertainty using some concepts stemming from the Possibility Theory. Furthermore, the PomAURPC-OVA_dist is capable to process the uncertainty issue even with imbalanced data. We notice that Bi-ADIPOK is first built and then validated using a possibilistic base of smell examples that simulates and mimics the subjectivity of software engineers opinions. Results: The statistical analysis of the obtained results on a set of comparative experiments with respect to four relevant state-of-the-art methods shows the merits of our proposal. The obtained detection results demonstrate that, for the uncertain environment, the PomAURPC-OVA_dist of Bi-ADIPOK ranges between 0.902 and 0.932 and its IAC lies between 0.9108 and 0.9407, while for the certain environment, the PomAURPC-OVA_dist lies between 0.928 and 0.955 and the IAC ranges between 0.9477 and 0.9622. Similarly, the identification results, for the uncertain environment, indicate that the PomAURPC-OVA_dist of Bi-ADIPOK varies between 0.8576 and 0.9273 and its IAC is between 0.8693 and 0.9318. For the certain environment, the PomAURPC-OVA_dist lies between 0.8613 and 0.9351 and the IAC values are between 0.8672 and 0.9476. With uncertain data, Bi-ADIPOK can find 35% more code smells than the second best approach (i.e., BLOP). Furthermore, Bi-ADIPOK has succeeded to reduce the number of false alarms (i.e., misclassified smelly instances) by 12%. In addition, our proposed approach can identify 43% more smell types than BLOP and reduces the number of false alarms by 32%. The same results have been obtained for the certain environment, demonstrating Bi-ADIPOK's ability to deal with such environment

Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges

Most machine learning algorithms are configured by a set of hyperparameters whose values must be carefully chosen and which often considerably impact performance. To avoid a time-consuming and irreproducible manual process of trial-and-error to find well-performing hyperparameter configurations, various automatic hyperparameter optimization (HPO) methods—for example, based on resampling error estimation for supervised machine learning—can be employed. After introducing HPO from a general perspective, this paper reviews important HPO methods, from simple techniques such as grid or random search to more advanced methods like evolution strategies, Bayesian optimization, Hyperband, and racing. This work gives practical recommendations regarding important choices to be made when conducting HPO, including the HPO algorithms themselves, performance evaluation, how to combine HPO with machine learning pipelines, runtime improvements, and parallelization. This article is categorized under: Algorithmic Development > Statistics Technologies > Machine Learning Technologies > Prediction