A review of population-based metaheuristics for large-scale black-box global optimization: Part B

This paper is the second part of a two-part survey series on large-scale global optimization. The first part covered two major algorithmic approaches to large-scale optimization, namely decomposition methods and hybridization methods such as memetic algorithms and local search. In this part we focus on sampling and variation operators, approximation and surrogate modeling, initialization methods, and parallelization. We also cover a range of problem areas in relation to large-scale global optimization, such as multi-objective optimization, constraint handling, overlapping components, the component imbalance issue, and benchmarks, and applications. The paper also includes a discussion on pitfalls and challenges of current research and identifies several potential areas of future research

Sample size estimation for power and accuracy in the experimental comparison of algorithms

Experimental comparisons of performance represent an important aspect of research on optimization algorithms. In this work we present a methodology for defining the required sample sizes for designing experiments with desired statistical properties for the comparison of two methods on a given problem class. The proposed approach allows the experimenter to define desired levels of accuracy for estimates of mean performance differences on individual problem instances, as well as the desired statistical power for comparing mean performances over a problem class of interest. The method calculates the required number of problem instances, and runs the algorithms on each test instance so that the accuracy of the estimated differences in performance is controlled at the predefined level. Two examples illustrate the application of the proposed method, and its ability to achieve the desired statistical properties with a methodologically sound definition of the relevant sample sizes

Large scale estimation of distribution algorithms for continuous optimisation

Modern real world optimisation problems are increasingly becoming large scale. However, searching in high dimensional search spaces is notoriously difficult. Many methods break down as dimensionality increases and Estimation of Distribution Algorithm (EDA) is especially prone to the curse of dimensionality. In this thesis, we device new EDA variants that are capable of searching in large dimensional continuous domains. We in particular (i) investigated heavy tails search distributions, (ii) we clarify a controversy in the literature about the capabilities of Gaussian versus Cauchy search distributions, (iii) we constructed a new way of projecting a large dimensional search space to low dimensional subspaces in a way that gives us control of the size of covariance of the search distribution and we develop adaptation techniques to exploit this and (iv) we proposed a random embedding technique in EDA that takes advantage of low intrinsic dimensional structure of problems. All these developments avail us with new techniques to tackle high dimensional optimization problems

Hybrid tabu search – strawberry algorithm for multidimensional knapsack problem

Multidimensional Knapsack Problem (MKP) has been widely used to model real-life combinatorial problems. It is also used extensively in experiments to test the performances of metaheuristic algorithms and their hybrids. For example, Tabu Search (TS) has been successfully hybridized with other techniques, including particle swarm optimization (PSO) algorithm and the two-stage TS algorithm to solve MKP. In 2011, a new metaheuristic known as Strawberry algorithm (SBA) was initiated. Since then, it has been vastly applied to solve engineering problems. However, SBA has never been deployed to solve MKP. Therefore, a new hybrid of TS-SBA is proposed in this study to solve MKP with the objective of maximizing the total profit. The Greedy heuristics by ratio was employed to construct an initial solution. Next, the solution was enhanced by using the hybrid TS-SBA. The parameters setting to run the hybrid TS-SBA was determined by using a combination of Factorial Design of Experiments and Decision Tree Data Mining methods. Finally, the hybrid TS-SBA was evaluated using an MKP benchmark problem. It consisted of 270 test problems with different sizes of constraints and decision variables. The findings revealed that on average the hybrid TS-SBA was able to increase 1.97% profit of the initial solution. However, the best-known solution from past studies seemed to outperform the hybrid TS-SBA with an average difference of 3.69%. Notably, the novel hybrid TS-SBA proposed in this study may facilitate decisionmakers to solve real applications of MKP. It may also be applied to solve other variants of knapsack problems (KPs) with minor modifications

From Parameter Tuning to Dynamic Heuristic Selection

The importance of balance between exploration and exploitation plays a crucial role while solving combinatorial optimization problems. This balance is reached by two general techniques: by using an appropriate problem solver and by setting its proper parameters. Both problems were widely studied in the past and the research process continues up until now. The latest studies in the field of automated machine learning propose merging both problems, solving them at design time, and later strengthening the results at runtime. To the best of our knowledge, the generalized approach for solving the parameter setting problem in heuristic solvers has not yet been proposed. Therefore, the concept of merging heuristic selection and parameter control have not been introduced. In this thesis, we propose an approach for generic parameter control in meta-heuristics by means of reinforcement learning (RL). Making a step further, we suggest a technique for merging the heuristic selection and parameter control problems and solving them at runtime using RL-based hyper-heuristic. The evaluation of the proposed parameter control technique on a symmetric traveling salesman problem (TSP) revealed its applicability by reaching the performance of tuned in online and used in isolation underlying meta-heuristic. Our approach provides the results on par with the best underlying heuristics with tuned parameters.:1 Introduction 1 1.1 Motivation 1 1.2 Research objective 2 1.3 Solution overview 2 2 Background and RelatedWork Analysis 3 2.1 Optimization Problems and their Solvers 3 2.2 Heuristic Solvers for Optimization Problems 9 2.3 Setting Algorithm Parameters 19 2.4 Combined Algorithm Selection and Hyper-Parameter Tuning Problem 27 2.5 Conclusion on Background and Related Work Analysis 28 3 Online Selection Hyper-Heuristic with Generic Parameter Control 31 3.1 Combined Parameter Control and Algorithm Selection Problem 31 3.2 Search Space Structure 32 3.3 Parameter Prediction Process 34 3.4 Low-Level Heuristics 35 3.5 Conclusion of Concept 36 4 Implementation Details 37 4.2 Search Space 40 4.3 Prediction Process 43 4.4 Low Level Heuristics 48 4.5 Conclusion 52 5 Evaluation 55 5.1 Optimization Problem 55 5.2 Environment Setup 56 5.3 Meta-heuristics Tuning 56 5.4 Concept Evaluation 60 5.5 Analysis of HH-PC Settings 74 5.6 Conclusion 79 6 Conclusion 81 7 FutureWork 83 7.1 Prediction Process 83 7.2 Search Space 84 7.3 Evaluations and Benchmarks 84 Bibliography 87 A Evaluation Results 99 A.1 Results in Figures 99 A.2 Results in numbers 10

SCCharts: Language and Interactive Incremental Compilation

Safety-critical systems are a subclass of reactive systems, a dominating class of computer systems these days. Such systems control the airbags in our cars, the flaps of an aircraft, nuclear power plants or pace makers. Software for these systems must be reliable. Hence, a language and tooling is needed that allows to build and maintain reliable software models. Furthermore, a reliable compiler is required to obtain decent machine-understandable and executable code from highly abstract models. This thesis presents SCCharts, a Statecharts-based synchronous and visual modeling language for specifying and designing safety-critical systems and for deriving their implementations. It elaborates on why a control-flow oriented and synchronous language is desirable and how incremental language features are chosen to flatten learning curve. It presents an interactive incremental model transformation based compilation approach termed SLIC. It shows how SLIC helps in supporting both, the modeler and the tool smith for building reliable models and maintaining a reliable compiler, respectively. A SLIC-based compiler for SCCharts including its high-level model transformations is presented. Furthermore, practicality aspects of the KIELER SCCharts language and tooling implementation complete the considerations to validate the proposed approach.Sicherheitskritische Systeme sind eine Unterklasse von reaktiven Systemen, welche heutzutage eine der wichtigsten und größten Klasse von Computersystemen darstellt. Solche Systeme kontrollieren die Airbags unserer Autos, die Landeklappen eines Passagierflugzeugs, Kernkraftwerke oder Herzschrittmacher. Software für solche Systeme muß absolut zuverlässig sein. Daher werden Computersprachen und Werkzeuge benötigt, die es erlauben, zuverlässige Softwaremodelle zu erstellen und zu warten. Weiterhin braucht es zuverlässige Kompiler, die aus solchen abstrakten Modellen korrekten maschinenlesbaren und ausführbaren Code erzeugen. Mit SCCharts präsentiert diese Arbeit eine zustandsmaschinenbasierte und synchrone Modellierungssprache für den Entwurf und zur Implementierung sicherheitskritischer Systeme. Es wird betrachtet, warum sich dafür eine kontrollflußorientierte und synchrone Sprache besonders gut eignet und welche Wahl inkrementeller Sprachbestandteile die Lernkurve senken können. Die Arbeit zeigt, wie ein als SLIC bezeichneter, interaktiver, inkrementeller und auf Modelltransformationen basierender Kompilierungsansatz sowohl dem Modellierer dabei helfen kann, zuverlässige Modelle zu erstellen, als auch den Werkzeugentwickler darin unterstützt, einen zuverlässigen Kompiler bereit zu stellen. Es wird ein auf SLIC basierender SCCharts Kompiler inklusive seiner high-level Modelltransformationen vorgestellt. Weiterhin wird der vorgestellte Ansatz mit Hilfe der beispielhaft umgesetzten KIELER SCCharts Sprach- und Werkzeugimplementierung auf seine Praktikabilität hin überprüft

Utilizing commercial soil sensing technology for agronomic decisions

Planters with mounted proximal soil sensing systems can densely quantify seed zone soil variability. Technology now allows for real-time sensor information to control multiple row-unit functions on-the-go (e.g., planting depth). These and other developing sensor-based control systems have the potential to greatly improve correctness when planting, and therefore row-crop performance. For sensor-based control to be widely adopted, practitioners must understand the precision and utility of the systems. Therefore, research was conducted to: (i) determine how well commercially available sensors can estimate soil organic matter (OM) and whether sensor output was repeatable among sensing dates; (ii) evaluate OM prediction accuracy across selected soils and soil volumetric water contents with both a commercially-available, planter-mounted sensor, and machine learning techniques applied to multiple combinations of soil reflectance bands within the visible and near infrared spectrum; and (iii) investigate if planter and other proximal soil sensor data, in combination with topographic features, could predict field-scale corn emergence rate at varying planting depths. Results found that commercial sensors could estimate general trends in spatial variability of OM, but that some inconsistencies were associated with a "global" calibration that appeared susceptible to temporal variations in soil water content. In the controlled environment, results for sensor estimation of OM were similar to the field study. Further, results showed that spectral information within the entire range used by the commercial systems evaluated was required to consistently predict OM at varying volumetric water contents. Lastly, the field-scale agronomic analysis found that inherent soil and landscape variability drove the emergence rate response at the site. However, planter metrics were still usefulIncludes bibliographical references

Review, challenges, design, and development

Peres, F., & Castelli, M. (2021). Combinatorial optimization problems and metaheuristics: Review, challenges, design, and development. Applied Sciences (Switzerland), 11(14), 1-39. [6449]. https://doi.org/10.3390/app11146449In the past few decades, metaheuristics have demonstrated their suitability in addressing complex problems over different domains. This success drives the scientific community towards the definition of new and better-performing heuristics and results in an increased interest in this research field. Nevertheless, new studies have been focused on developing new algorithms without providing consolidation of the existing knowledge. Furthermore, the absence of rigor and formalism to classify, design, and develop combinatorial optimization problems and metaheuristics represents a challenge to the field’s progress. This study discusses the main concepts and challenges in this area and proposes a formalism to classify, design, and code combinatorial optimization problems and metaheuristics. We believe these contributions may support the progress of the field and increase the maturity of metaheuristics as problem solvers analogous to other machine learning algorithms.publishersversionpublishe

Soft computing applied to optimization, computer vision and medicine

Artificial intelligence has permeated almost every area of life in modern society, and its significance continues to grow. As a result, in recent years, Soft Computing has emerged as a powerful set of methodologies that propose innovative and robust solutions to a variety of complex problems. Soft Computing methods, because of their broad range of application, have the potential to significantly improve human living conditions. The motivation for the present research emerged from this background and possibility. This research aims to accomplish two main objectives: On the one hand, it endeavors to bridge the gap between Soft Computing techniques and their application to intricate problems. On the other hand, it explores the hypothetical benefits of Soft Computing methodologies as novel effective tools for such problems. This thesis synthesizes the results of extensive research on Soft Computing methods and their applications to optimization, Computer Vision, and medicine. This work is composed of several individual projects, which employ classical and new optimization algorithms. The manuscript presented here intends to provide an overview of the different aspects of Soft Computing methods in order to enable the reader to reach a global understanding of the field. Therefore, this document is assembled as a monograph that summarizes the outcomes of these projects across 12 chapters. The chapters are structured so that they can be read independently. The key focus of this work is the application and design of Soft Computing approaches for solving problems in the following: Block Matching, Pattern Detection, Thresholding, Corner Detection, Template Matching, Circle Detection, Color Segmentation, Leukocyte Detection, and Breast Thermogram Analysis. One of the outcomes presented in this thesis involves the development of two evolutionary approaches for global optimization. These were tested over complex benchmark datasets and showed promising results, thus opening the debate for future applications. Moreover, the applications for Computer Vision and medicine presented in this work have highlighted the utility of different Soft Computing methodologies in the solution of problems in such subjects. A milestone in this area is the translation of the Computer Vision and medical issues into optimization problems. Additionally, this work also strives to provide tools for combating public health issues by expanding the concepts to automated detection and diagnosis aid for pathologies such as Leukemia and breast cancer. The application of Soft Computing techniques in this field has attracted great interest worldwide due to the exponential growth of these diseases. Lastly, the use of Fuzzy Logic, Artificial Neural Networks, and Expert Systems in many everyday domestic appliances, such as washing machines, cookers, and refrigerators is now a reality. Many other industrial and commercial applications of Soft Computing have also been integrated into everyday use, and this is expected to increase within the next decade. Therefore, the research conducted here contributes an important piece for expanding these developments. The applications presented in this work are intended to serve as technological tools that can then be used in the development of new devices