Improving (1+1) Covariance Matrix Adaptation Evolution Strategy: a simple yet efficient approach

The file attached to this record is the author's final peer reviewed versionIn recent years, part of the meta-heuristic optimisation research community has called for a simplification of the algorithmic design: indeed, while most of the state-of-the-art algorithms are characterised by a high level of complexity, complex algorithms are hard to understand and therefore tune for specific real-world applications. Here, we follow this reductionist approach by combining straightforwardly two methods recently proposed in the literature, namely the Re-sampling Inheritance Search (RIS) and the (1+1) Covariance Matrix Adaptation Evolution Strategy (CMA-ES). We propose an RI-(1+1)-CMA-ES algorithm that on the one hand improves upon the original (1+1)-CMA-ES, on the other it keeps the original spirit of simplicity at the basis of RIS. We show with an extensive experimental campaign that the proposed algorithm efficiently solves a large number of benchmark functions and is competitive with several modern optimisation algorithms much more complex in terms of algorithmic design

Novel Memetic Computing Structures for Continuous Optimisation

This thesis studies a class of optimisation algorithms, namely Memetic Computing Structures, and proposes a novel set of promising algorithms that move the first step towards an implementation for the automatic generation of optimisation algorithms for continuous domains. This thesis after a thorough review of local search algorithms and popular meta-heuristics, focuses on Memetic Computing in terms of algorithm structures and design philosophy. In particular, most of the design carried out during my doctoral studies is inspired by the lex parsimoniae, aka Ockham’s Razor. It has been shown how simple algorithms, when well implemented can outperform complex implementations. In order to achieve this aim, the design is always carried out by attempting to identify the role of each algorithmic component/operator. In this thesis, on the basis of this logic, a set of variants of a recently proposed algorithms are presented. Subsequently a novel memetic structure, namely Parallel Memetic Structure is proposed and tested against modern algorithms representing the state of the art in optimisation. Furthermore, an initial prototype of an automatic design platform is also included. This prototype performs an analysis on separability of the optimisation problem and, on the basis of the analysis results, designs some parts of the parallel structure. Promising results are included. Finally, an investigation of the correlation among the variables and problem dimensionality has been performed. An extremely interesting finding of this thesis work is that the degree of correlation among the variables decreases when the dimensionality increases. As a direct consequence of this fact, large scale problems are to some extent easier to handle than problems in low dimensionality since, due to the lack of correlation among the variables, they can effectively be tackled by an algorithm that performs moves along the axes

Compact Optimization Algorithms with Re-sampled Inheritance

The file attached to this record is the author's final peer reviewed version.Compact optimization algorithms are a class of Estimation of Distribution Algorithms (EDAs) characterized by extremely limited memory requirements (hence they are called \compact"). As all EDAs, compact algorithms build and update a probabilistic model of the distribution of solutions within the search space, as opposed to population-based algorithms that instead make use of an explicit population of solutions. In addition to that, to keep their memory consumption low, compact algorithms purposely employ simple probabilistic models that can be described with a small number of parameters. Despite their simplicity, compact algorithms have shown good performances on a broad range of benchmark functions and real-world problems. However, compact algorithms also come with some drawbacks, i.e. they tend to premature convergence and show poorer performance on non-separable problems. To overcome these limitations, here we investigate a possible algorithmic scheme obtained by combining compact algorithms with a non-disruptive restart mechanism taken from the literature, named Re-Sampled Inheritance (RI). The resulting compact algorithms with RI are tested on the CEC 2014 benchmark functions. The numerical results show on the one hand that the use of RI consistently enhances the performances of compact algorithms, still keeping a limited usage of memory. On the other hand, our experiments show that among the tested algorithms, the best performance is obtained by compact Differential Evolution with RI

Swarm Robotics

This study analyzes and designs the Swarm intelligence (SI) that Self-organizing migrating algorithm (SOMA) represents to solve industrial practice as well as academic optimization problems, and applies them to swarm robotics. Specifically, the characteristics of SOMA are clarified, shaping the basis for the analysis of SOMA's strengths and weaknesses for the release of SOMA T3A, SOMA Pareto, and iSOMA, with outstanding performance, confirmed by well-known test suites from IEEE CEC 2013, 2015, 2017, and 2019. Besides, the dynamic path planning problem for swarm robotics is handled by the proposed algorithms considered as a prime instance. The computational and simulation results on Matlab have proven the performance of the novel algorithms as well as the correctness of the obstacle avoidance method for mobile robots and drones. Furthermore, two out of the three proposed versions achieved the tie for 3rd (the same ranking with HyDE-DF) and 5th place in the 100-Digit Challenge at CEC 2019, GECCO 2019, and SEMCCO 2019 competition, something that any other version of SOMA has yet to do. They show promising possibilities that SOMA and SI algorithms offer.Tato práce se zabývá analýzou a vylepšením hejnové inteligence, kterou představuje samoorganizující se migrační algoritmus s možností využití v průmyslové praxi a se zaměřením na hejnovou robotiku. Je analyzován algoritmus SOMA, identifikovány silné a slabé stránky a navrženy nové verze SOMA jako SOMA T3A, SOMA Pareto, iSOMA s vynikajícím výkonem, potvrzeným známými testovacími sadami IEEE CEC 2013, 2015, 2017 a 2019. Tyto verze jsou pak aplikovány na problém s dynamickým plánováním dráhy pro hejnovou robotiku. Výsledky výpočtů a simulace v Matlabu prokázaly výkonnost nových algoritmů a správnost metody umožňující vyhýbání se překážkám u mobilních robotů a dronů. Kromě toho dvě ze tří navržených verzí dosáhly na 3. a 5. místo v soutěži 100-Digit Challenge na CEC 2019, GECCO 2019 a SEMCCO 2019, což je potvrzení navržených inovací. Práce tak demonstruje nejen vylepšení SOMA, ale i slibné možnosti hejnové inteligence.460 - Katedra informatikyvyhově

Differential evolution with an individual-dependent mechanism

Differential evolution (DE) is a well-known optimization algorithm that utilizes the difference of positions between individuals to perturb base vectors and thus generate new mutant individuals. However, the difference between the fitness values of individuals, which may be helpful to improve the performance of the algorithm, has not been used to tune parameters and choose mutation strategies. In this paper, we propose a novel variant of DE with an individual-dependent mechanism that includes an individual-dependent parameter (IDP) setting and an individual-dependent mutation (IDM) strategy. In the IDP setting, control parameters are set for individuals according to the differences in their fitness values. In the IDM strategy, four mutation operators with different searching characteristics are assigned to the superior and inferior individuals, respectively, at different stages of the evolution process. The performance of the proposed algorithm is then extensively evaluated on a suite of the 28 latest benchmark functions developed for the 2013 Congress on Evolutionary Computation special session. Experimental results demonstrate the algorithm's outstanding performance

A New Optimization Algorithm Based on Search and Rescue Operations

[EN] In this paper, a new optimization algorithm called the search and rescue optimization algorithm (SAR) is proposed for solving single-objective continuous optimization problems. SAR is inspired by the explorations carried out by humans during search and rescue operations. The performance of SAR was evaluated on fifty-five optimization functions including a set of classic benchmark functions and a set of modern CEC 2013 benchmark functions from the literature. The obtained results were compared with twelve optimization algorithms including well-known optimization algorithms, recent variants of GA, DE, CMA-ES, and PSO, and recent metaheuristic algorithms. The Wilcoxon signed-rank test was used for some of the comparisons, and the convergence behavior of SAR was investigated. The statistical results indicated SAR is highly competitive with the compared algorithms. Also, in order to evaluate the application of SAR on real-world optimization problems, it was applied to three engineering design problems, and the results revealed that SAR is able to find more accurate solutions with fewer function evaluations in comparison with the other existing algorithms. Thus, the proposed algorithm can be considered an efficient optimization method for real-world optimization problems.This study was partially supported by the Spanish Research Project (nos. TIN2016-80856-R and TIN2015-65515-C4-1-R).Shabani, A.; Asgarian, B.; Gharebaghi, SA.; Salido Gregorio, MA.; Giret Boggino, AS. (2019). A New Optimization Algorithm Based on Search and Rescue Operations. Mathematical Problems in Engineering. 2019:1-23. https://doi.org/10.1155/2019/2482543S1232019Bianchi, L., Dorigo, M., Gambardella, L. M., & Gutjahr, W. J. (2008). A survey on metaheuristics for stochastic combinatorial optimization. Natural Computing, 8(2), 239-287. doi:10.1007/s11047-008-9098-4Holland, J. H. (1992). Genetic Algorithms. Scientific American, 267(1), 66-72. doi:10.1038/scientificamerican0792-66Dorigo, M., Maniezzo, V., & Colorni, A. (1996). Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 26(1), 29-41. doi:10.1109/3477.484436Manjarres, D., Landa-Torres, I., Gil-Lopez, S., Del Ser, J., Bilbao, M. N., Salcedo-Sanz, S., & Geem, Z. W. (2013). A survey on applications of the harmony search algorithm. Engineering Applications of Artificial Intelligence, 26(8), 1818-1831. doi:10.1016/j.engappai.2013.05.008Karaboga, D., Gorkemli, B., Ozturk, C., & Karaboga, N. (2012). A comprehensive survey: artificial bee colony (ABC) algorithm and applications. Artificial Intelligence Review, 42(1), 21-57. doi:10.1007/s10462-012-9328-0Rao, R. V., Savsani, V. J., & Vakharia, D. P. (2011). Teaching–learning-based optimization: A novel method for constrained mechanical design optimization problems. Computer-Aided Design, 43(3), 303-315. doi:10.1016/j.cad.2010.12.015Zhang, C., Lin, Q., Gao, L., & Li, X. (2015). Backtracking Search Algorithm with three constraint handling methods for constrained optimization problems. Expert Systems with Applications, 42(21), 7831-7845. doi:10.1016/j.eswa.2015.05.050Yang, X. S. (2010). Firefly algorithm, stochastic test functions and design optimisation. International Journal of Bio-Inspired Computation, 2(2), 78. doi:10.1504/ijbic.2010.032124Punnathanam, V., & Kotecha, P. (2016). Yin-Yang-pair Optimization: A novel lightweight optimization algorithm. Engineering Applications of Artificial Intelligence, 54, 62-79. doi:10.1016/j.engappai.2016.04.004Zhao, C., Wu, C., Chai, J., Wang, X., Yang, X., Lee, J.-M., & Kim, M. J. (2017). Decomposition-based multi-objective firefly algorithm for RFID network planning with uncertainty. Applied Soft Computing, 55, 549-564. doi:10.1016/j.asoc.2017.02.009Zhao, C., Wu, C., Wang, X., Ling, B. W.-K., Teo, K. L., Lee, J.-M., & Jung, K.-H. (2017). Maximizing lifetime of a wireless sensor network via joint optimizing sink placement and sensor-to-sink routing. Applied Mathematical Modelling, 49, 319-337. doi:10.1016/j.apm.2017.05.001Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67-82. doi:10.1109/4235.585893Simon, D. (2008). Biogeography-Based Optimization. IEEE Transactions on Evolutionary Computation, 12(6), 702-713. doi:10.1109/tevc.2008.919004Garg, H. (2015). An efficient biogeography based optimization algorithm for solving reliability optimization problems. Swarm and Evolutionary Computation, 24, 1-10. doi:10.1016/j.swevo.2015.05.001Storn, R., & Price, K. (1997). Journal of Global Optimization, 11(4), 341-359. doi:10.1023/a:1008202821328Das, S., Mullick, S. S., & Suganthan, P. N. (2016). Recent advances in differential evolution – An updated survey. Swarm and Evolutionary Computation, 27, 1-30. doi:10.1016/j.swevo.2016.01.004Couzin, I. D., Krause, J., Franks, N. R., & Levin, S. A. (2005). Effective leadership and decision-making in animal groups on the move. Nature, 433(7025), 513-516. doi:10.1038/nature03236Gandomi, A. H., & Alavi, A. H. (2012). Krill herd: A new bio-inspired optimization algorithm. Communications in Nonlinear Science and Numerical Simulation, 17(12), 4831-4845. doi:10.1016/j.cnsns.2012.05.010Mirjalili, S., Mirjalili, S. M., & Lewis, A. (2014). Grey Wolf Optimizer. Advances in Engineering Software, 69, 46-61. doi:10.1016/j.advengsoft.2013.12.007Erol, O. K., & Eksin, I. (2006). A new optimization method: Big Bang–Big Crunch. Advances in Engineering Software, 37(2), 106-111. doi:10.1016/j.advengsoft.2005.04.005Kaveh, A., & Mahdavi, V. R. (2014). Colliding bodies optimization: A novel meta-heuristic method. Computers & Structures, 139, 18-27. doi:10.1016/j.compstruc.2014.04.005Rashedi, E., Nezamabadi-pour, H., & Saryazdi, S. (2009). GSA: A Gravitational Search Algorithm. Information Sciences, 179(13), 2232-2248. doi:10.1016/j.ins.2009.03.004Zheng, Y.-J. (2015). Water wave optimization: A new nature-inspired metaheuristic. Computers & Operations Research, 55, 1-11. doi:10.1016/j.cor.2014.10.008Kaveh, A., & Khayatazad, M. (2012). A new meta-heuristic method: Ray Optimization. Computers & Structures, 112-113, 283-294. doi:10.1016/j.compstruc.2012.09.003Glover, F. (1989). Tabu Search—Part I. ORSA Journal on Computing, 1(3), 190-206. doi:10.1287/ijoc.1.3.190Chiang, H.-P., Chou, Y.-H., Chiu, C.-H., Kuo, S.-Y., & Huang, Y.-M. (2013). A quantum-inspired Tabu search algorithm for solving combinatorial optimization problems. Soft Computing, 18(9), 1771-1781. doi:10.1007/s00500-013-1203-7Mousavirad, S. J., & Ebrahimpour-Komleh, H. (2017). Human mental search: a new population-based metaheuristic optimization algorithm. Applied Intelligence, 47(3), 850-887. doi:10.1007/s10489-017-0903-6Karaboga, D., & Basturk, B. (2007). A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. Journal of Global Optimization, 39(3), 459-471. doi:10.1007/s10898-007-9149-xRao, R. V., Savsani, V. J., & Vakharia, D. P. (2012). Teaching–Learning-Based Optimization: An optimization method for continuous non-linear large scale problems. Information Sciences, 183(1), 1-15. doi:10.1016/j.ins.2011.08.006Digalakis, J. G., & Margaritis, K. G. (2001). On benchmarking functions for genetic algorithms. International Journal of Computer Mathematics, 77(4), 481-506. doi:10.1080/00207160108805080Karaboga, D., & Akay, B. (2009). A comparative study of Artificial Bee Colony algorithm. Applied Mathematics and Computation, 214(1), 108-132. doi:10.1016/j.amc.2009.03.090Lim, T. Y., Al-Betar, M. A., & Khader, A. T. (2015). Adaptive pair bonds in genetic algorithm: An application to real-parameter optimization. Applied Mathematics and Computation, 252, 503-519. doi:10.1016/j.amc.2014.12.030Fleury, C., & Braibant, V. (1986). Structural optimization: A new dual method using mixed variables. International Journal for Numerical Methods in Engineering, 23(3), 409-428. doi:10.1002/nme.1620230307Derrac, J., García, S., Molina, D., & Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation, 1(1), 3-18. doi:10.1016/j.swevo.2011.02.002Gandomi, A. H., Yang, X.-S., & Alavi, A. H. (2011). Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Engineering with Computers, 29(1), 17-35. doi:10.1007/s00366-011-0241-yWang, G. G. (2003). Adaptive Response Surface Method Using Inherited Latin Hypercube Design Points. Journal of Mechanical Design, 125(2), 210-220. doi:10.1115/1.1561044Cheng, M.-Y., & Prayogo, D. (2014). Symbiotic Organisms Search: A new metaheuristic optimization algorithm. Computers & Structures, 139, 98-112. doi:10.1016/j.compstruc.2014.03.007CHICKERMANE, H., & GEA, H. C. (1996). STRUCTURAL OPTIMIZATION USING A NEW LOCAL APPROXIMATION METHOD. International Journal for Numerical Methods in Engineering, 39(5), 829-846. doi:10.1002/(sici)1097-0207(19960315)39:53.0.co;2-uChou, J.-S., & Ngo, N.-T. (2016). Modified firefly algorithm for multidimensional optimization in structural design problems. Structural and Multidisciplinary Optimization, 55(6), 2013-2028. doi:10.1007/s00158-016-1624-xSonmez, M. (2011). Artificial Bee Colony algorithm for optimization of truss structures. Applied Soft Computing, 11(2), 2406-2418. doi:10.1016/j.asoc.2010.09.003Degertekin, S. O. (2012). Improved harmony search algorithms for sizing optimization of truss structures. Computers & Structures, 92-93, 229-241. doi:10.1016/j.compstruc.2011.10.022Degertekin, S. O., & Hayalioglu, M. S. (2013). Sizing truss structures using teaching-learning-based optimization. Computers & Structures, 119, 177-188. doi:10.1016/j.compstruc.2012.12.011Talatahari, S., Kheirollahi, M., Farahmandpour, C., & Gandomi, A. H. (2012). A multi-stage particle swarm for optimum design of truss structures. Neural Computing and Applications, 23(5), 1297-1309. doi:10.1007/s00521-012-1072-5Kaveh, A., Bakhshpoori, T., & Afshari, E. (2014). An efficient hybrid Particle Swarm and Swallow Swarm Optimization algorithm. Computers & Structures, 143, 40-59. doi:10.1016/j.compstruc.2014.07.012Kaveh, A., & Bakhshpoori, T. (2016). A new metaheuristic for continuous structural optimization: water evaporation optimization. Structural and Multidisciplinary Optimization, 54(1), 23-43. doi:10.1007/s00158-015-1396-8Jalili, S., & Hosseinzadeh, Y. (2015). A Cultural Algorithm for Optimal Design of Truss Structures. Latin American Journal of Solids and Structures, 12(9), 1721-1747. doi:10.1590/1679-7825154

Evolutionary framework with reinforcement learning-based mutation adaptation

Although several multi-operator and multi-method approaches for solving optimization problems have been proposed, their performances are not consistent for a wide range of optimization problems. Also, the task of ensuring the appropriate selection of algorithms and operators may be inefficient since their designs are undertaken mainly through trial and error. This research proposes an improved optimization framework that uses the benefits of multiple algorithms, namely, a multi-operator differential evolution algorithm and a co-variance matrix adaptation evolution strategy. In the former, reinforcement learning is used to automatically choose the best differential evolution operator. To judge the performance of the proposed framework, three benchmark sets of bound-constrained optimization problems (73 problems) with 10, 30 and 50 dimensions are solved. Further, the proposed algorithm has been tested by solving optimization problems with 100 dimensions taken from CEC2014 and CEC2017 benchmark problems. A real-world application data set has also been solved. Several experiments are designed to analyze the effects of different components of the proposed framework, with the best variant compared with a number of state-of-the-art algorithms. The experimental results show that the proposed algorithm is able to outperform all the others considered.</p

Inferring Human Pose and Motion from Images

As optical gesture recognition technology advances, touchless human computer interfaces of the future will soon become a reality. One particular technology, markerless motion capture, has gained a large amount of attention, with widespread application in diverse disciplines, including medical science, sports analysis, advanced user interfaces, and virtual arts. However, the complexity of human anatomy makes markerless motion capture a non-trivial problem: I) parameterised pose configuration exhibits high dimensionality, and II) there is considerable ambiguity in surjective inverse mapping from observation to pose configuration spaces with a limited number of camera views. These factors together lead to multimodality in high dimensional space, making markerless motion capture an ill-posed problem. This study challenges these difficulties by introducing a new framework. It begins with automatically modelling specific subject template models and calibrating posture at the initial stage. Subsequent tracking is accomplished by embedding naturally-inspired global optimisation into the sequential Bayesian filtering framework. Tracking is enhanced by several robust evaluation improvements. Sparsity of images is managed by compressive evaluation, further accelerating computational efficiency in high dimensional space

Generalised Pattern Search with Restarting Fitness Landscape Analysis

Fitness landscape analysis for optimisation is a technique that involves analysing black-box optimisation problems to extract pieces of information about the problem, which can beneficially inform the design of the optimiser. Thus, the design of the algorithm aims to address the specific features detected during the analysis of the problem. Similarly, the designer aims to understand the behaviour of the algorithm, even though the problem is unknown and the optimisation is performed via a metaheuristic method. Thus, the algorithmic design made using fitness landscape analysis can be seen as an example of explainable AI in the optimisation domain. The present paper proposes a framework that performs fitness landscape analysis and designs a Pattern Search (PS) algorithm on the basis of the results of the analysis. The algorithm is implemented in a restarting fashion: at each restart, the fitness landscape analysis refines the analysis of the problem and updates the pattern matrix used by PS. A computationally efficient implementation is also presented in this study. Numerical results show that the proposed framework clearly outperforms standard PS and another PS implementation based on fitness landscape analysis. Furthermore, the two instances of the proposed framework considered in this study are competitive with popular algorithms present in the literature