Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities

Evolutionary algorithms (EA), a class of stochastic search methods based on the principles of natural evolution, have received widespread acclaim for their exceptional performance in various real-world optimization problems. While researchers worldwide have proposed a wide variety of EAs, certain limitations remain, such as slow convergence speed and poor generalization capabilities. Consequently, numerous scholars actively explore improvements to algorithmic structures, operators, search patterns, etc., to enhance their optimization performance. Reinforcement learning (RL) integrated as a component in the EA framework has demonstrated superior performance in recent years. This paper presents a comprehensive survey on integrating reinforcement learning into the evolutionary algorithm, referred to as reinforcement learning-assisted evolutionary algorithm (RL-EA). We begin with the conceptual outlines of reinforcement learning and the evolutionary algorithm. We then provide a taxonomy of RL-EA. Subsequently, we discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. The RL-assisted procedure is divided according to the implemented functions including solution generation, learnable objective function, algorithm/operator/sub-population selection, parameter adaptation, and other strategies. Finally, we analyze potential directions for future research. This survey serves as a rich resource for researchers interested in RL-EA as it overviews the current state-of-the-art and highlights the associated challenges. By leveraging this survey, readers can swiftly gain insights into RL-EA to develop efficient algorithms, thereby fostering further advancements in this emerging field.Comment: 26 pages, 16 figure

A Keyword, Taxonomy and Cartographic Research Review of Sustainability Concepts for Production Scheduling in Manufacturing Systems

The concept of sustainability is defined as composed of three pillars: social, environmental, and economic. Social sustainability implies a commitment to equity in terms of several “interrelated and mutually supportive” principles of a “sustainable society”; this concept includes attitude change, the Earth’s vitality and diversity conservation, and a global alliance to achieve sustainability. The social and environmental aspects of sustainability are related in the way sustainability indicators are related to “quality of life” and “ecological sustainability”. The increasing interest in green and sustainable products and production has influenced research interests regarding sustainable scheduling problems in manufacturing systems. This study is aimed both at reducing pollutant emissions and increasing production efficiency: this topic is known as Green Scheduling. Existing literature research reviews on Green Scheduling Problems have pointed out both theoretical and practical aspects of this topic. The proposed work is a critical review of the scientific literature with a three-pronged approach based on keywords, taxonomy analysis, and research mapping. Specific research questions have been proposed to highlight the benefits and related objectives of this review: to discover the most widely used methodologies for solving SPGs in manufacturing and identify interesting development models, as well as the least studied domains and algorithms. The literature was analysed in order to define a map of the main research fields on SPG, highlight mainstream SPG research, propose an efficient view of emerging research areas, propose a taxonomy of SPG by collecting multiple keywords into semantic clusters, and analyse the literature according to a semantic knowledge approach. At the same time, GSP researchers are provided with an efficient view of emerging research areas, allowing them to avoid missing key research areas and focus on emerging ones

Analýza dynamiky evolučních algoritmů pomocí komplexních sítí aplikovaných na kombinatorické optimalizační problémy

Import 05/08/2014This thesis explores the connection between Evolutionary Algorithms (EA's) and Complex Networks (CN's). EA's are bio-inspired algorithms which mimic naturally occurring phenomena in order to model and solve complex engineering tasks. One of its features is its population based paradigm. The behaviour of the population over the iterations is analysed in this thesis using CN analysis tools. Four distinct broad attributes are analysed; adjacency matrix, centralities, cliques and communities. Using these attributes, a number of experimentations and analysis were conducted, from which interesting information regarding population development, stagnation, network interconnection and hierarchical development was obtained. These data supported the concept of population dynamics and furthermore could be used for population and evolution control.Tato práce se zabývá spojením mezi evolučními algoritmy (EA) a komplexnímí sítěmi (KS). EA jsou biologicky inspirované algoritmy napodobující přirozené přírodní jevy s cílem modelovat a řešit složité technické problémy. Jednou z jejich funkcí je snaha napodobit evoluční dogma. Chování celé populace je skrze její vývoj sledováno pomocí nástrojů pro analýzu komplexních sítí. Analyzovány jsou tyto čtyři atributy: matice sjednocení, centralita, kliky a komunity. Byla provedena řada experimentů a analýz, ze kterých pomocí těchto atributů, byly získány zajímavé informace týkající se vývoje populace, stagnace, propojení a hierarchie. Získaná data nastínila koncepci populační dynamiky a dala by se využít ke kontrole samotné evoluce.440 - Katedra telekomunikační technikyvýborn

Evolutionary Computation 2020

Intelligent optimization is based on the mechanism of computational intelligence to refine a suitable feature model, design an effective optimization algorithm, and then to obtain an optimal or satisfactory solution to a complex problem. Intelligent algorithms are key tools to ensure global optimization quality, fast optimization efficiency and robust optimization performance. Intelligent optimization algorithms have been studied by many researchers, leading to improvements in the performance of algorithms such as the evolutionary algorithm, whale optimization algorithm, differential evolution algorithm, and particle swarm optimization. Studies in this arena have also resulted in breakthroughs in solving complex problems including the green shop scheduling problem, the severe nonlinear problem in one-dimensional geodesic electromagnetic inversion, error and bug finding problem in software, the 0-1 backpack problem, traveler problem, and logistics distribution center siting problem. The editors are confident that this book can open a new avenue for further improvement and discoveries in the area of intelligent algorithms. The book is a valuable resource for researchers interested in understanding the principles and design of intelligent algorithms

Application of lean scheduling and production control in non-repetitive manufacturing systems using intelligent agent decision support

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Lean Manufacturing (LM) is widely accepted as a world-class manufacturing paradigm, its currency and superiority are manifested in numerous recent success stories. Most lean tools including Just-in-Time (JIT) were designed for repetitive serial production systems. This resulted in a substantial stream of research which dismissed a priori the suitability of LM for non-repetitive non-serial job-shops. The extension of LM into non-repetitive production systems is opposed on the basis of the sheer complexity of applying JIT pull production control in non-repetitive systems fabricating a high variety of products. However, the application of LM in job-shops is not unexplored. Studies proposing the extension of leanness into non-repetitive production systems have promoted the modification of pull control mechanisms or reconfiguration of job-shops into cellular manufacturing systems. This thesis sought to address the shortcomings of the aforementioned approaches. The contribution of this thesis to knowledge in the field of production and operations management is threefold: Firstly, a Multi-Agent System (MAS) is designed to directly apply pull production control to a good approximation of a real-life job-shop. The scale and complexity of the developed MAS prove that the application of pull production control in non-repetitive manufacturing systems is challenging, perplex and laborious. Secondly, the thesis examines three pull production control mechanisms namely, Kanban, Base Stock and Constant Work-in-Process (CONWIP) which it enhances so as to prevent system deadlocks, an issue largely unaddressed in the relevant literature. Having successfully tested the transferability of pull production control to non-repetitive manufacturing, the third contribution of this thesis is that it uses experimental and empirical data to examine the impact of pull production control on job-shop performance. The thesis identifies issues resulting from the application of pull control in job-shops which have implications for industry practice and concludes by outlining further research that can be undertaken in this direction

Multi-objective pareto ant colony system based algorithm for generator maintenance scheduling

Existing multi-objective Generator Maintenance Scheduling (GMS) models have considered unit commitment problem together with unit maintenance problem based on a periodic maintenance strategy. These models are inefficient because unit commitment does not undergo maintenance and periodic strategy cannot be applied on different types of generators. Present graph models cannot generate schedule for the multi-objective GMS models while existing Pareto Ant Colony System (PACS) algorithms were not able to consider the two problems separately. A multi-objective PACS algorithm based on sequential strategy which considers unit commitment and GMS problem separately is proposed to obtain solution for a proposed GMS model. A graph model is developed to generate the units’ maintenance schedule. The Taguchi and Grey Relational Analysis methods are proposed to tune the PACS’s parameters. The IEEE RTS 26, 32 and 36-unit dataset systems were used in the performance evaluation of the PACS algorithm. The performance of PACS algorithm was compared against four benchmark multi-objective algorithms including the Nondominated Sorting Genetic, Strength Pareto Evolutionary, Simulated Annealing, and Particle Swarm Optimization using the metrics grey relational grade (GRG), coverage, distance to Pareto front, Pareto spread, and number of non-dominated solutions. Friedman test was performed to determine the significance of the results. The multiobjective GMS model is superior than the benchmark model in producing the GMS schedule in terms of reliability, and violation objective functions with an average improvement between 2.68% and 92.44%. Friedman test using GRG metric shows significant better performance (p-values<0.05) for PACS algorithm compared to benchmark algorithms. The proposed models and algorithm can be used to solve the multi-objective GMS problem while the new parameters’ values can be used to obtain optimal or near optimal maintenance scheduling of generators. The proposed models and algorithm can be applied on different types of generating units to minimize the interruptions of energy and extend their lifespan

Advances in Condition Monitoring, Optimization and Control for Complex Industrial Processes

The book documents 25 papers collected from the Special Issue “Advances in Condition Monitoring, Optimization and Control for Complex Industrial Processes”, highlighting recent research trends in complex industrial processes. The book aims to stimulate the research field and be of benefit to readers from both academic institutes and industrial sectors

From metaheuristics to learnheuristics: Applications to logistics, finance, and computing

Un gran nombre de processos de presa de decisions en sectors estratègics com el transport i la producció representen problemes NP-difícils. Sovint, aquests processos es caracteritzen per alts nivells d'incertesa i dinamisme. Les metaheurístiques són mètodes populars per a resoldre problemes d'optimització difícils en temps de càlcul raonables. No obstant això, sovint assumeixen que els inputs, les funcions objectiu, i les restriccions són deterministes i conegudes. Aquests constitueixen supòsits forts que obliguen a treballar amb problemes simplificats. Com a conseqüència, les solucions poden conduir a resultats pobres. Les simheurístiques integren la simulació a les metaheurístiques per resoldre problemes estocàstics d'una manera natural. Anàlogament, les learnheurístiques combinen l'estadística amb les metaheurístiques per fer front a problemes en entorns dinàmics, en què els inputs poden dependre de l'estructura de la solució. En aquest context, les principals contribucions d'aquesta tesi són: el disseny de les learnheurístiques, una classificació dels treballs que combinen l'estadística / l'aprenentatge automàtic i les metaheurístiques, i diverses aplicacions en transport, producció, finances i computació.Un gran número de procesos de toma de decisiones en sectores estratégicos como el transporte y la producción representan problemas NP-difíciles. Frecuentemente, estos problemas se caracterizan por altos niveles de incertidumbre y dinamismo. Las metaheurísticas son métodos populares para resolver problemas difíciles de optimización de manera rápida. Sin embargo, suelen asumir que los inputs, las funciones objetivo y las restricciones son deterministas y se conocen de antemano. Estas fuertes suposiciones conducen a trabajar con problemas simplificados. Como consecuencia, las soluciones obtenidas pueden tener un pobre rendimiento. Las simheurísticas integran simulación en metaheurísticas para resolver problemas estocásticos de una manera natural. De manera similar, las learnheurísticas combinan aprendizaje estadístico y metaheurísticas para abordar problemas en entornos dinámicos, donde los inputs pueden depender de la estructura de la solución. En este contexto, las principales aportaciones de esta tesis son: el diseño de las learnheurísticas, una clasificación de trabajos que combinan estadística / aprendizaje automático y metaheurísticas, y varias aplicaciones en transporte, producción, finanzas y computación.A large number of decision-making processes in strategic sectors such as transport and production involve NP-hard problems, which are frequently characterized by high levels of uncertainty and dynamism. Metaheuristics have become the predominant method for solving challenging optimization problems in reasonable computing times. However, they frequently assume that inputs, objective functions and constraints are deterministic and known in advance. These strong assumptions lead to work on oversimplified problems, and the solutions may demonstrate poor performance when implemented. Simheuristics, in turn, integrate simulation into metaheuristics as a way to naturally solve stochastic problems, and, in a similar fashion, learnheuristics combine statistical learning and metaheuristics to tackle problems in dynamic environments, where inputs may depend on the structure of the solution. The main contributions of this thesis include (i) a design for learnheuristics; (ii) a classification of works that hybridize statistical and machine learning and metaheuristics; and (iii) several applications for the fields of transport, production, finance and computing

3rd Many-core Applications Research Community (MARC) Symposium. (KIT Scientific Reports ; 7598)

This manuscript includes recent scientific work regarding the Intel Single Chip Cloud computer and describes approaches for novel approaches for programming and run-time organization