Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities

Evolutionary algorithms (EA), a class of stochastic search methods based on the principles of natural evolution, have received widespread acclaim for their exceptional performance in various real-world optimization problems. While researchers worldwide have proposed a wide variety of EAs, certain limitations remain, such as slow convergence speed and poor generalization capabilities. Consequently, numerous scholars actively explore improvements to algorithmic structures, operators, search patterns, etc., to enhance their optimization performance. Reinforcement learning (RL) integrated as a component in the EA framework has demonstrated superior performance in recent years. This paper presents a comprehensive survey on integrating reinforcement learning into the evolutionary algorithm, referred to as reinforcement learning-assisted evolutionary algorithm (RL-EA). We begin with the conceptual outlines of reinforcement learning and the evolutionary algorithm. We then provide a taxonomy of RL-EA. Subsequently, we discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. The RL-assisted procedure is divided according to the implemented functions including solution generation, learnable objective function, algorithm/operator/sub-population selection, parameter adaptation, and other strategies. Finally, we analyze potential directions for future research. This survey serves as a rich resource for researchers interested in RL-EA as it overviews the current state-of-the-art and highlights the associated challenges. By leveraging this survey, readers can swiftly gain insights into RL-EA to develop efficient algorithms, thereby fostering further advancements in this emerging field.Comment: 26 pages, 16 figure

Optimisation problems and resolution methods in satellite scheduling and space-craft operation: a survey

The fast development in the production of small, low-cost satellites is propelling an important increase in satellite mission planning and operations projects. Central to satellite mission planning is the resolution of scheduling problem for an optimised allocation of user requests for efficient communication between operations teams at the ground and spacecraft systems. The aim of this paper is to survey the state of the art in the satellite scheduling problem, analyse its mathematical formulations, examine its multi-objective nature and resolution through meta-heuristics methods. Finally, we consider some optimisation problems arising in spacecraft design, operation and satellite deployment systemsPeer ReviewedPostprint (author's final draft

Evolutionary Computation 2020

Intelligent optimization is based on the mechanism of computational intelligence to refine a suitable feature model, design an effective optimization algorithm, and then to obtain an optimal or satisfactory solution to a complex problem. Intelligent algorithms are key tools to ensure global optimization quality, fast optimization efficiency and robust optimization performance. Intelligent optimization algorithms have been studied by many researchers, leading to improvements in the performance of algorithms such as the evolutionary algorithm, whale optimization algorithm, differential evolution algorithm, and particle swarm optimization. Studies in this arena have also resulted in breakthroughs in solving complex problems including the green shop scheduling problem, the severe nonlinear problem in one-dimensional geodesic electromagnetic inversion, error and bug finding problem in software, the 0-1 backpack problem, traveler problem, and logistics distribution center siting problem. The editors are confident that this book can open a new avenue for further improvement and discoveries in the area of intelligent algorithms. The book is a valuable resource for researchers interested in understanding the principles and design of intelligent algorithms

Development of transportation and supply chain problems with the combination of agent-based simulation and network optimization

Demand drives a different range of supply chain and logistics location decisions, and agent-based modeling (ABM) introduces innovative solutions to address supply chain and logistics problems. This dissertation focuses on an agent-based and network optimization approach to resolve those problems and features three research projects that cover prevalent supply chain management and logistics problems. The first case study evaluates demographic densities in Norway, Finland, and Sweden, and covers how distribution center (DC) locations can be established using a minimizing trip distance approach. Furthermore, traveling time maps are developed for each scenario. In addition, the Nordic area consisting of those three countries is analyzed and five DC location optimization results are presented. The second case study introduces transportation cost modelling in the process of collecting tree logs from several districts and transporting them to the nearest collection point. This research project presents agent-based modelling (ABM) that incorporates comprehensively the key elements of the pick-up and delivery supply chain model and designs the components as autonomous agents communicating with each other. The modelling merges various components such as GIS routing, potential facility locations, random tree log pickup locations, fleet sizing, trip distance, and truck and train transportation. The entire pick-up and delivery operation are modeled by ABM and modeling outcomes are provided by time series charts such as the number of trucks in use, facilities inventory and travel distance. In addition, various scenarios of simulation based on potential facility locations and truck numbers are evaluated and the optimal facility location and fleet size are identified. In the third case study, an agent-based modeling strategy is used to address the problem of vehicle scheduling and fleet optimization. The solution method is employed to data from a real-world organization, and a set of key performance indicators are created to assess the resolution's effectiveness. The ABM method, contrary to other modeling approaches, is a fully customized method that can incorporate extensively various processes and elements. ABM applying the autonomous agent concept can integrate various components that exist in the complex supply chain and create a similar system to assess the supply chain efficiency.Tuotteiden kysyntä ohjaa erilaisia toimitusketju- ja logistiikkasijaintipäätöksiä, ja agenttipohjainen mallinnusmenetelmä (ABM) tuo innovatiivisia ratkaisuja toimitusketjun ja logistiikan ongelmien ratkaisemiseen. Tämä väitöskirja keskittyy agenttipohjaiseen mallinnusmenetelmään ja verkon optimointiin tällaisten ongelmien ratkaisemiseksi, ja sisältää kolme tapaustutkimusta, jotka voidaan luokitella kuuluvan yleisiin toimitusketjun hallinta- ja logistiikkaongelmiin. Ensimmäinen tapaustutkimus esittelee kuinka käyttämällä väestötiheyksiä Norjassa, Suomessa ja Ruotsissa voidaan määrittää strategioita jakelukeskusten (DC) sijaintiin käyttämällä matkan etäisyyden minimoimista. Kullekin skenaariolle kehitetään matka-aikakartat. Lisäksi analysoidaan näistä kolmesta maasta koostuvaa pohjoismaista aluetta ja esitetään viisi mahdollista sijaintia optimointituloksena. Toinen tapaustutkimus esittelee kuljetuskustannusmallintamisen prosessissa, jossa puutavaraa kerätään useilta alueilta ja kuljetetaan lähimpään keräyspisteeseen. Tämä tutkimusprojekti esittelee agenttipohjaista mallinnusta (ABM), joka yhdistää kattavasti noudon ja toimituksen toimitusketjumallin keskeiset elementit ja suunnittelee komponentit keskenään kommunikoiviksi autonomisiksi agenteiksi. Mallinnuksessa yhdistetään erilaisia komponentteja, kuten GIS-reititys, mahdolliset tilojen sijainnit, satunnaiset puunhakupaikat, kaluston mitoitus, matkan pituus sekä monimuotokuljetukset. ABM:n avulla mallinnetaan noutojen ja toimituksien koko ketju ja tuloksena saadaan aikasarjoja kuvaamaan käytössä olevat kuorma-autot, sekä varastomäärät ja ajetut matkat. Lisäksi arvioidaan erilaisia simuloinnin skenaarioita mahdollisten laitosten sijainnista ja kuorma-autojen lukumäärästä sekä tunnistetaan optimaalinen toimipisteen sijainti ja tarvittava autojen määrä. Kolmannessa tapaustutkimuksessa agenttipohjaista mallinnusstrategiaa käytetään ratkaisemaan ajoneuvojen aikataulujen ja kaluston optimoinnin ongelma. Ratkaisumenetelmää käytetään dataan, joka on peräisin todellisesta organisaatiosta, ja ratkaisun tehokkuuden arvioimiseksi luodaan lukuisia keskeisiä suorituskykyindikaattoreita. ABM-menetelmä, toisin kuin monet muut mallintamismenetelmät, on täysin räätälöitävissä oleva menetelmä, joka voi sisältää laajasti erilaisia prosesseja ja elementtejä. Autonomisia agentteja soveltava ABM voi integroida erilaisia komponentteja, jotka ovat olemassa monimutkaisessa toimitusketjussa ja luoda vastaavan järjestelmän toimitusketjun tehokkuuden arvioimiseksi yksityiskohtaisesti.fi=vertaisarvioitu|en=peerReviewed