Learning to Control Differential Evolution Operators

Evolutionary algorithms are widely used for optimsation by researchers in academia and industry. These algorithms have parameters, which have proven to highly determine the performance of an algorithm. For many decades, researchers have focused on determining optimal parameter values for an algorithm. Each parameter configuration has a performance value attached to it that is used to determine a good configuration for an algorithm. Parameter values depend on the problem at hand and are known to be set in two ways, by means of offline and online selection. Offline tuning assumes that the performance value of a configuration remains same during all generations in a run whereas online tuning assumes that the performance value varies from one generation to another. This thesis presents various adaptive approaches each learning from a range of feedback received from the evolutionary algorithm. The contributions demonstrate the benefits of utilising online and offline learning together at different levels for a particular task. Offline selection has been utilised to tune the hyper-parameters of proposed adaptive methods that control the parameters of evolutionary algorithm on-the-fly. All the contributions have been presented to control the mutation strategies of the differential evolution. The first contribution demonstrates an adaptive method that is mapped as markov reward process. It aims to maximise the cumulative future reward. Next chapter unifies various adaptive methods from literature that can be utilised to replicate existing methods and test new ones. The hyper-parameters of methods in first two chapters are tuned by an offline configurator, irace. Last chapter proposes four methods utilising deep reinforcement learning model. To test the applicability of the adaptive approaches presented in the thesis, all methods are compared to various adaptive methods from literature, variants of differential evolution and other state-of-the-art algorithms on various single objective noiseless problems from benchmark set, BBOB

Deep Reinforcement Learning for Adaptive Parameter Control in Differential Evolution for Multi-Objective Optimization

Evolutionary algorithms (EA) are efficient population-based stochastic algorithms for solving optimization problems. The performance of EAs largely depends on the configuration of values of parameters that control their search. Previous works studied how to configure EAs, though, there is a lack of a general approach to effectively tune EAs. To fill this gap, this paper presents a consistent, automated approach for tuning and controlling parameterized search of an EA. For this, we propose a deep reinforcement learning (DRL) based approach called ‘DRL-APC-DE’ for online controlling search parameter values for a multi-objective Differential Evolution algorithm. The proposed method is trained and evaluated on widely adopted multi-objective test problems. The experimental results show that the proposed approach performs competitively to a non-adaptive Differential Evolution algorithm, tuned by grid search on the same range of possible parameter values. Subsequently, the trained algorithms have been applied to unseen multi-objective problems for the adaptive control of parameters. Results show the successful ability of DRL-APC-DE to control parameters for solving these problems, which has the potential to significantly reduce the dependency on parameter tuning for the successful application of EAs

A New Hybrid-Adaptive Differential Evolution for a Smart Grid Application Under Uncertainty

Power systems are showing a dynamic evolution in the last few years, caused in part by the adoption of smart grid technologies. The integration of new elements that represent a source of uncertainty, such as renewables generation, electric vehicles, variable loads and electricity markets, poses a higher degree of complexity causing that traditional mathematical formulations struggle in finding efficient solutions to problems in the smart grid context. In some situations, where traditional approaches fail, computational intelligence has demonstrated being a very powerful tool for solving optimization problems. In this paper, we analyze the application of Differential Evolution (DE) to address an energy resource management problem under uncertain environments. We perform a systematic parameter tuning to determine the best set of parameters of four state-of-the-art DE strategies. Having knowledge of the sensitivity of DE to the parameter selection, self-adaptive parameter control DE algorithms are also implemented, showing that competitive results can be achieved without the application of parameter tuning methodologies. Finally, a new hybrid-adaptive DE algorithm, HyDE, which uses a new “DE/target - to - perturbed_best/1” strategy and an adaptive control parameter mechanism, is proposed to solve the problem. Results show that DE strategies with fixed parameters, despite very sensitive to the setting, can find better solutions than some adaptive DE versions. Overall, our HyDE algorithm excelled all the other tested algorithms, proving its effectiveness solving a smart grid application under uncertainty.his work has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 641794 (project DREAM-GO) and from FEDER Funds through COMPETE program and from National Funds through FCT under the project UID/EEA/00760/2013info:eu-repo/semantics/publishedVersio

Passive Target Localization Problem Based on Improved Hybrid Adaptive Differential Evolution and Nelder-Mead Algorithm

This paper considers a passive target localization problem in Wireless Sensor Networks (WSNs) using the noisy time of arrival (TOA) measurements, obtained from multiple receivers and a single transmitter. The objective function is formulated as a maximum likelihood (ML) estimation problem under the Gaussian noise assumption. Consequently, the objective function of the ML estimator is a highly nonlinear and nonconvex function, where conventional optimization methods are not suitable for this type of problem. Hence, an improved algorithm based on the hybridization of an adaptive differential evolution (ADE) and Nelder-Mead (NM) algorithms, named HADENM, is proposed to find the estimated position of a passive target. In this paper, the control parameters of the ADE algorithm are adaptively updated during the evolution process. In addition, an adaptive adjustment parameter is designed to provide a balance between the global exploration and the local exploitation abilities. Furthermore, the exploitation is strengthened using the NM method by improving the accuracy of the best solution obtained from the ADE algorithm. Statistical analysis has been conducted, to evaluate the benefits of the proposed modifications on the optimization performance of the HADENM algorithm. The comparison results between HADENM algorithm and its versions indicate that the modifications proposed in this paper can improve the overall optimization performance. Furthermore, the simulation shows that the proposed HADENM algorithm can attain the Cramer-Rao lower bound (CRLB) and outperforms the constrained weighted least squares (CWLS) and differential evolution (DE) algorithms. The obtained results demonstrate the high accuracy and robustness of the proposed algorithm for solving the passive target localization problem for a wide range of measurement noise levels

A multi-objective differential evolutionary algorithm for optimal sustainable pavement maintenance plan at the network level

Sustainable highway pavement maintenance is important for achieving sustainability in the transportation sector. Because the three aspects included in sustainability metrics (environment, economy, and society) often contradict each other, maximising the sustainability performance of highway pavements is difficult, especially at the network level. This study developed a novel multi-objective heuristic algorithm to formulate sustainable highway pavement network maintenance plans considering carbon emissions (CE), life cycle agency cost (LCAC), and pavement long-term performance (LTP). The proposed algorithm is a new variant of multi-objective differential evolution (MODE) that incorporates self-adaptive parameter control and hybrid mutation strategies embedded in its framework (MOSHDE). Three state-of-the-art multi-objective heuristics, namely, the non-dominated sorting genetic algorithm II(NSGA-II), classic MODE, and multi-objective particle swarm optimisation (MOPSO), as well as the proposed MOSHDE, were applied to an existing highway pavement network in China for performance evaluation. Compared with other heuristic algorithms, the proposed self-adaptive parameter control strategy enables the automatic adjustment of the control parameters, avoiding the time-consuming process of selecting them and enhancing the robustness and applicability of differential evolution. The hybrid mutation strategy uses both exploration and exploitation operators for the mutation operations, thus leveraging both global and local searches. The results of the numerical experiment demonstrate that MOSHDE outperforms the other tested heuristics in terms of efficiency and quality and diversity of the obtained approximate Pareto set. The optimal solutions obtained by the proposed method correspond to a proactive maintenance policy, as opposed to the reactive maintenance policy commonly adopted in current practice. In addition, these solutions are more cost-effective and environmentally friendly and can provide better pavement performance to highway users over the project life cycle. Therefore, the proposed MOSHDE may help practitioners in the transportation sector make their highway infrastructure more sustainable

Advances in global optimization of high brightness beams

High brightness electron beams play an important role in accelerator-based applications such as driving X-ray free electron laser (FEL) radiation. In this paper, we report on advances in global beam dynamics optimization of an accelerator design using start-to-end simulations and a new parallel multi-objective differential evolution optimization method. The global optimization results in significant improvement of the final electron beam brightness

Differential evolution with an evolution path: a DEEP evolutionary algorithm

Utilizing cumulative correlation information already existing in an evolutionary process, this paper proposes a predictive approach to the reproduction mechanism of new individuals for differential evolution (DE) algorithms. DE uses a distributed model (DM) to generate new individuals, which is relatively explorative, whilst evolution strategy (ES) uses a centralized model (CM) to generate offspring, which through adaptation retains a convergence momentum. This paper adopts a key feature in the CM of a covariance matrix adaptation ES, the cumulatively learned evolution path (EP), to formulate a new evolutionary algorithm (EA) framework, termed DEEP, standing for DE with an EP. Without mechanistically combining two CM and DM based algorithms together, the DEEP framework offers advantages of both a DM and a CM and hence substantially enhances performance. Under this architecture, a self-adaptation mechanism can be built inherently in a DEEP algorithm, easing the task of predetermining algorithm control parameters. Two DEEP variants are developed and illustrated in the paper. Experiments on the CEC'13 test suites and two practical problems demonstrate that the DEEP algorithms offer promising results, compared with the original DEs and other relevant state-of-the-art EAs

Differential evolution with two-level parameter adaptation

The performance of differential evolution (DE) largely depends on its mutation strategy and control parameters. In this paper, we propose an adaptive DE (ADE) algorithm with a new mutation strategy DE/lbest/1 and a two-level adaptive parameter control scheme. The DE/lbest/1 strategy is a variant of the greedy DE/best/1 strategy. However, the population is mutated under the guide of multiple locally best individuals in DE/lbest/1 instead of one globally best individual in DE/best/1. This strategy is beneficial to the balance between fast convergence and population diversity. The two-level adaptive parameter control scheme is implemented mainly in two steps. In the first step, the population-level parameters F p and CR p for the whole population are adaptively controlled according to the optimization states, namely, the exploration state and the exploitation state in each generation. These optimization states are estimated by measuring the population distribution. Then, the individual-level parameters F i and CR i for each individual are generated by adjusting the population-level parameters. The adjustment is based on considering the individual's fitness value and its distance from the globally best individual. This way, the parameters can be adapted to not only the overall state of the population but also the characteristics of different individuals. The performance of the proposed ADE is evaluated on a suite of benchmark functions. Experimental results show that ADE generally outperforms four state-of-the-art DE variants on different kinds of optimization problems. The effects of ADE components, parameter properties of ADE, search behavior of ADE, and parameter sensitivity of ADE are also studied. Finally, we investigate the capability of ADE for solving three real-world optimization problems

Freeze-drying modeling and monitoring using a new neuro-evolutive technique

This paper is focused on the design of a black-box model for the process of freeze-drying of pharmaceuticals. A new methodology based on a self-adaptive differential evolution scheme is combined with a back-propagation algorithm, as local search method, for the simultaneous structural and parametric optimization of the model represented by a neural network. Using the model of the freeze-drying process, both the temperature and the residual ice content in the product vs. time can be determine off-line, given the values of the operating conditions (the temperature of the heating shelf and the pressure in the drying chamber). This makes possible to understand if the maximum temperature allowed by the product is trespassed and when the sublimation drying is complete, thus providing a valuable tool for recipe design and optimization. Besides, the black box model can be applied to monitor the freeze-drying process: in this case, the measurement of product temperature is used as input variable of the neural network in order to provide in-line estimation of the state of the product (temperature and residual amount of ice). Various examples are presented and discussed, thus pointing out the strength of the too