A combined genetic algorithms-shooting method approach to solving optimal control problems

10.1080/002077200291488International Journal of Systems Science31183-89IJSY

Efficient evolutionary algorithms for optimal control

If optimal control problems are solved by means of gradient based local search methods, convergence to local solutions is likely. Recently, there has been an increasing interest in the use of global optimisation algorithms to solve optimal control problems, which are expected to have local solutions. Evolutionary Algorithms (EAs) are global optimisation algorithms that have mainly been applied to solve static optimisation problems. Only rarely Evolutionary Algorithms have been used to solve optimal control problems. This may be due to the belief that their computational efficiency is insufficient to solve this type of problems. In addition, the application of Evolutionary Algorithms is a relatively young area of research. As demonstrated in this thesis, Evolutionary Algorithms exist which have significant advantages over other global optimisation methods for optimal control, while their efficiency is comparable.The purpose of this study was to investigate and search for efficient evolutionary algorithms to solve optimal control problems that are expected to have local solutions. These optimal control problems are called multi-modal. An important additional requirement for the practical application of these algorithms is that they preferably should not require any algorithm parameter tuning. Therefore algorithms with less algorithm parameters should be preferred. In addition guidelines for the choice of algorithm parameter values, and the possible development of automatic algorithm parameter adjustment strategies, are important issues.This study revealed that Differential Evolution (DE) algorithms are a class of evolutionary algorithms that do not share several theoretical and practical limitations that other Genetic Algorithms have. As a result they are significantly more efficient than other Genetic Algorithms, such as Breeder Genetic Algorithms (BGA), when applied to multi-modal optimal control problems. Their efficiency is comparable to the efficiency of Iterative Dynamic Programming (IDP), a global optimisation approach specifically designed for optimal control. Moreover the DE algorithms turned out to be significantly less sensitive to problems concerning the selection or tuning of algorithm parameters and the initialisation of the algorithm.Although it is not a DE algorithm, the GENOCOP algorithm is considered to be one of the most efficient genetic algorithms with real-valued individuals and specialized evolutionary operators. This algorithm was the starting point of our research. In Chapter 2 it was applied to some optimal control problems from chemical engineering. These problems were high dimensional, non-linear, multivariable, multi-modal and non-differentiable. Basically with GENOCOP the same solutions were obtained as with Iterative Dynamic Programming. Moreover GENOCOP is more successful in locating the global solution in comparison with other local optimisation algorithms. GENOCOP'S efficiency however is rather poor and the algorithm parameter tuning rather complicated. This motivated us to seek for more efficient evolutionary algorithms.Mathematical arguments found in the literature state that DE algorithms outperform other Evolutionary Algorithms in terms of computational efficiency. Therefore in Chapter 3, DE algorithms, generally used to solve continuous parameter optimisation problems, were used to solve two multi-modal (benchmark) optimal control problems. Also some Breeder Genetic Algorithms (BGA) were applied to solve these problems. The results obtained with these algorithms were compared to one another, and to the results obtained with IDP. The comparison confirmed that DE algorithms stand out in terms of efficiency as compared to the Breeder Genetic algorithms. Moreover, in contrast to the majority of Evolutionary Algorithms, which have many algorithm parameters that need to be selected or tuned, DE has only three algorithm parameters that have to be selected or tuned. These are the population size (µ), the crossover constant (CR) and the differential variation amplification (F). The population size plays a crucial role in solving multi-modal optimal control problems. Selecting a smaller population size enhances the computational efficiency but reduces the probability of finding the global solution. During our investigations we tried to find the best trade-off. One of the most efficient DE algorithms is denoted by DE/best/2/bin . All the investigated DE algorithms solved the two benchmark multi-modal optimal control problems properly and efficiently. The computational efficiency achieved by the DE algorithms in solving the first low multi-modal problem, was comparable to that of IDP. When applied to the second highly multi-modal problem, the computational efficiency of DE was slightly inferior to the one of IDP, after tuning of the algorithm parameters. However, the selection or tuning of the algorithm parameters for IDP is more difficult and more involved.From our investigation the following guidelines were obtained for the selection of the DE algorithm parameters. Take the population size less than or equal to two times the number of variables to be optimised that result from the control parameterisation of the original optimal control problem ( µ ≤ 2n u ). Highly multi-modal optimal control problems require a large value of the differential variation amplification ( F ≥0.9) and a very small or zero value for the crossover constant (0≤ CR ≤0.2). Low multi-modal optimal control problems need a medium value for the differential variation amplification (0.4≤ CR ≤0.6) and a large or medium value for the crossover constant (0.2≤ CR ≤0.5). In contrast to IDP, finding near-optimal values for the algorithm parameters is very simple for DE algorithms.Generally, the DE algorithm parameters are kept constant during the optimization process. A more effective and efficient algorithm may be obtained if they are adjusted on-line. In Chapter 4, a strategy that on-line adjusts the differential variation amplification ( F ) and the crossover constant ( CR ) using a measure of the diversity of the individuals in the population, was proposed. Roughly, the proposed strategy takes large values for F and small values for CR at the beginning of the optimization in order to promote a global search. When the population approaches the solution, F is decreased in order to promote a local search, and the crossover parameter CR is enlarged to increase the speed of convergence. When implemented on the DE algorithm DE/rand/1/bin and applied to the two benchmark multi-modal optimal control problems, the computational efficiency significantly improved and also the probability of locating the global solution.To judge the opportunities and advantages of using Evolutionary Algorithms to solve problems related to optimal control, in Chapter 5 several engineering applications concerning optimal greenhouse cultivation control are considered. In Chapter 5.1 genetic algorithms with binary individuals (Simple Genetic Algorithm) and floating-point representation (GENOCOP) for the individuals are used to estimate some of the parameters of a two-state dynamic model of a lettuce crop, the so-called NICOLET model. This model is intended to predict dry weight and nitrate content of lettuce at harvest time. Parameter estimation problems usually suffer from local minima. This study showed that Evolutionary Algorithms are suitable to calibrate the parameters of a dynamic model. However the required computation time is significant. Partly this is due to the high computational load of a single objective function evaluation, which for parameter optimisation problems involves a system simulation. Even though parameter optimisation is very often performed off-line, thus making computation time perhaps less important, more efficient evolutionary algorithms like DE are to be preferred.In Chapter 5.2 an optimal control problem of nitrate concentration in a lettuce crop was solved by means of two different algorithms. The ACW (Adjustable Control-variation Weight) gradient algorithm, which searches for local solutions, and the DE algorithm DE/best/2/bin that searches for a global solution. The dynamic system is a modified two-state dynamic model of a lettuce crop (NICOLET B3) and the control problem has a fixed final time and control and terminal state constraints. The DE algorithm was extended in order to deal with this.The results showed that this problem probably does not have local solutions and that the control parameterisation required by the DE algorithm causes some difficulties in accurately approximating the continuous solution obtained by the ACW algorithm. On the other hand the computational efficiency of the evolutionary algorithm turned out to be impressive. An almost natural conclusion therefore is to combine a DE algorithm with a gradient algorithm.In Chapter 5.3 the combination of a DE algorithm and a first order gradient algorithm is used to solve an optimal control problem. The DE algorithm is used to approximate the global solution sufficiently close after which the gradient algorithm can converge to it efficiently. This approach was successfully tried on the optimal control of nitrate in lettuce, which unfortunately in this case, seems to have no local solutions. Still the feasibility of this approach, which is important for all types of optimal control problems of which it is unknown a-priori whether they have local solutions, was clearly demonstrated.Finally, in Chapter six this thesis ends with an overall discussion, conclusions and suggestions for future research