Parameter estimation for a mechanistic model of high dose irradiation damage using Nelder-Mead simplex method and genetic algorithm

Radiation therapy is one of the cancer cells treatments that use high-energy radiation to shrink tumors and kill cancer cells. Radiation therapy kills cancer cells by damaging their DNA directly or creates charged particles within the cells that can in turn damage the DNA. As a side effect of the treatment, the radiation therapy can also damage the normal cell that located at parts of our body. The main goals of radiation therapy are to maximize the damaging of tumors cell and minimize the damage of normal tissue cell. Hence, in this study, we adopt an existing model of high dose irradiation damage. The purpose of this study is to estimate the six parameters of the model which are involved. Two optimization algorithms are used in order to estimate the parameters: Nelder-Mead (NM) simplex method and Genetic Algorithm (GA). Both methods have to achieve the objective function which is to minimize the sum of square error (SSE) between the experimental data and the simulation data. The performances of both algorithms are compared based on the computational time, number of iteration and value of sum of square error. The optimization process is carried out using MATLAB programming built-in functions. The parameters estimation results shown that Nelder-Mead simplex method is more superior compare to Genetic Algorithm for this problem

A new evolutionary search strategy for global optimization of high-dimensional problems

Global optimization of high-dimensional problems in practical applications remains a major challenge to the research community of evolutionary computation. The weakness of randomization-based evolutionary algorithms in searching high-dimensional spaces is demonstrated in this paper. A new strategy, SP-UCI is developed to treat complexity caused by high dimensionalities. This strategy features a slope-based searching kernel and a scheme of maintaining the particle population's capability of searching over the full search space. Examinations of this strategy on a suite of sophisticated composition benchmark functions demonstrate that SP-UCI surpasses two popular algorithms, particle swarm optimizer (PSO) and differential evolution (DE), on high-dimensional problems. Experimental results also corroborate the argument that, in high-dimensional optimization, only problems with well-formative fitness landscapes are solvable, and slope-based schemes are preferable to randomization-based ones. © 2011 Elsevier Inc. All rights reserved

Shuffled Complex-Self Adaptive Hybrid EvoLution (SC-SAHEL) optimization framework

Simplicity and flexibility of meta-heuristic optimization algorithms have attracted lots of attention in the field of optimization. Different optimization methods, however, hold algorithm-specific strengths and limitations, and selecting the best-performing algorithm for a specific problem is a tedious task. We introduce a new hybrid optimization framework, entitled Shuffled Complex-Self Adaptive Hybrid EvoLution (SC-SAHEL), which combines the strengths of different evolutionary algorithms (EAs) in a parallel computing scheme. SC-SAHEL explores performance of different EAs, such as the capability to escape local attractions, speed, convergence, etc., during population evolution as each individual EA suits differently to various response surfaces. The SC-SAHEL algorithm is benchmarked over 29 conceptual test functions, and a real-world hydropower reservoir model case study. Results show that the hybrid SC-SAHEL algorithm is rigorous and effective in finding global optimum for a majority of test cases, and that it is computationally efficient in comparison to algorithms with individual EA

Training Single Walled Carbon Nanotube based Materials to perform computation

This thesis illustrates the use of Single Walled Carbon Nanotube based materials for the solution of various computational problems by using the process of computer controlled evolution. The study aims to explore and identify three dimensions of a form of unconventional computing called, `Evolution-in-materio'. First, it focuses on identifying suitable materials for computation. Second, it explores suitable methods, i.e. optimisation and evolutionary algorithms to train these materials to perform computation. And third, it aims to identify suitable computational problems to test with these materials. Different carbon based materials, mainly single walled carbon nano-tubes with their varying concentrations in polymers have been studied to be trained for different computational problems using the principal of `evolution-in-materio'. The conductive property of the materials is used to train these materials to perform some meaningful computation. The training process is formulated as an optimisation problem with hardware in loop. It involves the application of an external stimuli (voltages) on the material which brings changes in its electrical properties. In order to train the material for a specific computational problem, a large number of configuration signals need to be tested to find the one that transforms the incident signal in such a way that a meaningful computation can be extracted from the material. An evolutionary algorithm is used to identify this configuration data and using a hardware platform, this data is transformed into incident signals. Depending on the computational problem, the specific voltages signals when applied at specific points on to the material, as identified by an evolutionary algorithm, can make the material behave as a Logic gate, a tone discriminator or a data classifier. The problem is implemented on two types of hardware platforms, one a more simple implementation using mbed ( a micro- controller) and other is a purpose-built platform for `Evolution-in-materio" called Mecobo. The results of this study showed that the single walled carbon nanotube composites can be trained to perform simple computational tasks (such as tone discriminator, AND, OR logic gates and a Half adder circuit), as well as complex computational problems such as Full Adder circuit and various binary and multiple class machine learning problems. The study has also identified the suitability of using evolutionary algorithms such as Particle Swarm Optimisation algorithm (PSO) and Differential evolution for finding solutions of complex computational problems such as complex logic gates and various machine learning classification problems. The implementation of classification problem with the carbon nanotube based materials also identified the role of a classifier. It has been found that K-nearest neighbour method and its variant kNN ball tree algorithm are more suitable to train carbon nanotube based materials for different classification problems. The study of varying concentrations of single walled carbon nanotubes in fixed polymer ratio for the solution of different computational problems provided an indication of the link between single walled carbon nanotubes concentration and ability to solve computational problem. The materials used in this study showed stability in the results for all the considered computational problems. These material systems can compliment the current electronic technology and can be used to create a new type of low energy and low cost electronic devices. This offers a promising new direction for evolutionary computation

A comparison of action selection methods for implicit policy method reinforcement learning in continuous action-space

In this paper I investigate methods of applying reinforcement learning to continuous state- and action-space problems without a policy function. I compare the performance of four methods, one of which is the discretisation of the action-space, and the other three are optimisation techniques applied to finding the greedy action without discretisation. The optimisation methods I apply are gradient descent, Nelder-Mead and Newton's Method. The action selection methods are applied in conjunction with the SARSA algorithm, with a multilayer perceptron utilized for the approximation of the value function. The approaches are applied to two simulated continuous state- and action-space control problems: Cart-Pole and double Cart-Pole. The results are compared both in terms of action selection time and the number of trials required to train on the benchmark problems

A comparison of action selection methods for implicit policy method reinforcement learning in continuous action-space

In this paper I investigate methods of applying reinforcement learning to continuous state- and action-space problems without a policy function. I compare the performance of four methods, one of which is the discretisation of the action-space, and the other three are optimisation techniques applied to finding the greedy action without discretisation. The optimisation methods I apply are gradient descent, Nelder-Mead and Newton's Method. The action selection methods are applied in conjunction with the SARSA algorithm, with a multilayer perceptron utilized for the approximation of the value function. The approaches are applied to two simulated continuous state- and action-space control problems: Cart-Pole and double Cart-Pole. The results are compared both in terms of action selection time and the number of trials required to train on the benchmark problems

Reinforcement learning in continuous state- and action-space

Reinforcement learning in the continuous state-space poses the problem of the inability to store the values of all state-action pairs in a lookup table, due to both storage limitations and the inability to visit all states sufficiently often to learn the correct values. This can be overcome with the use of function approximation techniques with generalisation capability, such as artificial neural networks, to store the value function. When this is applied we can select the optimal action by comparing the values of each possible action; however, when the action-space is continuous this is not possible. In this thesis we investigate methods to select the optimal action when artificial neural networks are used to approximate the value function, through the application of numerical optimization techniques. Although it has been stated in the literature that gradient-ascent methods can be applied to the action selection [47], it is also stated that solving this problem would be infeasible, and therefore, is claimed that it is necessary to utilise a second artificial neural network to approximate the policy function [21, 55]. The major contributions of this thesis include the investigation of the applicability of action selection by numerical optimization methods, including gradient-ascent along with other derivative-based and derivative-free numerical optimization methods,and the proposal of two novel algorithms which are based on the application of two alternative action selection methods: NM-SARSA [40] and NelderMead-SARSA. We empirically compare the proposed methods to state-of-the-art methods from the literature on three continuous state- and action-space control benchmark problems from the literature: minimum-time full swing-up of the Acrobot; Cart-Pole balancing problem; and a double pole variant. We also present novel results from the application of the existing direct policy search method genetic programming to the Acrobot benchmark problem [12, 14]

Encog: Library of Interchangeable Machine Learning Models for Java and C#

This paper introduces the Encog library for Java and C#, a scalable, adaptable, multiplatform machine learning framework that was 1st released in 2008. Encog allows a variety of machine learning models to be applied to datasets using regression, classification, and clustering. Various supported machine learning models can be used interchangeably with minimal recoding. Encog uses efficient multithreaded code to reduce training time by exploiting modern multicore processors. The current version of Encog can be downloaded from http://www.encog.org