The influence of population size in geometric semantic GP

In this work, we study the influence of the population size on the learning ability of Geometric Semantic Genetic Programming for the task of symbolic regression. A large set of experiments, considering different population size values on different regression problems, has been performed. Results show that, on real-life problems, having small populations results in a better training fitness with respect to the use of large populations after the same number of fitness evaluations. However, performance on the test instances varies among the different problems: in datasets with a high number of features, models obtained with large populations present a better performance on unseen data, while in datasets characterized by a relative small number of variables a better generalization ability is achieved by using small population size values. When synthetic problems are taken into account, large population size values represent the best option for achieving good quality solutions on both training and test instances

Neural Network Guided Transfer Learning for Genetic Programming

Programming-by-Example, and code synthesis in general, is a field with many different sub-fields, involving many forms of machine learning and computational logic. With advantages and disadvantages to each, attempts to build effective hybrid solutions would seem to be a promising direction. Transfer Learning (TL) provides a good framework for this, as it allows one of the classic code synthesis techniques, Genetic Programming, to be augmented by past success, to target a particular code synthesis system to the problem domain it is facing. TL allows one type of machine learning algorithm, in this thesis a neural network, to support the core GP process, and combine the strengths of both. This thesis explores the concept of hybrid code synthesis approaches, and then brings the identified strongest elements of each approach together into a single neural network driven Transfer Learning system for Genetic Programming. The TL system operates autonomously, without any human intervention required after the problem set (in example only format) is presented to the system. The thesis first studies how to structure a training corpus for a neural network, across two different experiments, exploring how the constraints placed on a corpus can result in superior training. After this, it studies how GP processes can be guided, to ensure that a hypothetical NN guide would be useful if it could be created and how it can best assist the GP. Finally, it combines the previous studies together into the full end-to-end TL system and tests its performance across two separate problem domain

An Investigation of Supervised Learning in Genetic Programming

Centre for Intelligent Systems and their Applicationsstudentship 9314680This thesis is an investigation into Supervised Learning (SL) in Genetic Programming (GP). With its flexible tree-structured representation, GP is a type of Genetic Algorithm, using the Darwinian idea of natural selection and genetic recombination, evolving populations of solutions over many generations to solve problems. SL is a common approach in Machine Learning where the problem is presented as a set of examples. A good or fit solution is one which can successfully deal with all of the examples.In common with most Machine Learning approaches, GP has been used to solve many trivial problems. When applied to larger and more complex problems, however, several difficulties become apparent. When focusing on the basic features of GP, this thesis highlights the immense size of the GP search space, and describes an approach to measure this space. A stupendously flexible but frustratingly useless representation, Anarchically Automatically Defined Functions, is described. Some difficulties associated with the normal use of the GP operator Crossover (perhaps the most common method of combining GP trees to produce new trees) are demonstrated in the simple MAX problem. Crossover can lead to irreversible sub-optimal GP performance when used in combination with a restriction on tree size. There is a brief study of tournament selection which is a common method of selecting fit individuals from a GP population to act as parents in the construction of the next generation.The main contributions of this thesis however are two approaches for avoiding the fitness evaluation bottleneck resulting from the use of SL in GP. to establish the capability of a GP individual using SL, it must be tested or evaluated against each example in the set of training examples

Applications and enhancements of aircraft design optimization techniques

The aircraft industry has been at the forefront in developing design optimization strategies ever since the advent of high performance computing. Thanks to the large computational resources now available, many new as well as more mature optimization methods have become well established. However, the same cannot be said for other stages along the optimization process - chiefly, and this is where the present thesis seeks to make its first main contribution, at the geometry parameterization stage.The first major part of the thesis is dedicated to the goal of reducing the size of the search space by reducing the dimensionality of existing parameterization schemes, thus improving the effectiveness of search strategies based upon them. Specifically, a refinement to the Kulfan parameterization method is presented, based on using Genetic Programming and a local search within a Baldwinian learning strategy to evolve a set of analytical expressions to replace the standard 'class function' at the basis of the Kulfan method. The method is shown to significantly reduce the number of parameters and improves optimization performance - this is demonstrated using a simple aerodynamic design case study.The second part describes an industrial level case study, combining sophisticated, high fidelity, as well as fast, low fidelity numerical analysis with a complex physical experiment. The objective is the analysis of a topical design question relating to reducing the environmental impact of aviation: what is the optimum layout of an over-the-wing turbofan engine installation designed to enable the airframe to shield near-airport communities on the ground from fan noise. An experiment in an anechoic chamber reveals that a simple half-barrier noise model can be used as a first order approximation to the change of inlet broadband noise shielding by the airframe with engine position, which can be used within design activities. Moreover, the experimental results are condensed into an acoustic shielding performance metric to be used in a Multidisciplinary Design Optimization study, together with drag and engine performance values acquired through CFD. By using surrogate models of these three performance metrics we are able to find a set of non-dominated engine positions comprising a Pareto Front of these objectives. This may give designers of future aircraft an insight into an appropriate engine position above a wing, as well as a template for blending multiple levels of computational analysis with physical experiments into a multidisciplinary design optimization framework

Metamodelling for auxetic materials

The use of Finite Element (FE) based homogenisation has improved the study of composite material properties. A homogenisation is a method of averaging a heterogeneous domain by using a replacement unit cell according to the proportions of constituents in the domain. However, the homogenisation method involves enormous computational effort when implemented in engineering design problems, such as optimisation of a sandwich panel. The large number of computations involved can rule out many approaches due to the expense of carrying out many runs. One way of circumnavigating this problem is to replace the true system by an approximate surrogate model, which is fast-running compared to the original. In traditional approaches using response surfaces, a simple least-squares multinomial model is often adopted. In this thesis, a Genetic Programming model was developed to extend the class of possible models by carrying out a general symbolic regression. The approach is demonstrated on both univariate and multivariate problems with both computational and experimental data. Its performances were compared with Neural Networks - Multi-Layer Perceptrons (MLP) and polynomials. The material system studied here was the auxetic materials. The auxetic behaviour means that the structure exhibits a negative Poisson's ratio during extension. A novel auxetic structure, chiral honeycomb, is introduced in this work, with its experiments, analytical and simulations. The implementations of the auxetic material surrogate models were demonstrated using optimisation problems. One of the optimisation problems was the shape optimisation of the auxetic sandwich using Differential Evolution. The shape optimisation gives the optimal geometry of honeycomb based on the desired mechanical properties specified by the user. The thesis has shown a good performance of numerical homogenisation technique and the robustness of the GP models. A detailed study of the chiral honeycomb has also given insight to the potential application of the auxetic materials

Automatic control program creation using concurrent Evolutionary Computing

Over the past decade, Genetic Programming (GP) has been the subject of a significant amount of research, but this has resulted in the solution of few complex real -world problems. In this work, I propose that, for some relatively simple, non safety -critical embedded control applications, GP can be used as a practical alternative to software developed by humans. Embedded control software has become a branch of software engineering with distinct temporal, interface and resource constraints and requirements. This results in a characteristic software structure, and by examining this, the effective decomposition of an overall problem into a number of smaller, simpler problems is performed. It is this type of problem amelioration that is suggested as a method whereby certain real -world problems may be rendered into a soluble form suitable for GP. In the course of this research, the body of published GP literature was examined and the most important changes to the original GP technique of Koza are noted; particular focus is made upon GP techniques involving an element of concurrency -which is central to this work. This search highlighted few applications of GP for the creation of software for complex, real -world problems -this was especially true in the case of multi thread, multi output solutions. To demonstrate this Idea, a concurrent Linear GP (LGP) system was built that creates a multiple input -multiple output solution using a custom low -level evolutionary language set, combining both continuous and Boolean data types. The system uses a multi -tasking model to evolve and execute the required LGP code for each system output using separate populations: Two example problems -a simple fridge controller and a more complex washing machine controller are described, and the problems encountered and overcome during the successful solution of these problems, are detailed. The operation of the complete, evolved washing machine controller is simulated using a graphical LabVIEWapplication. The aim of this research is to propose a general purpose system for the automatic creation of control software for use in a range of problems from the target problem class -without requiring any system tuning: In order to assess the system search performance sensitivity, experiments were performed using various population and LGP string sizes; the experimental data collected was also used to examine the utility of abandoning stalled searches and restarting. This work is significant because it identifies a realistic application of GP that can ease the burden of finite human software design resources, whilst capitalising on accelerating computing potential

Occupational therapy to improve outdoor mobility after stroke

People who have suffered a stroke can become housebound and miserable because they cannot access suitable transport. They can have difficulty getting to the shops, doctors and hospital and this can have an effect on their quality of life. Occupational therapists routinely aim to help these people overcome their outdoor mobility problems by providing information and verbal instructions but these interventions do not appear to be effective. The aim of this research was to design and evaluate a new occupational therapy outdoor mobility intervention. The intervention was modeled on travel training that is provided for other conditions and the outdoor mobility experiences and needs of people with stroke. Qualitative semi structured interviews were used to investigate 24 peoples experiences of both using transport and their outdoor mobility after they had suffered a stroke. It was found that people wanted to travel for a variety of reasons; shopping, work, getting to the doctors, social reasons, meeting friends, visiting family and just for the sake of traveling. People were prevented from traveling because of physical difficulties such as stepping onto the bus, psychological problems such as confidence and environmental barriers such as the weather or lack of information. The results were used to define the main components of an Occupational Therapy Outdoor Mobility Intervention. A randomised controlled trial was used to evaluate the effects of this Occupational Therapy Outdoor Mobility Intervention (OTOMI) by comparing it to the routine occupational therapy intervention. Participants with stroke in the last 36 months were recruited from primary care services and randomly allocated to receive either the OTOMI or the routine occupational therapy. Participants in the OTOMI received up to seven individualised occupational therapy sessions. The sessions aimed to increase confidence, encourage use of different types of transport and provided tailor-made information. Outcomes were measured by postal assessment 4 and 10 months after recruitment. The primary outcome measure was a yes/ no question, Do you get out of the house as much as you would like? Secondary outcomes included the number of journeys, mood, performance of activities of daily living and leisure. 168 participants who had had a stroke in the last 36 months were recruited into the study over eighteen months, 82 in the control group and 86 to the OTOMI group. 10 people were unable to provide follow-up information at the four month assessment and 21 people at the ten month assessment. Intention-to-treat analyses were undertaken. For the principal outcome measure, participants who were dead at the point of assessment were allocated the worst outcome, and for others lost to follow up their baseline or last recorded responses were used. For the other analyses all missing values were imputed using baseline values. Participants in the treatment group were more likely to get out of their house as often as they wanted at 4 months (RR 1.72,95% CI 1.25 to 2.37) and at 10 months (RR 1.74,95 Cl 1.24 to 2.44). The treatment group recorded more journeys outdoors in the month prior to assessment at 4 months (intervention group median 37, control group median 14, Mann-Whitney p<0.01) and at 10 months (intervention group median 42, control group median 14, Mann-Whitney: p<0.01). At 4 months the NEADL mobility scores were significantly higher in the intervention group, but there were no significant differences in the other secondary outcomes. There were no significant differences in these measures at 10 months. The interview study demonstrated that participating in outdoor mobility is a major problem for people who have had a stroke. The randomised controlled trial demonstrated that a relatively simple and feasible, individualized, properly organised, focused and adequately resourced occupational therapy outdoor mobility intervention can increase participation in outdoor mobility activities, allowing people to get out of the house as much as they wish

Automatically evolving rule induction algorithms with grammar-based genetic programming

In the last 30 years, research in the field of rule induction algorithms produced a large number of algorithms. However, these algorithms are usually obtained from the combination of a basic rule induction algorithm (typically following the sequential covering approach) with new evaluation functions, pruning methods and stopping criteria for refining or producing rules, generating many "new" and more sophisticated sequential covering algorithms. We cannot deny that these attempts to improve the basic sequential covering approach have succeeded. Hence, if manually changing these major components of rule induction algorithms can result in new, significantly better ones, why not to automate this process to make it more cost-effective? This is the core idea of this work: to automate the process of designing rule induction algorithms by means of grammar-based genetic programming. Grammar-based Genetic Programming (GGP) is a special type of evolutionary algorithm used to automatically evolve computer programs. The most interesting feature of this type of algorithm is that it incorporates a grammar into its search mechanism, which expresses prior knowledge about the problem being solved. Since we have a lot of previous knowledge about how humans design rule induction algorithms, this type of algorithm is intuitively a suitable tool to automatically evolve rule induction algorithms. The grammar given to the proposed GGP system includes knowledge about how humans- design rule induction algorithms, and also presents some new elements which could work in rule induction algorithms, but to the best of our knowledge were not previously tested. The GG P system aims to evolve rule induction algorithms under two different frameworks, as follows. In the first framework, the GGP is used to evolve robust rule induction algorithms, i.e., algorithms which were designed to be applied to virtually any classification data set, like a manually-designed rule induction algorithm. In the second framework, the GGP is applied to evolve rule induction algorithms tailored to a specific application XVI domain, i.e., rule induction algorithms tailored to a single data set. Note that the latter framework is hardly feasible on a hard scale in the case of conventional, manually-designed algorithms, since the number of classification data sets greatly outnumbers the number of rule induction algorithms designers. However, it is clearly feasible on a large scale when using the proposed system, which automates the process of rule induction algorithm design and implementation. Overall, extensive computational experiments with 20 VCI data sets and 5 bioinformatics data sets showed that effective rule induction algorithms can be automatically generated using the GGP in both frameworks. Moreover, the automatically evolved rule induction algorithms were shown to be competitive with (and overall slightly better than) four well-known manually designed rule induction algorithms when comparing their predictive accuracies. The proposed GGP system was also compared to a grammar-based hillclimbing system, and experimental results showed that the GGP system is a more effective method to evolve rule induction algorithms than the grammar-based hillclimbing method. At last, a multi-objective version of the GGP (based on the concept of Pareto dominance) was also proposed, and experiments were performed to evolve robust rule induction algorithms which generate both accurate and simple models. The results showed that in most of the cases the GGP system can produce rule induction algorithms which are competitive in predictive accuracy to wellknown human-designed rule induction algorithms, but generate simpler classification modes - i.e., smaller rule sets, intuitively easier to be interpreted by the user