Automatic synthesis of fuzzy systems: An evolutionary overview with a genetic programming perspective

Studies in Evolutionary Fuzzy Systems (EFSs) began in the 90s and have experienced a fast development since then, with applications to areas such as pattern recognition, curve‐fitting and regression, forecasting and control. An EFS results from the combination of a Fuzzy Inference System (FIS) with an Evolutionary Algorithm (EA). This relationship can be established for multiple purposes: fine‐tuning of FIS's parameters, selection of fuzzy rules, learning a rule base or membership functions from scratch, and so forth. Each facet of this relationship creates a strand in the literature, as membership function fine‐tuning, fuzzy rule‐based learning, and so forth and the purpose here is to outline some of what has been done in each aspect. Special focus is given to Genetic Programming‐based EFSs by providing a taxonomy of the main architectures available, as well as by pointing out the gaps that still prevail in the literature. The concluding remarks address some further topics of current research and trends, such as interpretability analysis, multiobjective optimization, and synthesis of a FIS through Evolving methods

Automatic synthesis of sorting algorithms by gene expression programming + (geometric) semantic gene expression programming + encouraging phenotype variation with a new semantic operator: semantic conditional crossover

Gene Expression Programming (GEP) is an alternative to Genetic Programming (GP). Given its characteristics compared to GP, we question if GEP should be the standard choice for evolutionary program synthesis, both as base for research and practical application. We raise the question if such a shift could increase the rate of investigation, applicability and the quality of results obtained from evolutionary techniques for code optimization. We present three distinct and unprecedented studies using GEP in an attempt to develop understanding, investigate the potential and forward the branch. Each study has an individual contribution on its own involving GEP. As a whole, the three studies try to investigate di erent aspects that might be critical to answer the questions raised in the previous paragraph. In the rst individual contribution, we investigate GEP's applicability to automatically synthesize sorting algorithms. Performance is compared against GP under similar experimental conditions. GEP is shown to be capable of producing sorting algorithms and outperforms GP in doing so. As a second experiment, we enhanced GEP's evolutionary process with semantic awareness of candidate programs, originating Semantic Gene Expression Programming (SGEP), similarly to how Semantic Genetic Programming (SGP) builds over GP. Geometric semantic concepts are then introduced to SGEP, forming Geometric Semantic Gene Expression Programming (GSGEP). A comparative experiment between GP, GEP, SGP and SGEP is performed using di erent problems and setup combinations. Results were mixed when comparing SGEP and SGP, suggesting performance is signi cantly related to the problem addressed. By out-performing the alternatives in many of the benchmarks, SGEP demonstrates practical potential. The results are analyzed in di erent perspectives, also providing insight on the potential of di erent crossover variations when applied along GP/GEP. GEP' compatibility with innovation developed to work with GP is demonstrated possible without extensive adaptation. Considerations for integration of SGEP are discussed. In the last contribution, a new semantic operator is proposed, SCC, which applies crossover conditionally only when elements are semantically di erent enough, performing mutation otherwise. The strategy attempts to encourage semantic diversity and wider the portion of the semantic-solution space searched. A practical experiment was performed alternating the integration of SCC in the evolutionary process. When using the operator, the quality of obtained solutions alternated between slight improvements and declines. The results don't show a relevant indication of possible advantage from its employment and don't con rm what was expected in the theory. We discuss ways in which further work might investigate this concept and assess if it has practical potential under di erent circumstances. On the other hand, in regards to the basilar questions of this investigation, the process of development and testing of SCC is performed completely on a GEP/SGEP base, suggesting how the latest can be used as the base for future research on evolutionary program synthesis.Programa c~ao Gen etica por Express~oes (GEP) e uma alternativa recente a Programa c~ao Gen etica (GP). Neste estudo observamos o GEP e colocamos a quest~ao se este n~ao deveria ser tratado como primeira escolha quando se trata de sintetiza c~ao autom atica de programas atrav es de m etodos evolutivos. Dadas as caracteristicas do GEP perguntamonos se esta mudan ca de perspectiva poderia aumentar a investiga c~ao, aplicabilidade e qualidade dos resultados obtidos para a optimiza c~ao de c odigo por m etodos evolutivos. Neste estudo apresentamos tr^es contribui c~oes in editas e distintas usando o algoritmo GEP. Cada uma das contribui c~oes apresenta um avan co ou investiga c~ao no campo da GEP. Como um todo, estas contribui c~oes tentam obter cohecimento e informa c~oes para se abordar a quest~ao geral apresentada no p aragrafo anterior. Na primeira contribui c~ao, investiga-mos e testamos o GEP no problema da sintese autom atica de algoritmos de ordena c~ao. Para o melhor do nosso conhecimento, esta e a primeira vez que este problema e abordado com o GEP. A performance e comparada a do GP em condi c~oes semelhantes, de modo a isolar as caracteristicas de cada algoritmo como factor de distin c~ao. As a second experiment, we enhanced GEP's evolutionary process with semantic awareness of candidate programs, originating Semantic Gene Expression Programming (SGEP), similarly to how Semantic Genetic Programming (SGP) builds over GP. Geometric semantic concepts are then introduced to SGEP, forming Geometric Semantic Gene Expression Programming (GSGEP). A comparative experiment between GP, GEP, SGP and SGEP is performed using di erent problems and setup combinations. Results were mixed when comparing SGEP and SGP, suggesting performance is signi cantly related to the problem addressed. By out-performing the alternatives in many of the benchmarks, SGEP demonstrates practical potential. The results are analyzed in di erent perspectives, also providing insight on the potential of di erent crossover variations when applied along GP/GEP. GEP's compatibility with innovation developed to work with GP is demonstrated possible without extensive adaptation. Considerations for integration of SGEP are discussed. Na segunda contribui c~ao, adicionamos ao processo evolutivo do GEP a capacidade de medir o valor sem^antico dos programas que constituem a popula c~ao. A esta variante damos o nome de Programa c~ao Gen etica por Express~oes Sem^antica (SGEP). Esta variante tr as para o GEP as mesmas caracteristicas que a Programa c~ao Gen etica Sem^antica(SGP) trouxe para o GP convencional. Conceitos geom etricos s~ao tamb em apresentados para o SGEP, extendendo assim a variante e criando a Programa c~ao Gen etica por Express~oes Geom etrica Sem^antica (GSGEP). De forma a testar estas novas variantes, efectuamos uma experi^encia onde s~ao comparados o GP, GEP, SGP e SGEP entre diferentes problemas e combina c~oes de operadores de cruzamento. Os resultados mostraram que n~ao houve um algoritmo que se destaca-se em todas as experi^encias, sugerindo que a performance est a signi cativamente relacionada com o problema a ser abordado. De qualquer modo, o SGEP obteve vantagem em bastantes dos benchmarks, dando assim ind cios de pot^encial ter utilidade pr atica. De um modo geral, esta contribui c~ao demonstra que e possivel utilizar tecnologia desenvolvida a pensar em GP no GEP sem grande esfor co na adapta c~ao. No m da contribui c~ao, s~ao discutidas algumas considera c~oes sobre o SGEP. Na terceira contribui c~ao propomos um novo operador, o Cruzamento Sem^antico Condicional (SCC). Este operador, baseado na dist^ancia sem^antica entre dois elementos propostos, decide se os elementos s~ao propostos para cruzamento, ou se um deles e mutato e ambos re-introduzidos na popula c~ao. Esta estrat egia tem como objectivo aumentar a diversidade gen etica na popula c~ao em fases cruciais do processo evolutivo e alargar a por c~ao do espa co sem^antico pesquisado. Para avaliar o pot^encial deste operador, realizamos uma experi^encia pr atica e comparamos processos evolutivos semelhantes onde o uso ou n~ao uso do SCC e o factor de distin c~ao. Os resultados obtidos n~ao demonstraram vantagens no uso do SCC e n~ao con rmam o esperado em teoria. No entanto s~ao discutidas maneiras em que o conceito pode ser reaproveitado para novos testes em que possa ter pot^encial para demonstrar resultados possitivos. Em rela c~ao a quest~ao central da tese, visto este estudo ter sido desenvolvido com base em GEP/SGEP e visto a teoria do SCC ser compativel com GP, e demonstrado que um estudo geral a area da sintese de algoritmos por meios evolutivos, pode ser conduzido com base no GEP

AI Feynman: a Physics-Inspired Method for Symbolic Regression

A core challenge for both physics and artificial intellicence (AI) is symbolic regression: finding a symbolic expression that matches data from an unknown function. Although this problem is likely to be NP-hard in principle, functions of practical interest often exhibit symmetries, separability, compositionality and other simplifying properties. In this spirit, we develop a recursive multidimensional symbolic regression algorithm that combines neural network fitting with a suite of physics-inspired techniques. We apply it to 100 equations from the Feynman Lectures on Physics, and it discovers all of them, while previous publicly available software cracks only 71; for a more difficult test set, we improve the state of the art success rate from 15% to 90%.Comment: 15 pages, 2 figs. Our code is available at https://github.com/SJ001/AI-Feynman and our Feynman Symbolic Regression Database for benchmarking can be downloaded at https://space.mit.edu/home/tegmark/aifeynman.htm

A Field Guide to Genetic Programming

xiv, 233 p. : il. ; 23 cm.Libro ElectrónicoA Field Guide to Genetic Programming (ISBN 978-1-4092-0073-4) is an introduction to genetic programming (GP). GP is a systematic, domain-independent method for getting computers to solve problems automatically starting from a high-level statement of what needs to be done. Using ideas from natural evolution, GP starts from an ooze of random computer programs, and progressively refines them through processes of mutation and sexual recombination, until solutions emerge. All this without the user having to know or specify the form or structure of solutions in advance. GP has generated a plethora of human-competitive results and applications, including novel scientific discoveries and patentable inventions. The authorsIntroduction -- Representation, initialisation and operators in Tree-based GP -- Getting ready to run genetic programming -- Example genetic programming run -- Alternative initialisations and operators in Tree-based GP -- Modular, grammatical and developmental Tree-based GP -- Linear and graph genetic programming -- Probalistic genetic programming -- Multi-objective genetic programming -- Fast and distributed genetic programming -- GP theory and its applications -- Applications -- Troubleshooting GP -- Conclusions.Contents xi 1 Introduction 1.1 Genetic Programming in a Nutshell 1.2 Getting Started 1.3 Prerequisites 1.4 Overview of this Field Guide I Basics 2 Representation, Initialisation and GP 2.1 Representation 2.2 Initialising the Population 2.3 Selection 2.4 Recombination and Mutation Operators in Tree-based 3 Getting Ready to Run Genetic Programming 19 3.1 Step 1: Terminal Set 19 3.2 Step 2: Function Set 20 3.2.1 Closure 21 3.2.2 Sufficiency 23 3.2.3 Evolving Structures other than Programs 23 3.3 Step 3: Fitness Function 24 3.4 Step 4: GP Parameters 26 3.5 Step 5: Termination and solution designation 27 4 Example Genetic Programming Run 4.1 Preparatory Steps 29 4.2 Step-by-Step Sample Run 31 4.2.1 Initialisation 31 4.2.2 Fitness Evaluation Selection, Crossover and Mutation Termination and Solution Designation Advanced Genetic Programming 5 Alternative Initialisations and Operators in 5.1 Constructing the Initial Population 5.1.1 Uniform Initialisation 5.1.2 Initialisation may Affect Bloat 5.1.3 Seeding 5.2 GP Mutation 5.2.1 Is Mutation Necessary? 5.2.2 Mutation Cookbook 5.3 GP Crossover 5.4 Other Techniques 32 5.5 Tree-based GP 39 6 Modular, Grammatical and Developmental Tree-based GP 47 6.1 Evolving Modular and Hierarchical Structures 47 6.1.1 Automatically Defined Functions 48 6.1.2 Program Architecture and Architecture-Altering 50 6.2 Constraining Structures 51 6.2.1 Enforcing Particular Structures 52 6.2.2 Strongly Typed GP 52 6.2.3 Grammar-based Constraints 53 6.2.4 Constraints and Bias 55 6.3 Developmental Genetic Programming 57 6.4 Strongly Typed Autoconstructive GP with PushGP 59 7 Linear and Graph Genetic Programming 61 7.1 Linear Genetic Programming 61 7.1.1 Motivations 61 7.1.2 Linear GP Representations 62 7.1.3 Linear GP Operators 64 7.2 Graph-Based Genetic Programming 65 7.2.1 Parallel Distributed GP (PDGP) 65 7.2.2 PADO 67 7.2.3 Cartesian GP 67 7.2.4 Evolving Parallel Programs using Indirect Encodings 68 8 Probabilistic Genetic Programming 8.1 Estimation of Distribution Algorithms 69 8.2 Pure EDA GP 71 8.3 Mixing Grammars and Probabilities 74 9 Multi-objective Genetic Programming 75 9.1 Combining Multiple Objectives into a Scalar Fitness Function 75 9.2 Keeping the Objectives Separate 76 9.2.1 Multi-objective Bloat and Complexity Control 77 9.2.2 Other Objectives 78 9.2.3 Non-Pareto Criteria 80 9.3 Multiple Objectives via Dynamic and Staged Fitness Functions 80 9.4 Multi-objective Optimisation via Operator Bias 81 10 Fast and Distributed Genetic Programming 83 10.1 Reducing Fitness Evaluations/Increasing their Effectiveness 83 10.2 Reducing Cost of Fitness with Caches 86 10.3 Parallel and Distributed GP are Not Equivalent 88 10.4 Running GP on Parallel Hardware 89 10.4.1 Master–slave GP 89 10.4.2 GP Running on GPUs 90 10.4.3 GP on FPGAs 92 10.4.4 Sub-machine-code GP 93 10.5 Geographically Distributed GP 93 11 GP Theory and its Applications 97 11.1 Mathematical Models 98 11.2 Search Spaces 99 11.3 Bloat 101 11.3.1 Bloat in Theory 101 11.3.2 Bloat Control in Practice 104 III Practical Genetic Programming 12 Applications 12.1 Where GP has Done Well 12.2 Curve Fitting, Data Modelling and Symbolic Regression 12.3 Human Competitive Results – the Humies 12.4 Image and Signal Processing 12.5 Financial Trading, Time Series, and Economic Modelling 12.6 Industrial Process Control 12.7 Medicine, Biology and Bioinformatics 12.8 GP to Create Searchers and Solvers – Hyper-heuristics xiii 12.9 Entertainment and Computer Games 127 12.10The Arts 127 12.11Compression 128 13 Troubleshooting GP 13.1 Is there a Bug in the Code? 13.2 Can you Trust your Results? 13.3 There are No Silver Bullets 13.4 Small Changes can have Big Effects 13.5 Big Changes can have No Effect 13.6 Study your Populations 13.7 Encourage Diversity 13.8 Embrace Approximation 13.9 Control Bloat 13.10 Checkpoint Results 13.11 Report Well 13.12 Convince your Customers 14 Conclusions Tricks of the Trade A Resources A.1 Key Books A.2 Key Journals A.3 Key International Meetings A.4 GP Implementations A.5 On-Line Resources 145 B TinyGP 151 B.1 Overview of TinyGP 151 B.2 Input Data Files for TinyGP 153 B.3 Source Code 154 B.4 Compiling and Running TinyGP 162 Bibliography 167 Inde

Optimisation of multiplier-less FIR filter design techniques

This thesis is concerned with the design of multiplier-less (ML) finite impulse response (FIR) digital filters. The use of multiplier-less digital filters results in simplified filtering structures, better throughput rates and higher speed. These characteristics are very desirable in many DSP systems. This thesis concentrates on the design of digital filters with power-of-two coefficients that result in simplified filtering structures. Two distinct classesof ML FIR filter design algorithms are developed and compared with traditional techniques. The first class is based on the sensitivity of filter coefficients to rounding to power-of-two. Novel elements include extending of the algorithm for multiple-bands filters and introducing mean square error as the sensitivity criterion. This improves the performance of the algorithm and reduces the complexity of resulting filtering structures. The second class of filter design algorithms is based on evolutionary techniques, primarily genetic algorithms. Three different algorithms based on genetic algorithm kernel are developed. They include simple genetic algorithm, knowledge-based genetic algorithm and hybrid of genetic algorithm and simulated annealing. Inclusion of the additional knowledge has been found very useful when re-designing filters or refining previous designs. Hybrid techniques are useful when exploring large, N-dimensional searching spaces. Here, the genetic algorithm is used to explore searching space rapidly, followed by fine search using simulated annealing. This approach has been found beneficial for design of high-order filters. Finally, a formula for estimation of the filter length from its specification and complementing both classes of design algorithms, has been evolved using techniques of symbolic regression and genetic programming. Although the evolved formula is very complex and not easily understandable, statistical analysis has shown that it produces more accurate results than traditional Kaiser's formula. In summary, several novel algorithms for the design of multiplier-less digital filters have been developed. They outperform traditional techniques that are used for the design of ML FIR filters and hence contributed to the knowledge in the field of ML FIR filter design