A Field Guide to Genetic Programming

xiv, 233 p. : il. ; 23 cm.Libro ElectrónicoA Field Guide to Genetic Programming (ISBN 978-1-4092-0073-4) is an introduction to genetic programming (GP). GP is a systematic, domain-independent method for getting computers to solve problems automatically starting from a high-level statement of what needs to be done. Using ideas from natural evolution, GP starts from an ooze of random computer programs, and progressively refines them through processes of mutation and sexual recombination, until solutions emerge. All this without the user having to know or specify the form or structure of solutions in advance. GP has generated a plethora of human-competitive results and applications, including novel scientific discoveries and patentable inventions. The authorsIntroduction -- Representation, initialisation and operators in Tree-based GP -- Getting ready to run genetic programming -- Example genetic programming run -- Alternative initialisations and operators in Tree-based GP -- Modular, grammatical and developmental Tree-based GP -- Linear and graph genetic programming -- Probalistic genetic programming -- Multi-objective genetic programming -- Fast and distributed genetic programming -- GP theory and its applications -- Applications -- Troubleshooting GP -- Conclusions.Contents xi 1 Introduction 1.1 Genetic Programming in a Nutshell 1.2 Getting Started 1.3 Prerequisites 1.4 Overview of this Field Guide I Basics 2 Representation, Initialisation and GP 2.1 Representation 2.2 Initialising the Population 2.3 Selection 2.4 Recombination and Mutation Operators in Tree-based 3 Getting Ready to Run Genetic Programming 19 3.1 Step 1: Terminal Set 19 3.2 Step 2: Function Set 20 3.2.1 Closure 21 3.2.2 Sufficiency 23 3.2.3 Evolving Structures other than Programs 23 3.3 Step 3: Fitness Function 24 3.4 Step 4: GP Parameters 26 3.5 Step 5: Termination and solution designation 27 4 Example Genetic Programming Run 4.1 Preparatory Steps 29 4.2 Step-by-Step Sample Run 31 4.2.1 Initialisation 31 4.2.2 Fitness Evaluation Selection, Crossover and Mutation Termination and Solution Designation Advanced Genetic Programming 5 Alternative Initialisations and Operators in 5.1 Constructing the Initial Population 5.1.1 Uniform Initialisation 5.1.2 Initialisation may Affect Bloat 5.1.3 Seeding 5.2 GP Mutation 5.2.1 Is Mutation Necessary? 5.2.2 Mutation Cookbook 5.3 GP Crossover 5.4 Other Techniques 32 5.5 Tree-based GP 39 6 Modular, Grammatical and Developmental Tree-based GP 47 6.1 Evolving Modular and Hierarchical Structures 47 6.1.1 Automatically Defined Functions 48 6.1.2 Program Architecture and Architecture-Altering 50 6.2 Constraining Structures 51 6.2.1 Enforcing Particular Structures 52 6.2.2 Strongly Typed GP 52 6.2.3 Grammar-based Constraints 53 6.2.4 Constraints and Bias 55 6.3 Developmental Genetic Programming 57 6.4 Strongly Typed Autoconstructive GP with PushGP 59 7 Linear and Graph Genetic Programming 61 7.1 Linear Genetic Programming 61 7.1.1 Motivations 61 7.1.2 Linear GP Representations 62 7.1.3 Linear GP Operators 64 7.2 Graph-Based Genetic Programming 65 7.2.1 Parallel Distributed GP (PDGP) 65 7.2.2 PADO 67 7.2.3 Cartesian GP 67 7.2.4 Evolving Parallel Programs using Indirect Encodings 68 8 Probabilistic Genetic Programming 8.1 Estimation of Distribution Algorithms 69 8.2 Pure EDA GP 71 8.3 Mixing Grammars and Probabilities 74 9 Multi-objective Genetic Programming 75 9.1 Combining Multiple Objectives into a Scalar Fitness Function 75 9.2 Keeping the Objectives Separate 76 9.2.1 Multi-objective Bloat and Complexity Control 77 9.2.2 Other Objectives 78 9.2.3 Non-Pareto Criteria 80 9.3 Multiple Objectives via Dynamic and Staged Fitness Functions 80 9.4 Multi-objective Optimisation via Operator Bias 81 10 Fast and Distributed Genetic Programming 83 10.1 Reducing Fitness Evaluations/Increasing their Effectiveness 83 10.2 Reducing Cost of Fitness with Caches 86 10.3 Parallel and Distributed GP are Not Equivalent 88 10.4 Running GP on Parallel Hardware 89 10.4.1 Master–slave GP 89 10.4.2 GP Running on GPUs 90 10.4.3 GP on FPGAs 92 10.4.4 Sub-machine-code GP 93 10.5 Geographically Distributed GP 93 11 GP Theory and its Applications 97 11.1 Mathematical Models 98 11.2 Search Spaces 99 11.3 Bloat 101 11.3.1 Bloat in Theory 101 11.3.2 Bloat Control in Practice 104 III Practical Genetic Programming 12 Applications 12.1 Where GP has Done Well 12.2 Curve Fitting, Data Modelling and Symbolic Regression 12.3 Human Competitive Results – the Humies 12.4 Image and Signal Processing 12.5 Financial Trading, Time Series, and Economic Modelling 12.6 Industrial Process Control 12.7 Medicine, Biology and Bioinformatics 12.8 GP to Create Searchers and Solvers – Hyper-heuristics xiii 12.9 Entertainment and Computer Games 127 12.10The Arts 127 12.11Compression 128 13 Troubleshooting GP 13.1 Is there a Bug in the Code? 13.2 Can you Trust your Results? 13.3 There are No Silver Bullets 13.4 Small Changes can have Big Effects 13.5 Big Changes can have No Effect 13.6 Study your Populations 13.7 Encourage Diversity 13.8 Embrace Approximation 13.9 Control Bloat 13.10 Checkpoint Results 13.11 Report Well 13.12 Convince your Customers 14 Conclusions Tricks of the Trade A Resources A.1 Key Books A.2 Key Journals A.3 Key International Meetings A.4 GP Implementations A.5 On-Line Resources 145 B TinyGP 151 B.1 Overview of TinyGP 151 B.2 Input Data Files for TinyGP 153 B.3 Source Code 154 B.4 Compiling and Running TinyGP 162 Bibliography 167 Inde

Genetic Algorithms and Their Application to the Protein Folding Problem

The protein folding problem involves the prediction of the secondary and tertiary structure of a molecule given the primary structure. The primary structure defines sequence of amino-acid residues, while the secondary structure describes the local 3-dimensional arrangement of amino-acid residues within the molecule. The relative orientation of the secondary structural motifs, namely the tertiary structure, defines the shape of the entire biomolecule. The exact, mechanism by which a sequence of amino acids protein folds into its 3- dimensional conformation is unknown Current approaches to the protein folding problem include calculus-based methods, systematic search, model building and symbolic methods, random methods such as Monte Carlo simulation and simulated annealing, distance geometry, and molecular dynamics. Many of these current approaches search for conformations which minimize the internal energy of the molecule. A genetic algorithm GA, a stochastic search technique modeled after natural adaptive systems, potentially offers significant speedup over other search algorithms because of its inherent parallelizability. The results of applying a parallel GA to the protein folding problem show significant improvement in execution time when compared to serial implementations of the GA. In addition, the parallel GA demonstrates good scalability characteristics since the communications strategy used to manage the population can be tailored to the parallel architecture

Genome-wide definition of promoter and enhancer usage during neural induction of human embryonic stem cells

Genome-wide mapping of transcriptional regulatory elements is an essential tool for understanding the molecular events orchestrating self-renewal, commitment and differentiation of stem cells. We combined high-throughput identification of transcription start sites with genome-wide profiling of histones modifications to map active promoters and enhancers in embryonic stem cells (ESCs) induced to neuroepithelial-like stem cells (NESCs). Our analysis showed that most promoters are active in both cell types while approximately half of the enhancers are cell-specific and account for most of the epigenetic changes occurring during neural induction, and most likely for the modulation of the promoters to generate cell-specific gene expression programs. Interestingly, the majority of the promoters activated or up-regulated during neural induction have a "bivalent" histone modification signature in ESCs, suggesting that developmentally-regulated promoters are already poised for transcription in ESCs, which are apparently pre-committed to neuroectodermal differentiation. Overall, our study provides a collection of differentially used enhancers, promoters, transcription starts sites, protein-coding and non-coding RNAs in human ESCs and ESC-derived NESCs, and a broad, genome-wide description of promoter and enhancer usage and of gene expression programs characterizing the transition from a pluripotent to a neural-restricted cell fate

Of One Mind: Proposal for a Non-Cartesian Cognitive Architecture

Intellectually, we may reject Cartesian Dualism, but dualism often dominates our everyday thinking: we talk of “mental” illness as though it were non-physical; we tend to blame people for the symptoms of brain malfunctions in a way that differs from how we treat other illnesses. An examination of current theories of mind will reveal that some form of dualism is not always limited to the non-scientific realm. While very few, if any, cognitive scientists support mind-body dualism, those who support the view of the mind as a symbol-manipulator are often constrained to postulate more than one cognitive system in response to the failure of the symbol-system model to account for all aspects of human cognition. In this dissertation, I argue for an empiricist, rather than a realist, theory of perception, for an internalist semantics, and for a model of cognitive architecture which combines a connectionist approach with highly-specialized, symbolic, computational component which includes functions that provide input to a a causally-inert conscious mind. I reject the symbol-system hypothesis and propose a cognitive architecture which, I contend, is biologically-plausible and more consistent with the results of recent neuroscientific studies. This hybrid model can accommodate the processes commonly discussed by dual-process theorists and can also accommodate the processes which have proved to be so problematic for models based on the symbol-system hypothesis

Vulnerability and robustness in the essential gene complement of two bacterial species, profiled with CRISPRi

Bacterial essential genes contribute to the most fundamental processes of cellular life. The study of their functions in vivo has long been intractable to systematic genetic approaches, which are fundamental to understanding pathway level connections that govern cellular life and are a requirement for dissecting the complex cellular processes to which essential genes contribute. In Chapter 1 of this work I review recent advances in mapping gene-phenotype relationships in bacteria using the CRISPR-based technology, CRISPR interference (CRISPRi) for titratable gene knockdowns, focusing on their applications to the studies of essential genes, the exploration of chemical-genetic interactions, and the prospects for disentangling complex phenotypes in diverse bacterial species. In Chapter 2 I describe my analysis of the essential gene functions in the model Gram-negative bacterium Escherichia coli and the model Gram-positive Bacillus subtilis using datasets from paired chemical-genetic screens. In this work I identify both shared and Gram-negative specific mechanisms of collateral sensitization to antibiotic action. In Chapter 3 I investigate a fundamental property of essential genes, which is the relationship between their expression level and the cellular growth rate. Here, further developing CRISPRi tools in bacteria to predictably titrate knockdown efficacy, I interpret the knockdown-fitness relationships of each essential gene in E. coli and B. subtilis, discovering broad conservation of constraints setting and maintaining expression levels across these diverged species

Proceedings of the Third International Workshop on Neural Networks and Fuzzy Logic, volume 1

Documented here are papers presented at the Neural Networks and Fuzzy Logic Workshop sponsored by the National Aeronautics and Space Administration and cosponsored by the University of Houston, Clear Lake. The workshop was held June 1-3, 1992 at the Lyndon B. Johnson Space Center in Houston, Texas. During the three days approximately 50 papers were presented. Technical topics addressed included adaptive systems; learning algorithms; network architectures; vision; robotics; neurobiological connections; speech recognition and synthesis; fuzzy set theory and application, control, and dynamics processing; space applications; fuzzy logic and neural network computers; approximate reasoning; and multiobject decision making

Massively parallel reasoning in transitive relationship hierarchies

This research focuses on building a parallel knowledge representation and reasoning system for the purpose of making progress in realizing human-like intelligence. To achieve human-like intelligence, it is necessary to model human reasoning processes by programs. Knowledge in the real world is huge in size, complex in structure, and is also constantly changing even in limited domains. Unfortunately, reasoning algorithms are very often intractable, which means that they are too slow for any practical applications. One technique to deal with this problem is to design special-purpose reasoners. Many past Al systems have worked rather nicely for limited problem sizes, but attempts to extend them to realistic subsets of world knowledge have led to difficulties. Even special purpose reasoners are not immune to this impasse. In this work, to overcome this problem, we are combining special purpose reasoners with massive We have developed and implemented a massively parallel transitive closure reasoner, called Hydra, that can dynamically assimilate any transitive, binary relation and efficiently answer queries using the transitive closure of all those relations. Within certain limitations, we achieve constant-time responses for transitive closure queries. Hydra can dynamically insert new concepts or new links into a. knowledge base for realistic problem sizes. To get near human-like reasoning capabilities requires the possibility of dynamic updates of the transitive relation hierarchies. Our incremental, massively parallel, update algorithms can achieve almost constant time updates of large knowledge bases. Hydra expands the boundaries of Knowledge Representation and Reasoning in a number of different directions: (1) Hydra improves the representational power of current systems. We have developed a set-based representation for class hierarchies that makes it easy to represent class hierarchies on arrays of processors. Furthermore, we have developed and implemented two methods for mapping this set-based representation onto the processor space of a Connection Machine. These two representations, the Grid Representation and the Double Strand Representation successively improve transitive closure reasoning in terms of speed and processor utilization. (2) Hydra allows fast rerieval and dynamic update of a large knowledge base. New fast update algorithms are formulated to dynamically insert new concepts or new relations into a knowledge base of thousands of nodes. (3) Hydra provides reasoning based on mixed hierarchical representations. We have designed representational tools and massively parallel reasoning algorithms to model reasoning in combined IS-A, Part-of, and Contained-in hierarchies. (4) Hydra\u27s reasoning facilities have been successfully applied to the Medical Entities Dictionary, a large medical vocabulary of Columbia Presbyterian Medical Center. As a result of (1) - (3), Hydra is more general than many current special-purpose reasoners, faster than currently existing general-purpose reasoners, and its knowledge base can be updated dynamically

Regulation of gene expression by RNA binding proteins and microRNAs

Regulation of gene expression is essential to life. Post-transcriptional regulation of gene expression is a complex process with many inputs that lead to changes in localization, translation and stability of mRNAs. The translation and stability of many mRNAs is regulated by cis-elements, such as mRNA-structure or codon optimality; and by trans-acting factors such as RBPs and miRNAs. Here I report on the complex interactions between RBPs, miRNAs and characteristics of their target mRNAs in respect to effects on translation and RNA stability. Using a reporter based approach we studied modulation of microRNA-mediated repression by various mRNA characteristics. We observed the influence of codon optimality, 5’UTR structure, uORFs and translation efficiency on the magnitude of miRNA-mediated repression. To study functional interactions between RBPs and miRNAs, we developed a new method: PTRE-seq. This method utilizes a massively parallel reporter library to study the individual and combined effects of RBPs and miRNAs on translation and RNA stability. Using PTRE-seq we observed epistatic interactions between AU-rich elements and miRNA binding sites. In addition to PTRE-seq, we developed a novel method for immunoprecipitation of mRNAs that will facilitate the identification of miRNAs and RBPs bound to mRNAs of interest