General Program Synthesis from Examples Using Genetic Programming with Parent Selection Based on Random Lexicographic Orderings of Test Cases

Software developers routinely create tests before writing code, to ensure that their programs fulfill their requirements. Instead of having human programmers write the code to meet these tests, automatic program synthesis systems can create programs to meet specifications without human intervention, only requiring examples of desired behavior. In the long-term, we envision using genetic programming to synthesize large pieces of software. This dissertation takes steps toward this goal by investigating the ability of genetic programming to solve introductory computer science programming problems. We present a suite of 29 benchmark problems intended to test general program synthesis systems, which we systematically selected from sources of introductory computer science programming problems. This suite is suitable for experiments with any program synthesis system driven by input/output examples. Unlike existing benchmarks that concentrate on constrained problem domains such as list manipulation, symbolic regression, or boolean functions, this suite contains general programming problems that require a range of programming constructs, such as multiple data types and data structures, control flow statements, and I/O. The problems encompass a range of difficulties and requirements as necessary to thoroughly assess the capabilities of a program synthesis system. Besides describing the specifications for each problem, we make recommendations for experimental protocols and statistical methods to use with the problems. This dissertation\u27s second contribution is an investigation of behavior-based parent selection in genetic programming, concentrating on a new method called lexicase selection. Most parent selection techniques aggregate errors from test cases to compute a single scalar fitness value; lexicase selection instead treats test cases separately, never comparing error values of different test cases. This property allows it to select parents that specialize on some test cases even if they perform poorly on others. We compare lexicase selection to other parent selection techniques on our benchmark suite, showing better performance for lexicase selection. After observing that lexicase selection increases exploration of the search space while also increasing exploitation of promising programs, we conduct a range of experiments to identify which characteristics of lexicase selection influence its utility

MTGP: Combining Metamorphic Testing and Genetic Programming

Genetic programming is an evolutionary approach known for its performance in program synthesis. However, it is not yet mature enough for a practical use in real-world software development, since usually many training cases are required to generate programs that generalize to unseen test cases. As in practice, the training cases have to be expensively hand-labeled by the user, we need an approach to check the program behavior with a lower number of training cases. Metamorphic testing needs no labeled input/output examples. Instead, the program is executed multiple times, first on a given (randomly generated) input, followed by related inputs to check whether certain user-defined relations between the observed outputs hold. In this work, we suggest MTGP, which combines metamorphic testing and genetic programming and study its performance and the generalizability of the generated programs. Further, we analyze how the generalizability depends on the number of given labeled training cases. We find that using metamorphic testing combined with labeled training cases leads to a higher generalization rate than the use of labeled training cases alone in almost all studied configurations. Consequently, we recommend researchers to use metamorphic testing in their systems if the labeling of the training data is expensive

Lexicase Selection for Multi-Task Evolutionary Robotics

In Evolutionary Robotics, Lexicase selection has proven effective when a single task is broken down into many individual parameterizations. Evolved individuals have generalized across unique configurations of an overarching task. Here, we investigate the ability of Lexicase selection to generalize across multiple tasks, with each task again broken down into many instances. There are three objectives: to determine the feasibility of introducing additional tasks to the existing platform; to investigate any consequential effects of introducing these additional tasks during evolutionary adaptation; and to explore whether the schedule of presentation of the additional tasks over evolutionary time affects the final outcome. To address these aims we use a quadruped animat controlled by a feed-forward neural network with joint-angle, bearing-to-target, and spontaneous sinusoidal inputs. Weights in this network are trained using evolution with Lexicase-based parent selection. Simultaneous adaptation in a wall crossing task (labelled wall-cross) is explored when one of two different alternative tasks is also present: turn-and-seek or cargo-carry. Each task is parameterized into 100 distinct variants, and these variants are used as environments for evaluation and selection with Lexicase. We use performance in a single-task wall-cross environment as a baseline against which to examine the multi-task configurations. In addition, the objective sampling strategy (the manner in which tasks are presented over evolutionary time) is varied, and so data for treatments implementing uniform sampling, even sampling, or degrees of generational sampling are also presented. The Lexicase mechanism successfully integrates evolution of both turn-and-seek and cargo-carry with wall-cross, though there is a performance penalty compared to single task evolution. The size of the penalty depends on the similarity of the tasks. Complementary tasks (wallcross/turn-and-seek) show better performance than antagonistic tasks (wall-cross/cargo-carry). In complementary tasks performance is not affected by the sampling strategy. Where tasks are antagonistic, uniform and even sampling strategies yield significantly better performance than generational sampling. In all cases the generational sampling requires more evaluations and consequently more computational resources. The results indicate that Lexicase is a viable mechanism for multitask evolution of animat neurocontrollers, though the degree of interference between tasks is a key consideration. The results also support the conclusion that the naive, uniform random sampling strategy is the best choice when considering final task performance, simplicity of implementation, and computational efficiency

Enhancing Inter-Document Similarity Using Sub Max

Document similarity, a core theme in Information Retrieval (IR), is a machine learning (ML) task associated with natural language processing (NLP). It is a measure of the distance between two documents given a set of rules. For the purpose of this thesis, two documents are similar if they are semantically alike, and describe similar concepts. While document similarity can be applied to multiple tasks, we focus our work on the accuracy of models in detecting referenced papers as similar documents using their sub max similarity. Multiple approaches have been used to determine the similarity of documents in regards to literature reviews. Some of such approaches use the number of similar citations, the similarity between the body of text, and the figures present in those documents. This researcher hypothesized that documents with sections of high similarity(sub max) but a global low similarity are prone to being overlooked by existing models as the global score of the documents are used to measure similarity. In this study, we aim to detect, measure, and show the similarity of documents based on the maximum similarity of their subsections. The sub max of any two given documents is the subsections of those documents with the highest similarity. By comparing subsections of the documents in our corpus and using the sub max, we were able to improve the performance of some models by over 100%

Philippine Linguistics: The State of the Art: 1970-1980

In 1971, two articles appeared (McKaughan 1971, and Constantino 1971c) which together summarized the state of the art of Philippine linguistics until that time. Another decade has passed and it seems an appropriate time to review what has been accomplished in the last ten years, to see how the field has developed and to suggest directions for the future. The first section of this chapter provides an informal overview of the settings within which the study of Philippine languages is taking place. This section primarily discusses the main organizations, and some of the individuals, that have made major contributions to the field in the seventies. The following section begins with an overview of the grammars and dictionaries that have appeared in the last decade. It then discusses the main contributions that have been made within each of the four major subfields in the discipline: phonology; morphology and syntax; discourse studies (including sentence and paragraph); and comparative-historical linguistics. The concluding section assesses the significance for the field, of current events in the Philippines, and suggests possible responses

Genetic Improvement of Software: From Program Landscapes to the Automatic Improvement of a Live System

In today’s technology driven society, software is becoming increasingly important in more areas of our lives. The domain of software extends beyond the obvious domain of computers, tablets, and mobile phones. Smart devices and the internet-of-things have inspired the integra- tion of digital and computational technology into objects that some of us would never have guessed could be possible or even necessary. Fridges and freezers connected to social media sites, a toaster activated with a mobile phone, physical buttons for shopping, and verbally asking smart speakers to order a meal to be delivered. This is the world we live in and it is an exciting time for software engineers and computer scientists. The sheer volume of code that is currently in use has long since outgrown beyond the point of any hope for proper manual maintenance. The rate of which mobile application stores such as Google’s and Apple’s have expanded is astounding. The research presented here aims to shed a light on an emerging field of research, called Genetic Improvement ( GI ) of software. It is a methodology to change program code to improve existing software. This thesis details a framework for GI that is then applied to explore fitness landscape of bug fixing Python software, reduce execution time in a C ++ program, and integrated into a live system. We show that software is generally not fragile and although fitness landscapes for GI are flat they are not impossible to search in. This conclusion applies equally to bug fixing in small programs as well as execution time improvements. The framework’s application is shown to be transportable between programming languages with minimal effort. Additionally, it can be easily integrated into a system that runs a live web service. The work within this thesis was funded by EPSRC grant EP/J017515/1 through the DAASE project