Layered genetic programming for feature extraction in classification problems

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsGenetic programming has been proven to be a successful technique for feature extraction in various applications. In this thesis, we present a Layered Genetic Programming system which implements genetic programming-based feature extraction mechanism. The proposed system uses a layered structure where instead of evolving just one population of individuals, several populations are evolved sequentially. Each such population transforms the input data received from the previous population into a lower dimensional space with the aim of improving classification performance. The performance of the proposed system was experimentally tested on 5 real-world problems using different dimensionality reduction step sizes and different classifiers. The proposed method was able to outperform a simple classifier applied directly on the original data on two problems. On the remaining problems, the classifier performed better using the original data. The best solutions were often obtained in the first few layers which implied that increasing the size of the system, i.e. adding more layers was not useful. However, the layered structure allowed control of the size of individuals

Evolving recursive programs by using adaptive grammar based genetic programming

Genetic programming (GP) extends traditional genetic algorithms to automatically induce computer programs. GP has been applied in a wide range of applications such as software re-engineering, electrical circuits synthesis, knowledge engineering, and data mining. One of the most important and challenging research areas in GP is the investigation of ways to successfully evolve recursive programs. A recursive program is one that calls itself either directly or indirectly through other programs. Because recursions lead to compact and general programs and provide a mechanism for reusing program code, they facilitate GP to solve larger and more complicated problems. Nevertheless, it is commonly agreed that the recursive program learning problem is very difficult for GP. In this paper, we propose techniques to tackle the difficulties in learning recursive programs. The techniques are incorporated into an adaptive Grammar Based Genetic Programming system (adaptive GBGP). A number of experiments have been performed to demonstrate that the system improves the effectiveness and efficiency in evolving recursive programs

Annual report on research activities 2005-2006

https://commons.ln.edu.hk/research_annual_report/1004/thumbnail.jp

Schemagesteuerte Induktive Funktionale Programmsynthese durch Automatische Erkennung von Typmorphismen

Inductive functional programming systems can be characterised by two diametric approaches: Either they apply exhaustive program enumeration which uses input/output examples (IO) as test cases, or they perform an analytical, data-driven structural generalisation of the IO examples. Enumerative approaches ignore the structural information provided with the IO examples, but use type information to guide and restrict the search. They use higher-order functions which capture recursion schemes during their enumeration, but apply them randomly in a uninformed manner. Analytical approaches on the other side heavily exploit this structural information, but have ignored the benefits of a strong type system so far and use only recursion schemes either fixed and built in, or selected by an expert user. In category theory universal constructs, such as natural transformations or type morphisms, describe recursion schemes which can be defined on any inductively defined data type. They can be characterised by specific universal properties. Those type morphisms and related concepts provide a categorical approach to functional programming, which is often called categorical programming. This work shows how categorical programming can be applied to Inductive Programming and how universal constructs, such as catamorphisms, paramorphisms, and type functors, can be used as recursive program schemes for inductive functional programming. The use of program schemes for Inductive Programming is not new. The special appeal and novelty of this work is that, contrary to previous approaches, the program schemes are neither fixed, nor selected by an expert user: The applicability of those recursion schemes can be automatically detected in the given IO examples of a target function by checking the universal properties of the corresponding type morphisms. Applying this to the analytical system Igor2, both the capabilities and the expressiveness can be extended without a decrease in efficiency. An extension of the analytical functional inductive programming system Igor2 is proposed and its algorithms described. An empirical evaluation demonstrates the improvements with respect to efficiency and effectiveness that can be achieved by the use of type morphisms for Igor2 due to a reduction in search space complexity.Systeme zur induktiven Programmsynthese können bezüglich zweier gegensätzlicher Ansätze beschrieben werden: Enumerative Systeme zählen Programme vollständig auf und verwenden Eingabe/Ausgabe Beispiele (E/A) lediglich zum Testen; analytische, datengetriebene Systeme hingegegen generieren ein Programm durch strukturelle Generalisierung der E/A Beispiele. Aufzählende Ansätze ignorieren die in den E/A Beispielen enthaltene strukturelle Information völlig, benutzen aber Typinformation, um den Suchraum zu beschränken und die Suche zu steuern. Sie verwenden Funktionen höherer Ordnung als rekursive Programmschemata während der Aufzählung, wenden diese aber beliebig und nicht zielgerichtet an. Analytische Ansätze hingegen nutzen extensiv die strukturelle Information der E/A Beispiele, vernachlässigen aber die Vorzüge eines starken Typsystems. Programmschemata verwenden sie lediglich starr und fest codiert oder durch Auswahl eines Experten. In der Kategorientheorie beschreiben universelle Konstrukte wie zum Beispiel natürliche Transformationen und Typmorphismen Rekursionsschemata auf beliebigen, induktiv definierten Datentypen. Diese Konstrukte zeichnen sich durch spezifische, universelle Eigenschaften aus. Derartige Typmorphismen bieten einen kategorientheoretischen Zugang zur funktionalen Programmierung. Diese Arbeit zeigt, wie Catamorphismen, Paramorphismen und Typfunktoren als universelle Konstrukte in der induktiven Programmsynthese als rekursive Programmschemata verwendet werden können. Die Verwendung von Schemata in der induktiven Programmierung ist an sich nichts Neues, die Innovation liegt jedoch in der Art und Weise der Einführung der Schemata. Im Gegensatz zu herkömmlichen Ansätzen wird weder ein festes Schema verwendet, noch wählt ein Experte ein Schema aus. Die vorliegende Arbeit zeigt, dass die Anwendbarkeit eines bestimmten Schemas sich aus den E/A Beispielen einer konkreten Zielfunktion ableiten lässt, wenn man die universellen Eigenschaften das dem Programmschema entsprechenden Typmorphismus in den Beispielen erfüllen kann. Im Folgenden wird eine Erweiterung des funktionalen, induktiven Programmsynthesesystems Igor2 vorgestellt und der neue Algorithmus beschrieben. Ein empirischer Vergleich untermauert die Vorzüge der Erweiterung und macht die Steigerung der Effizienz und der Effektivität, die durch die Verwendung von Typmorphismen durch Komplexitätsreduktion des Suchraums erzielt werden kann, deutlich