Programming Not Only by Example

In recent years, there has been tremendous progress in automated synthesis techniques that are able to automatically generate code based on some intent expressed by the programmer. A major challenge for the adoption of synthesis remains in having the programmer communicate their intent. When the expressed intent is coarse-grained (for example, restriction on the expected type of an expression), the synthesizer often produces a long list of results for the programmer to choose from, shifting the heavy-lifting to the user. An alternative approach, successfully used in end-user synthesis is programming by example (PBE), where the user leverages examples to interactively and iteratively refine the intent. However, using only examples is not expressive enough for programmers, who can observe the generated program and refine the intent by directly relating to parts of the generated program. We present a novel approach to interacting with a synthesizer using a granular interaction model. Our approach employs a rich interaction model where (i) the synthesizer decorates a candidate program with debug information that assists in understanding the program and identifying good or bad parts, and (ii) the user is allowed to provide feedback not only on the expected output of a program, but also on the underlying program itself. That is, when the user identifies a program as (partially) correct or incorrect, they can also explicitly indicate the good or bad parts, to allow the synthesizer to accept or discard parts of the program instead of discarding the program as a whole. We show the value of our approach in a controlled user study. Our study shows that participants have strong preference to using granular feedback instead of examples, and are able to provide granular feedback much faster

User Interaction Models for Disambiguation in Programming by Example

Programming by Examples (PBE) has the potential to revolutionize end-user programming by enabling end users, most of whom are non-programmers, to create small scripts for automating repetitive tasks. However, examples, though often easy to provide, are an ambiguous specification of the user's intent. Because of that, a key impedance in adoption of PBE systems is the lack of user confidence in the correctness of the program that was synthesized by the system. We present two novel user interaction models that communicate actionable information to the user to help resolve ambiguity in the examples. One of these models allows the user to effectively navigate between the huge set of programs that are consistent with the examples provided by the user. The other model uses active learning to ask directed example-based questions to the user on the test input data over which the user intends to run the synthesized program. Our user studies show that each of these models significantly reduces the number of errors in the performed task without any difference in completion time. Moreover, both models are perceived as useful, and the proactive active-learning based model has a slightly higher preference regarding the users' confidence in the result

Interactive Programming by Example

As of today, programming has never been so accessible. Yet, it remains a challenge for end-users: students, non-technical employees, experts in their domains outside of computer science, and so on. With its forecast potential for solving problems by only observing inputs and outputs, programming-by-example was supposed to alleviate complex tasks requiring programming for end-users. The initial ideas of macro-based editors paved the way to subsequent practical solutions, such as spreadsheet transformations from examples. Finding the right program is the core of the programming-by-example systems. However, users find it difficult to trust such generated programs. In this thesis, we contribute to proving that some forms of interaction alleviate, by having users provide examples, the problem of finding correct and reliable programs. We first report on two experiments that enable us to conjecture what kind of interaction brings benefits to programming-by-example. First, we present a new kind of game engine, Pong Designer. In this game engine, by using their finger, users program rules on the fly, by modifying the game state. We analyze its potential, and its eventual downsides that have probably prevented its wide adoption. Second, we present StriSynth, an interactive command-line tool that uses programming-by-example to transform string and collections. The resulting programs can also rename or otherwise manage files. We obtained the result that confirms that many users preferred StriSynth over usual programming languages, but would appreciate to have both. We then report on two new exciting experiments with verified results, using two forms of interaction truly benefiting programming-by-example. Third, on top of a programmingby- example-based engine for extracting structured data out of text files, in this thesis we study two interaction models implemented in a tool named FlashProg: a view of the program with notification about ambiguities, and the asking of clarification questions. In this thesis, we prove that these two interaction models enable users to perform tasks with less errors and to be more confident with the results. Last, for learning recursive tree-to-string functions (e.g., pretty-printers), in this thesis we prove that questioning breaks down the learning complexity from a cubic to a linear number of questions, in practice making programming-by-example even more accessible than regular programming. The implementation, named Prosy, could be easily added to integrated development environments

Schemagesteuerte Induktive Funktionale Programmsynthese durch Automatische Erkennung von Typmorphismen

Inductive functional programming systems can be characterised by two diametric approaches: Either they apply exhaustive program enumeration which uses input/output examples (IO) as test cases, or they perform an analytical, data-driven structural generalisation of the IO examples. Enumerative approaches ignore the structural information provided with the IO examples, but use type information to guide and restrict the search. They use higher-order functions which capture recursion schemes during their enumeration, but apply them randomly in a uninformed manner. Analytical approaches on the other side heavily exploit this structural information, but have ignored the benefits of a strong type system so far and use only recursion schemes either fixed and built in, or selected by an expert user. In category theory universal constructs, such as natural transformations or type morphisms, describe recursion schemes which can be defined on any inductively defined data type. They can be characterised by specific universal properties. Those type morphisms and related concepts provide a categorical approach to functional programming, which is often called categorical programming. This work shows how categorical programming can be applied to Inductive Programming and how universal constructs, such as catamorphisms, paramorphisms, and type functors, can be used as recursive program schemes for inductive functional programming. The use of program schemes for Inductive Programming is not new. The special appeal and novelty of this work is that, contrary to previous approaches, the program schemes are neither fixed, nor selected by an expert user: The applicability of those recursion schemes can be automatically detected in the given IO examples of a target function by checking the universal properties of the corresponding type morphisms. Applying this to the analytical system Igor2, both the capabilities and the expressiveness can be extended without a decrease in efficiency. An extension of the analytical functional inductive programming system Igor2 is proposed and its algorithms described. An empirical evaluation demonstrates the improvements with respect to efficiency and effectiveness that can be achieved by the use of type morphisms for Igor2 due to a reduction in search space complexity.Systeme zur induktiven Programmsynthese können bezüglich zweier gegensätzlicher Ansätze beschrieben werden: Enumerative Systeme zählen Programme vollständig auf und verwenden Eingabe/Ausgabe Beispiele (E/A) lediglich zum Testen; analytische, datengetriebene Systeme hingegegen generieren ein Programm durch strukturelle Generalisierung der E/A Beispiele. Aufzählende Ansätze ignorieren die in den E/A Beispielen enthaltene strukturelle Information völlig, benutzen aber Typinformation, um den Suchraum zu beschränken und die Suche zu steuern. Sie verwenden Funktionen höherer Ordnung als rekursive Programmschemata während der Aufzählung, wenden diese aber beliebig und nicht zielgerichtet an. Analytische Ansätze hingegen nutzen extensiv die strukturelle Information der E/A Beispiele, vernachlässigen aber die Vorzüge eines starken Typsystems. Programmschemata verwenden sie lediglich starr und fest codiert oder durch Auswahl eines Experten. In der Kategorientheorie beschreiben universelle Konstrukte wie zum Beispiel natürliche Transformationen und Typmorphismen Rekursionsschemata auf beliebigen, induktiv definierten Datentypen. Diese Konstrukte zeichnen sich durch spezifische, universelle Eigenschaften aus. Derartige Typmorphismen bieten einen kategorientheoretischen Zugang zur funktionalen Programmierung. Diese Arbeit zeigt, wie Catamorphismen, Paramorphismen und Typfunktoren als universelle Konstrukte in der induktiven Programmsynthese als rekursive Programmschemata verwendet werden können. Die Verwendung von Schemata in der induktiven Programmierung ist an sich nichts Neues, die Innovation liegt jedoch in der Art und Weise der Einführung der Schemata. Im Gegensatz zu herkömmlichen Ansätzen wird weder ein festes Schema verwendet, noch wählt ein Experte ein Schema aus. Die vorliegende Arbeit zeigt, dass die Anwendbarkeit eines bestimmten Schemas sich aus den E/A Beispielen einer konkreten Zielfunktion ableiten lässt, wenn man die universellen Eigenschaften das dem Programmschema entsprechenden Typmorphismus in den Beispielen erfüllen kann. Im Folgenden wird eine Erweiterung des funktionalen, induktiven Programmsynthesesystems Igor2 vorgestellt und der neue Algorithmus beschrieben. Ein empirischer Vergleich untermauert die Vorzüge der Erweiterung und macht die Steigerung der Effizienz und der Effektivität, die durch die Verwendung von Typmorphismen durch Komplexitätsreduktion des Suchraums erzielt werden kann, deutlich