Ein kombinierter analytischer und suchbasierter Ansatz zur induktiven Synthese funktionaler Programme

This thesis is concerned with the inductive synthesis of recursive declarative programs and in particular with the analytical inductive synthesis of functional programs. Program synthesis addresses the problem of (semi-)automatically generating computer programs from specifications. In inductive program synthesis, recursive programs are constructed by generalizing over incomplete specifications such as finite sets of input/output examples (I/O examples). Classical methods for induction of functional programs are analytical, that is, a recursive function definition is derived by detecting and generalizing recurrent patterns between the given I/O examples. Most recent methods, on the other side, are generate-and-test based, that is, they repeatedly generate programs independently from the provided I/O examples until a program is found that correctly computes the examples. Analytical methods are much faster than generate-and-test methods, because they do not rely on search in a program space. Therefore, however, the schemas that generatable programs conform to, must be much more restricted. This thesis at first provides a comprehensive overview of current approaches and methods to inductive program synthesis. Then we present a new algorithm to the inductive synthesis of functional programs that generalizes the analytical approach and combines it with search in a program space. Thereby, the strong restrictions of analytical methods can be resolved for the most part. At the same time, applying analytical techniques allows for pruning large parts of the problem space so that solutions can often be found faster than with generate-and-test methods. By means of several experiments with an implementation of the described algorithm, we demonstrate its capabilities.Diese Arbeit befasst sich mit der induktiven Synthese rekursiver deklarativer Programme und speziell mit der analytischen induktiven Synthese funktionaler Programme. Die Programmsynthese beschäftigt sich mit der (semi-)automatischen Konstruktion von Computer-Programmen aus Spezifikationen. In der induktiven Programmsynthese werden rekursive Programme durch das Generalisieren über unvollständige Spezifikationen, wie zum Beispiel endliche Mengen von Eingabe/Ausgabe Beispielen (E/A-Beispielen), generiert. Klassische Methoden der induktiven Synthese funktionaler Programme sind analytisch; eine rekursive Funktionsdefinition wird generiert, indem rekurrente Strukturen zwischen den einzelnen E/A-Beispielen gefunden und generalisiert werden. Die meisten aktuellen Ansätze basieren hingegen auf erzeugen und testen, das heißt, es werden unabhängig von den bereitgestellten E/A-Beispielen solange Programme einer Klasse generiert, bis schließlich ein Programm gefunden wurde das alle Beispiele korrekt berechnet. Analytische Methoden sind sehr viel schneller, weil sie nicht auf Suche in einem Programmraum beruhen. Allerdings müssen dafür auch die Schemata, denen die generierbaren Programme gehorchen, sehr viel beschränkter sein. Diese Arbeit bietet zunächst einen umfassenden Überblick über bestehende Ansätze und Methoden der induktiven Programmsynthese. Anschließend wird ein neuer Algorithmus zur induktiven Synthese funktionaler Programme beschrieben, der den analytischen Ansatz generalisiert und mit Suche in einem Programmraum kombiniert. Dadurch lassen sich die starken Restriktionen des analytischen Ansatzes zu großen Teilen überwinden. Gleichzeitig erlaubt der Einsatz analytischer Techniken das Beschneiden großer Teile des Problemraums, so dass Lösungsprogramme oft schneller gefunden werden können als mit Methoden, die auf erzeugen und testen beruhen. Mittels einer Reihe von Experimenten mit einer Implementation des beschriebenen Algorithmus' werden seine Möglichkeiten gezeigt

Inductive programming meets the real world

© Gulwani, S. et al. | ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Communications of the ACM, http://dx.doi.org/10.1145/2736282[EN] Since most end users lack programming skills they often spend considerable time and effort performing tedious and repetitive tasks such as capitalizing a column of names manually. Inductive Programming has a long research tradition and recent developments demonstrate it can liberate users from many tasks of this kind.Gulwani, S.; Hernández-Orallo, J.; Kitzelmann, E.; Muggleton, SH.; Schmid, U.; Zorn, B. (2015). Inductive programming meets the real world. Communications of the ACM. 58(11):90-99. doi:10.1145/2736282S90995811Bengio, Y., Courville, A. and Vincent, P. Representation learning: A review and new perspectives.Pattern Analy. Machine Intell. 35, 8 (2013), 1798--1828.Bielawski, B. Using the convertfrom-string cmdlet to parse structured text.PowerShell Magazine, (Sept. 9, 2004); http://www.powershellmagazine.com/2014/09/09/using-the-convertfrom-string-cmdlet-to-parse-structured-text/Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka-Jr, E.R. and T.M. Mitchell, T.M. Toward an architecture for never-ending language learning. InAAAI, 2010.Chandola, V., Banerjee, A. and V. Kumar, V. Anomaly detection: A survey.ACM Computing Surveys 41, 3 (2009), 15.Cypher, A. (Ed).Watch What I Do: Programming by Demonstration.MIT Press, Cambridge, MA, 1993.Ferri-Ramírez, C., Hernández-Orallo, J. and Ramírez-Quintana, M.J. Incremental learning of functional logic programs. InProceedings of FLOPS, 2001, 233--247.Flener, P. and Schmid, U. An introduction to inductive programming.AI Review 29, 1 (2009), 45--62.Gulwani, S. Dimensions in program synthesis. InProceedings of PPDP, 2010.Gulwani, S. Automating string processing in spreadsheets using input-output examples. InProceedings of POPL, 2011; http://research.microsoft.com/users/sumitg/flashfill.html.Gulwani, S. Example-based learning in computer-aided STEM education.Commun. ACM 57, 8 (Aug 2014), 70--80.Gulwani, S., Harris, W. and Singh, R. Spreadsheet data manipulation using examples.Commun. ACM 55, 8 (Aug. 2012), 97--105.Henderson, R.J. and Muggleton, S.H. Automatic invention of functional abstractions.Latest Advances in Inductive Logic Programming, 2012.Hernández-Orallo, J. Deep knowledge: Inductive programming as an answer, Dagstuhl TR 13502, 2013.Hofmann, M. and Kitzelmann, E. I/O guided detection of list catamorphisms---towards problem specific use of program templates in IP. InACM SIGPLAN PEPM, 2010.Jha, J., Gulwani, S., Seshia, S. and Tiwari, A. Oracle-guided component-based program synthesis. InProceedings of the ICSE, 2010.Katayama, S. Efficient exhaustive generation of functional programs using Monte-Carlo search with iterative deepening. InProceedings of PRICAI, 2008.Kitzelmann, E. Analytical inductive functional programming.LOPSTR 2008, LNCS 5438.Springer, 2009, 87--102.Kitzelmann, E. Inductive programming: A survey of program synthesis techniques. InAAIP, Springer, 2010, 50--73.Kitzelmann, E. and Schmid, U. Inductive synthesis of functional programs: An explanation based generalization approach.J. Machine Learning Research 7, (Feb. 2006), 429--454.Kotovsky, K., Hayes, J.R. and Simon, H.A. Why are some problems hard? Evidence from Tower of Hanoi.Cognitive Psychology 17, 2 (1985), 248--294.Lau, T.A. Why programming-by-demonstration systems fail: Lessons learned for usable AI.AI Mag. 30, 4, (2009), 65--67.Lau, T.A., Wolfman, S.A., Domingos, P. and Weld, D.S. Programming by demonstration using version space algebra.Machine Learning 53, 1-2 (2003), 111--156.Le, V. and Gulwani, S. FlashExtract: A framework for data extraction by examples. InProceedings of PLDI, 2014.Lieberman, H. (Ed).Your Wish is My Command: Programming by Example.Morgan Kaufmann, 2001.Lin, D., Dechter, E., Ellis, K., Tenenbaum, J.B. and Muggleton, S.H. Bias reformulation for one-shot function induction. InProceedings of ECAI, 2014.Marcus, G.F. The Algebraic Mind.Integrating Connectionism and Cognitive Science.Bradford, Cambridge, MA, 2001.Martìnez-Plumed, C. Ferri, Hernández-Orallo, J. and M.J. Ramírez-Quintana. On the definition of a general learning system with user-defined operators.arXiv preprint arXiv:1311.4235, 2013.Menon, A., Tamuz, O., Gulwani, S., Lampson, B. and Kalai, A. A machine learning framework for programming by example. InProceedings of the ICML, 2013.Miller, R.C. and Myers, B.A. Multiple selections in smart text editing. InProceedings of IUI, 2002, 103--110.Muggleton, S.H. Inductive Logic Programming.New Generation Computing 8, 4 (1991), 295--318.Muggleton, S.H. and Lin, D. Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited.IJCAI 2013, 1551--1557.Muggleton, S.H., Lin, D., Pahlavi, N. and Tamaddoni-Nezhad, A. Meta-interpretive learning: application to grammatical inference.Machine Learning 94(2014), 25--49.Muggleton, S.H., De Raedt, L., Poole, D., Bratko, I., Flach, P. and Inoue, P. ILP turns 20: Biography and future challenges.Machine Learning 86, 1 (2011), 3--23.Olsson, R. Inductive functional programming using incremental program transformation.Artificial Intelligence 74, 1 (1995), 55--83.Perelman, D., Gulwani, S., Grossman, D. and Provost, P. Test-driven synthesis.PLDI, 2014.Raza, M., Gulwani, S. and Milic-Frayling, N. Programming by example using least general generalizations.AAAI, 2014.Schmid, U. and Kitzelmann, E. Inductive rule learning on the knowledge level.Cognitive Systems Research 12, 3 (2011), 237--248.Schmid, U. and Wysotzki, F. Induction of recursive program schemes.ECML 1398 LNAI(1998), 214--225.Shapiro, E.Y. An algorithm that infers theories from facts.IJCAI(1981), 446--451.Solar-Lezama, A.Program Synthesis by Sketching.Ph.D thesis, UC Berkeley, 2008.Summers, P.D. A methodology for LISP program construction from examples.JACM 24, 1 (1977), 162--175.Tenenbaum, J.B., Griffiths, T.L. and Kemp, C. Theory-based Bayesian models of inductive learning and reasoning.Trends in Cognitive Sciences 10, 7 (2006), 309--318.Young, S. Cognitive user interfaces.IEEE Signal Processing 27, 3 (2010), 128--140

Proceedings of the ACM SIGPLAN Workshop on Approaches and Applications of Inductive Programming (AAIP 2009)

Inductive programming is concerned with the automated construction of declarative, often functional, recursive programs from incomplete specifications such as input/output examples. The inferred program must be correct with respect to the provided examples in a generalising sense: it should be neither equivalent to them, nor inconsistent. Inductive programming algorithms are guided explicitly or implicitly by a language bias (the class of programs that can be induced) and a search bias (determining which generalised program is constructed first). Induction strategies are either generate-and-test or example-driven. In generate-and-test approaches, hypotheses about candidate programs are generated independently from the given specifications. Program candidates are tested against the given specification and one or more of the best evaluated candidates are developed further. In analytical approaches, candidate programs are constructed in an example-driven way. While generate-and-test approaches can -- in principle -- construct any kind of program, analytical approaches have a more limited scope. On the other hand, efficiency of induction is much higher in analytical approaches. Inductive programming is still mainly a topic of basic research, exploring how the intellectual ability of humans to infer generalised recursive procedures from incomplete evidence can be captured in the form of synthesis methods. Intended applications are mainly in the domain of programming assistance -- either to relieve professional programmers from routine tasks or to enable non-programmers to some limited form of end-user programming. Furthermore, in the future, inductive programming techniques might be applied to further areas such as supporting the inference of lemmata in theorem proving or learning grammar rules. Inductive automated program construction has been originally addressed by researchers in artificial intelligence and machine learning. During the last years, some work on exploiting induction techniques has been started also in the functional programming community. Therefore, the third workshop on |Approaches and Applications of Inductive Programming| took place for the first time in conjunction with the ACM SIGPLAN International Conference on Functional Programming (ICFP 2009). The first and second workshop were associated with the International Conference on Machine Learning (ICML 2005) and the European Conference on Machine Learning (ECML 2007). AAIP´09 aimed to bring together researchers from the functional programming and the artificial intelligence communities, working in the field of inductive functional programming, and advance fruitful interactions between these communities with respect to programming techniques for inductive programming algorithms, the identification of challenge problems and potential applications. For everybody interested in inductive programming we recommend to visit the website: www.inductive-programming.org

Abstract Learning Frameworks for Synthesis

We develop abstract learning frameworks (ALFs) for synthesis that embody the principles of CEGIS (counter-example based inductive synthesis) strategies that have become widely applicable in recent years. Our framework defines a general abstract framework of iterative learning, based on a hypothesis space that captures the synthesized objects, a sample space that forms the space on which induction is performed, and a concept space that abstractly defines the semantics of the learning process. We show that a variety of synthesis algorithms in current literature can be embedded in this general framework. While studying these embeddings, we also generalize some of the synthesis problems these instances are of, resulting in new ways of looking at synthesis problems using learning. We also investigate convergence issues for the general framework, and exhibit three recipes for convergence in finite time. The first two recipes generalize current techniques for convergence used by existing synthesis engines. The third technique is a more involved technique of which we know of no existing instantiation, and we instantiate it to concrete synthesis problems

Inductive programming meets the real world

Since most end users lack programming skills they often spend considerable time and effort performing tedious and repetitive tasks such as capitalizing a column of names manually. Inductive Programming has a long research tradition and recent developments demonstrate it can liberate users from many tasks of this kind. Key insights • Real-world applications emerge with spreadsheet tools, scripting, and intelligent program tutors. • Learning from few examples is possible because users and systems share the same background knowledge. • Search is guided by domain-specific languages and the use of higher-order knowledge. Much of the world's population use computers for everyday tasks, but most fail to benefit from the power of computation due to their inability to program. Most crucially, users often have to perform repetitive actions manually because they are not able to use the macro languages which are available for many application programs. Recently, a first mass-market product was presented in the form of the Flash Fill feature in Microsoft Excel 2013. Flash Fill allows end users to automatically generate string processing programs for spreadsheets from one or more user-provided examples. Flash Fill is able to learn a large variety of quite complex programs from only a few examples because of incorporation of inductive programming methods. Inductive Programming (IP) is an inter-disciplinary domain of research in computer science, artificial intelligence, and cognitive science that studies the automatic synthesis of computer programs from examples and background knowledge. IP developed from research on inductive program synthesis, now called inductive functional programming (IFP), and from inductive inference techniques using logic, nowadays termed inductive logic programming (ILP). IFP addresses the synthesis of recursive functional programs generalized from regularities detected in (traces of) input/output examples ILP originated from research on induction in a logical framework Over the last decade Inductive Programming has attracted a series of international workshops. Recent surveys In the domain of end-user programming, programming by demonstration approaches were proposed which support the learning of small routines from observing the input behavior of users In this paper, several of these current applications are presented. We contrast the specific characteristics of IP with those of typical machine learning approaches and we show how IP is related to cognitive models of human inductive learning. We finally discuss recent techniques -such as use of domain-specific languages and meta-level learning-that widen the scope and power of IP and discuss new challenges. REAL-WORLD APPLICATIONS Originally, IP was applied to synthesizing functional or logic programs for general purpose tasks such as manipulating data structures (e.g., sorting or reversing a list). These investigations showed that small programs could be synthesized from a few input/output examples. The recent IT revolution has created real-world opportunities for such techniques. Most of today's large number of computer users are non-programmers and are limited to being passive consumers of the software that is made available to them. IP can empower such users to more effectively leverage computers for automating their daily repetitive tasks. We discuss below some such opportunities, especially in the areas of End-user Programming and Education