Towards Data Wrangling Automation through Dynamically-Selected Background Knowledge

[ES] El proceso de ciencia de datos es esencial para extraer valor de los datos. Sin embargo, la parte más tediosa del proceso, la preparación de los datos, implica una serie de formateos, limpieza e identificación de problemas que principalmente son tareas manuales. La preparación de datos todavía se resiste a la automatización en parte porque el problema depende en gran medida de la información del dominio, que se convierte en un cuello de botella para los sistemas de última generación a medida que aumenta la diversidad de dominios, formatos y estructuras de los datos. En esta tesis nos enfocamos en generar algoritmos que aprovechen el conocimiento del dominio para la automatización de partes del proceso de preparación de datos. Mostramos la forma en que las técnicas generales de inducción de programas, en lugar de los lenguajes específicos del dominio, se pueden aplicar de manera flexible a problemas donde el conocimiento es importante, mediante el uso dinámico de conocimiento específico del dominio. De manera más general, sostenemos que una combinación de enfoques de aprendizaje dinámicos y basados en conocimiento puede conducir a buenas soluciones. Proponemos varias estrategias para seleccionar o construir automáticamente el conocimiento previo apropiado en varios escenarios de preparación de datos. La idea principal se basa en elegir las mejores primitivas especializadas de acuerdo con el contexto del problema particular a resolver. Abordamos dos escenarios. En el primero, manejamos datos personales (nombres, fechas, teléfonos, etc.) que se presentan en formatos de cadena de texto muy diferentes y deben ser transformados a un formato unificado. El problema es cómo construir una transformación compositiva a partir de un gran conjunto de primitivas en el dominio (por ejemplo, manejar meses, años, días de la semana, etc.). Desarrollamos un sistema (BK-ADAPT) que guía la búsqueda a través del conocimiento previo extrayendo varias meta-características de los ejemplos que caracterizan el dominio de la columna. En el segundo escenario, nos enfrentamos a la transformación de matrices de datos en lenguajes de programación genéricos como R, utilizando como ejemplos una matriz de entrada y algunas celdas de la matriz de salida. También desarrollamos un sistema guiado por una búsqueda basada en árboles (AUTOMAT[R]IX) que usa varias restricciones, probabilidades previas para las primitivas y sugerencias textuales, para aprender eficientemente las transformaciones. Con estos sistemas, mostramos que la combinación de programación inductiva, con la selección dinámica de las primitivas apropiadas a partir del conocimiento previo, es capaz de mejorar los resultados de otras herramientas actuales específicas para la preparación de datos.[CA] El procés de ciència de dades és essencial per extraure valor de les dades. No obstant això, la part més tediosa del procés, la preparació de les dades, implica una sèrie de transformacions, neteja i identificació de problemes que principalment són tasques manuals. La preparació de dades encara es resisteix a l'automatització en part perquè el problema depén en gran manera de la informació del domini, que es converteix en un coll de botella per als sistemes d'última generació a mesura que augmenta la diversitat de dominis, formats i estructures de les dades. En aquesta tesi ens enfoquem a generar algorismes que aprofiten el coneixement del domini per a l'automatització de parts del procés de preparació de dades. Mostrem la forma en què les tècniques generals d'inducció de programes, en lloc dels llenguatges específics del domini, es poden aplicar de manera flexible a problemes on el coneixement és important, mitjançant l'ús dinàmic de coneixement específic del domini. De manera més general, sostenim que una combinació d'enfocaments d'aprenentatge dinàmics i basats en coneixement pot conduir a les bones solucions. Proposem diverses estratègies per seleccionar o construir automàticament el coneixement previ apropiat en diversos escenaris de preparació de dades. La idea principal es basa a triar les millors primitives especialitzades d'acord amb el context del problema particular a resoldre. Abordem dos escenaris. En el primer, manegem dades personals (noms, dates, telèfons, etc.) que es presenten en formats de cadena de text molt diferents i han de ser transformats a un format unificat. El problema és com construir una transformació compositiva a partir d'un gran conjunt de primitives en el domini (per exemple, manejar mesos, anys, dies de la setmana, etc.). Desenvolupem un sistema (BK-ADAPT) que guia la cerca a través del coneixement previ extraient diverses meta-característiques dels exemples que caracteritzen el domini de la columna. En el segon escenari, ens enfrontem a la transformació de matrius de dades en llenguatges de programació genèrics com a R, utilitzant com a exemples una matriu d'entrada i algunes dades de la matriu d'eixida. També desenvolupem un sistema guiat per una cerca basada en arbres (AUTOMAT[R]IX) que usa diverses restriccions, probabilitats prèvies per a les primitives i suggeriments textuals, per aprendre eficientment les transformacions. Amb aquests sistemes, mostrem que la combinació de programació inductiva amb la selecció dinàmica de les primitives apropiades a partir del coneixement previ, és capaç de millorar els resultats d'altres enfocaments de preparació de dades d'última generació i més específics.[EN] Data science is essential for the extraction of value from data. However, the most tedious part of the process, data wrangling, implies a range of mostly manual formatting, identification and cleansing manipulations. Data wrangling still resists automation partly because the problem strongly depends on domain information, which becomes a bottleneck for state-of-the-art systems as the diversity of domains, formats and structures of the data increases. In this thesis we focus on generating algorithms that take advantage of the domain knowledge for the automation of parts of the data wrangling process. We illustrate the way in which general program induction techniques, instead of domain-specific languages, can be applied flexibly to problems where knowledge is important, through the dynamic use of domain-specific knowledge. More generally, we argue that a combination of knowledge-based and dynamic learning approaches leads to successful solutions. We propose several strategies to automatically select or construct the appropriate background knowledge for several data wrangling scenarios. The key idea is based on choosing the best specialised background primitives according to the context of the particular problem to solve. We address two scenarios. In the first one, we handle personal data (names, dates, telephone numbers, etc.) that are presented in very different string formats and have to be transformed into a unified format. The problem is how to build a compositional transformation from a large set of primitives in the domain (e.g., handling months, years, days of the week, etc.). We develop a system (BK-ADAPT) that guides the search through the background knowledge by extracting several meta-features from the examples characterising the column domain. In the second scenario, we face the transformation of data matrices in generic programming languages such as R, using an input matrix and some cells of the output matrix as examples. We also develop a system guided by a tree-based search (AUTOMAT[R]IX) that uses several constraints, prior primitive probabilities and textual hints to efficiently learn the transformations. With these systems, we show that the combination of inductive programming with the dynamic selection of the appropriate primitives from the background knowledge is able to improve the results of other state-of-the-art and more specific data wrangling approaches.This research was supported by the Spanish MECD Grant FPU15/03219;and partially by the Spanish MINECO TIN2015-69175-C4-1-R (Lobass) and RTI2018-094403-B-C32-AR (FreeTech) in Spain; and by the ERC Advanced Grant Synthesising Inductive Data Models (Synth) in Belgium.Contreras Ochando, L. (2020). Towards Data Wrangling Automation through Dynamically-Selected Background Knowledge [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/160724TESI

Schemagesteuerte Induktive Funktionale Programmsynthese durch Automatische Erkennung von Typmorphismen

Inductive functional programming systems can be characterised by two diametric approaches: Either they apply exhaustive program enumeration which uses input/output examples (IO) as test cases, or they perform an analytical, data-driven structural generalisation of the IO examples. Enumerative approaches ignore the structural information provided with the IO examples, but use type information to guide and restrict the search. They use higher-order functions which capture recursion schemes during their enumeration, but apply them randomly in a uninformed manner. Analytical approaches on the other side heavily exploit this structural information, but have ignored the benefits of a strong type system so far and use only recursion schemes either fixed and built in, or selected by an expert user. In category theory universal constructs, such as natural transformations or type morphisms, describe recursion schemes which can be defined on any inductively defined data type. They can be characterised by specific universal properties. Those type morphisms and related concepts provide a categorical approach to functional programming, which is often called categorical programming. This work shows how categorical programming can be applied to Inductive Programming and how universal constructs, such as catamorphisms, paramorphisms, and type functors, can be used as recursive program schemes for inductive functional programming. The use of program schemes for Inductive Programming is not new. The special appeal and novelty of this work is that, contrary to previous approaches, the program schemes are neither fixed, nor selected by an expert user: The applicability of those recursion schemes can be automatically detected in the given IO examples of a target function by checking the universal properties of the corresponding type morphisms. Applying this to the analytical system Igor2, both the capabilities and the expressiveness can be extended without a decrease in efficiency. An extension of the analytical functional inductive programming system Igor2 is proposed and its algorithms described. An empirical evaluation demonstrates the improvements with respect to efficiency and effectiveness that can be achieved by the use of type morphisms for Igor2 due to a reduction in search space complexity.Systeme zur induktiven Programmsynthese können bezüglich zweier gegensätzlicher Ansätze beschrieben werden: Enumerative Systeme zählen Programme vollständig auf und verwenden Eingabe/Ausgabe Beispiele (E/A) lediglich zum Testen; analytische, datengetriebene Systeme hingegegen generieren ein Programm durch strukturelle Generalisierung der E/A Beispiele. Aufzählende Ansätze ignorieren die in den E/A Beispielen enthaltene strukturelle Information völlig, benutzen aber Typinformation, um den Suchraum zu beschränken und die Suche zu steuern. Sie verwenden Funktionen höherer Ordnung als rekursive Programmschemata während der Aufzählung, wenden diese aber beliebig und nicht zielgerichtet an. Analytische Ansätze hingegen nutzen extensiv die strukturelle Information der E/A Beispiele, vernachlässigen aber die Vorzüge eines starken Typsystems. Programmschemata verwenden sie lediglich starr und fest codiert oder durch Auswahl eines Experten. In der Kategorientheorie beschreiben universelle Konstrukte wie zum Beispiel natürliche Transformationen und Typmorphismen Rekursionsschemata auf beliebigen, induktiv definierten Datentypen. Diese Konstrukte zeichnen sich durch spezifische, universelle Eigenschaften aus. Derartige Typmorphismen bieten einen kategorientheoretischen Zugang zur funktionalen Programmierung. Diese Arbeit zeigt, wie Catamorphismen, Paramorphismen und Typfunktoren als universelle Konstrukte in der induktiven Programmsynthese als rekursive Programmschemata verwendet werden können. Die Verwendung von Schemata in der induktiven Programmierung ist an sich nichts Neues, die Innovation liegt jedoch in der Art und Weise der Einführung der Schemata. Im Gegensatz zu herkömmlichen Ansätzen wird weder ein festes Schema verwendet, noch wählt ein Experte ein Schema aus. Die vorliegende Arbeit zeigt, dass die Anwendbarkeit eines bestimmten Schemas sich aus den E/A Beispielen einer konkreten Zielfunktion ableiten lässt, wenn man die universellen Eigenschaften das dem Programmschema entsprechenden Typmorphismus in den Beispielen erfüllen kann. Im Folgenden wird eine Erweiterung des funktionalen, induktiven Programmsynthesesystems Igor2 vorgestellt und der neue Algorithmus beschrieben. Ein empirischer Vergleich untermauert die Vorzüge der Erweiterung und macht die Steigerung der Effizienz und der Effektivität, die durch die Verwendung von Typmorphismen durch Komplexitätsreduktion des Suchraums erzielt werden kann, deutlich

General Program Synthesis from Examples Using Genetic Programming with Parent Selection Based on Random Lexicographic Orderings of Test Cases

Software developers routinely create tests before writing code, to ensure that their programs fulfill their requirements. Instead of having human programmers write the code to meet these tests, automatic program synthesis systems can create programs to meet specifications without human intervention, only requiring examples of desired behavior. In the long-term, we envision using genetic programming to synthesize large pieces of software. This dissertation takes steps toward this goal by investigating the ability of genetic programming to solve introductory computer science programming problems. We present a suite of 29 benchmark problems intended to test general program synthesis systems, which we systematically selected from sources of introductory computer science programming problems. This suite is suitable for experiments with any program synthesis system driven by input/output examples. Unlike existing benchmarks that concentrate on constrained problem domains such as list manipulation, symbolic regression, or boolean functions, this suite contains general programming problems that require a range of programming constructs, such as multiple data types and data structures, control flow statements, and I/O. The problems encompass a range of difficulties and requirements as necessary to thoroughly assess the capabilities of a program synthesis system. Besides describing the specifications for each problem, we make recommendations for experimental protocols and statistical methods to use with the problems. This dissertation\u27s second contribution is an investigation of behavior-based parent selection in genetic programming, concentrating on a new method called lexicase selection. Most parent selection techniques aggregate errors from test cases to compute a single scalar fitness value; lexicase selection instead treats test cases separately, never comparing error values of different test cases. This property allows it to select parents that specialize on some test cases even if they perform poorly on others. We compare lexicase selection to other parent selection techniques on our benchmark suite, showing better performance for lexicase selection. After observing that lexicase selection increases exploration of the search space while also increasing exploitation of promising programs, we conduct a range of experiments to identify which characteristics of lexicase selection influence its utility

How functional programming mattered

In 1989 when functional programming was still considered a niche topic, Hughes wrote a visionary paper arguing convincingly ‘why functional programming matters’. More than two decades have passed. Has functional programming really mattered? Our answer is a resounding ‘Yes!’. Functional programming is now at the forefront of a new generation of programming technologies, and enjoying increasing popularity and influence. In this paper, we review the impact of functional programming, focusing on how it has changed the way we may construct programs, the way we may verify programs, and fundamentally the way we may think about programs

Proceedings of the ACM SIGPLAN Workshop on Approaches and Applications of Inductive Programming (AAIP 2009)

Inductive programming is concerned with the automated construction of declarative, often functional, recursive programs from incomplete specifications such as input/output examples. The inferred program must be correct with respect to the provided examples in a generalising sense: it should be neither equivalent to them, nor inconsistent. Inductive programming algorithms are guided explicitly or implicitly by a language bias (the class of programs that can be induced) and a search bias (determining which generalised program is constructed first). Induction strategies are either generate-and-test or example-driven. In generate-and-test approaches, hypotheses about candidate programs are generated independently from the given specifications. Program candidates are tested against the given specification and one or more of the best evaluated candidates are developed further. In analytical approaches, candidate programs are constructed in an example-driven way. While generate-and-test approaches can -- in principle -- construct any kind of program, analytical approaches have a more limited scope. On the other hand, efficiency of induction is much higher in analytical approaches. Inductive programming is still mainly a topic of basic research, exploring how the intellectual ability of humans to infer generalised recursive procedures from incomplete evidence can be captured in the form of synthesis methods. Intended applications are mainly in the domain of programming assistance -- either to relieve professional programmers from routine tasks or to enable non-programmers to some limited form of end-user programming. Furthermore, in the future, inductive programming techniques might be applied to further areas such as supporting the inference of lemmata in theorem proving or learning grammar rules. Inductive automated program construction has been originally addressed by researchers in artificial intelligence and machine learning. During the last years, some work on exploiting induction techniques has been started also in the functional programming community. Therefore, the third workshop on |Approaches and Applications of Inductive Programming| took place for the first time in conjunction with the ACM SIGPLAN International Conference on Functional Programming (ICFP 2009). The first and second workshop were associated with the International Conference on Machine Learning (ICML 2005) and the European Conference on Machine Learning (ECML 2007). AAIP´09 aimed to bring together researchers from the functional programming and the artificial intelligence communities, working in the field of inductive functional programming, and advance fruitful interactions between these communities with respect to programming techniques for inductive programming algorithms, the identification of challenge problems and potential applications. For everybody interested in inductive programming we recommend to visit the website: www.inductive-programming.org

Symbolic XAI: automatic programming II

Explainable artificial intelligence (XAI) is a field blooming right now. With the popularity of opaque systems, the need of explanation methods that shed light on how this systems works has risen as well. In this work, we propose the usage of symbolic machine learning systems as explanation methods, a line that is yet to be fully explored. We will do this by reviewing this symbolic systems, analyzing the existing taxonomies of explanation methods and fitting the systems within the taxonomies. Finally, we will also do some testing on solving numerical problems with symbolic systems

Inductive programming meets the real world

© Gulwani, S. et al. | ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Communications of the ACM, http://dx.doi.org/10.1145/2736282[EN] Since most end users lack programming skills they often spend considerable time and effort performing tedious and repetitive tasks such as capitalizing a column of names manually. Inductive Programming has a long research tradition and recent developments demonstrate it can liberate users from many tasks of this kind.Gulwani, S.; Hernández-Orallo, J.; Kitzelmann, E.; Muggleton, SH.; Schmid, U.; Zorn, B. (2015). Inductive programming meets the real world. Communications of the ACM. 58(11):90-99. doi:10.1145/2736282S90995811Bengio, Y., Courville, A. and Vincent, P. Representation learning: A review and new perspectives.Pattern Analy. Machine Intell. 35, 8 (2013), 1798--1828.Bielawski, B. Using the convertfrom-string cmdlet to parse structured text.PowerShell Magazine, (Sept. 9, 2004); http://www.powershellmagazine.com/2014/09/09/using-the-convertfrom-string-cmdlet-to-parse-structured-text/Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka-Jr, E.R. and T.M. Mitchell, T.M. Toward an architecture for never-ending language learning. InAAAI, 2010.Chandola, V., Banerjee, A. and V. Kumar, V. Anomaly detection: A survey.ACM Computing Surveys 41, 3 (2009), 15.Cypher, A. (Ed).Watch What I Do: Programming by Demonstration.MIT Press, Cambridge, MA, 1993.Ferri-Ramírez, C., Hernández-Orallo, J. and Ramírez-Quintana, M.J. Incremental learning of functional logic programs. InProceedings of FLOPS, 2001, 233--247.Flener, P. and Schmid, U. An introduction to inductive programming.AI Review 29, 1 (2009), 45--62.Gulwani, S. Dimensions in program synthesis. InProceedings of PPDP, 2010.Gulwani, S. Automating string processing in spreadsheets using input-output examples. InProceedings of POPL, 2011; http://research.microsoft.com/users/sumitg/flashfill.html.Gulwani, S. Example-based learning in computer-aided STEM education.Commun. ACM 57, 8 (Aug 2014), 70--80.Gulwani, S., Harris, W. and Singh, R. Spreadsheet data manipulation using examples.Commun. ACM 55, 8 (Aug. 2012), 97--105.Henderson, R.J. and Muggleton, S.H. Automatic invention of functional abstractions.Latest Advances in Inductive Logic Programming, 2012.Hernández-Orallo, J. Deep knowledge: Inductive programming as an answer, Dagstuhl TR 13502, 2013.Hofmann, M. and Kitzelmann, E. I/O guided detection of list catamorphisms---towards problem specific use of program templates in IP. InACM SIGPLAN PEPM, 2010.Jha, J., Gulwani, S., Seshia, S. and Tiwari, A. Oracle-guided component-based program synthesis. InProceedings of the ICSE, 2010.Katayama, S. Efficient exhaustive generation of functional programs using Monte-Carlo search with iterative deepening. InProceedings of PRICAI, 2008.Kitzelmann, E. Analytical inductive functional programming.LOPSTR 2008, LNCS 5438.Springer, 2009, 87--102.Kitzelmann, E. Inductive programming: A survey of program synthesis techniques. InAAIP, Springer, 2010, 50--73.Kitzelmann, E. and Schmid, U. Inductive synthesis of functional programs: An explanation based generalization approach.J. Machine Learning Research 7, (Feb. 2006), 429--454.Kotovsky, K., Hayes, J.R. and Simon, H.A. Why are some problems hard? Evidence from Tower of Hanoi.Cognitive Psychology 17, 2 (1985), 248--294.Lau, T.A. Why programming-by-demonstration systems fail: Lessons learned for usable AI.AI Mag. 30, 4, (2009), 65--67.Lau, T.A., Wolfman, S.A., Domingos, P. and Weld, D.S. Programming by demonstration using version space algebra.Machine Learning 53, 1-2 (2003), 111--156.Le, V. and Gulwani, S. FlashExtract: A framework for data extraction by examples. InProceedings of PLDI, 2014.Lieberman, H. (Ed).Your Wish is My Command: Programming by Example.Morgan Kaufmann, 2001.Lin, D., Dechter, E., Ellis, K., Tenenbaum, J.B. and Muggleton, S.H. Bias reformulation for one-shot function induction. InProceedings of ECAI, 2014.Marcus, G.F. The Algebraic Mind.Integrating Connectionism and Cognitive Science.Bradford, Cambridge, MA, 2001.Martìnez-Plumed, C. Ferri, Hernández-Orallo, J. and M.J. Ramírez-Quintana. On the definition of a general learning system with user-defined operators.arXiv preprint arXiv:1311.4235, 2013.Menon, A., Tamuz, O., Gulwani, S., Lampson, B. and Kalai, A. A machine learning framework for programming by example. InProceedings of the ICML, 2013.Miller, R.C. and Myers, B.A. Multiple selections in smart text editing. InProceedings of IUI, 2002, 103--110.Muggleton, S.H. Inductive Logic Programming.New Generation Computing 8, 4 (1991), 295--318.Muggleton, S.H. and Lin, D. Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited.IJCAI 2013, 1551--1557.Muggleton, S.H., Lin, D., Pahlavi, N. and Tamaddoni-Nezhad, A. Meta-interpretive learning: application to grammatical inference.Machine Learning 94(2014), 25--49.Muggleton, S.H., De Raedt, L., Poole, D., Bratko, I., Flach, P. and Inoue, P. ILP turns 20: Biography and future challenges.Machine Learning 86, 1 (2011), 3--23.Olsson, R. Inductive functional programming using incremental program transformation.Artificial Intelligence 74, 1 (1995), 55--83.Perelman, D., Gulwani, S., Grossman, D. and Provost, P. Test-driven synthesis.PLDI, 2014.Raza, M., Gulwani, S. and Milic-Frayling, N. Programming by example using least general generalizations.AAAI, 2014.Schmid, U. and Kitzelmann, E. Inductive rule learning on the knowledge level.Cognitive Systems Research 12, 3 (2011), 237--248.Schmid, U. and Wysotzki, F. Induction of recursive program schemes.ECML 1398 LNAI(1998), 214--225.Shapiro, E.Y. An algorithm that infers theories from facts.IJCAI(1981), 446--451.Solar-Lezama, A.Program Synthesis by Sketching.Ph.D thesis, UC Berkeley, 2008.Summers, P.D. A methodology for LISP program construction from examples.JACM 24, 1 (1977), 162--175.Tenenbaum, J.B., Griffiths, T.L. and Kemp, C. Theory-based Bayesian models of inductive learning and reasoning.Trends in Cognitive Sciences 10, 7 (2006), 309--318.Young, S. Cognitive user interfaces.IEEE Signal Processing 27, 3 (2010), 128--140

Extrapolate: generalizing counterexamples of functional test properties

This paper presents a new tool called Extrapolate that automatically generalizes counterexamples found by property-based testing in Haskell. Example applications show that generalized counterexamples can inform the programmer more fully and more immediately what characterises failures. Extrapolate is able to produce more general results than similar tools. Although it is intrinsically unsound, as reported generalizations are based on testing, it works well for examples drawn from previous published work in this area

Inductive programming meets the real world

Since most end users lack programming skills they often spend considerable time and effort performing tedious and repetitive tasks such as capitalizing a column of names manually. Inductive Programming has a long research tradition and recent developments demonstrate it can liberate users from many tasks of this kind. Key insights • Real-world applications emerge with spreadsheet tools, scripting, and intelligent program tutors. • Learning from few examples is possible because users and systems share the same background knowledge. • Search is guided by domain-specific languages and the use of higher-order knowledge. Much of the world's population use computers for everyday tasks, but most fail to benefit from the power of computation due to their inability to program. Most crucially, users often have to perform repetitive actions manually because they are not able to use the macro languages which are available for many application programs. Recently, a first mass-market product was presented in the form of the Flash Fill feature in Microsoft Excel 2013. Flash Fill allows end users to automatically generate string processing programs for spreadsheets from one or more user-provided examples. Flash Fill is able to learn a large variety of quite complex programs from only a few examples because of incorporation of inductive programming methods. Inductive Programming (IP) is an inter-disciplinary domain of research in computer science, artificial intelligence, and cognitive science that studies the automatic synthesis of computer programs from examples and background knowledge. IP developed from research on inductive program synthesis, now called inductive functional programming (IFP), and from inductive inference techniques using logic, nowadays termed inductive logic programming (ILP). IFP addresses the synthesis of recursive functional programs generalized from regularities detected in (traces of) input/output examples ILP originated from research on induction in a logical framework Over the last decade Inductive Programming has attracted a series of international workshops. Recent surveys In the domain of end-user programming, programming by demonstration approaches were proposed which support the learning of small routines from observing the input behavior of users In this paper, several of these current applications are presented. We contrast the specific characteristics of IP with those of typical machine learning approaches and we show how IP is related to cognitive models of human inductive learning. We finally discuss recent techniques -such as use of domain-specific languages and meta-level learning-that widen the scope and power of IP and discuss new challenges. REAL-WORLD APPLICATIONS Originally, IP was applied to synthesizing functional or logic programs for general purpose tasks such as manipulating data structures (e.g., sorting or reversing a list). These investigations showed that small programs could be synthesized from a few input/output examples. The recent IT revolution has created real-world opportunities for such techniques. Most of today's large number of computer users are non-programmers and are limited to being passive consumers of the software that is made available to them. IP can empower such users to more effectively leverage computers for automating their daily repetitive tasks. We discuss below some such opportunities, especially in the areas of End-user Programming and Education