We present Louise, a new Meta-Interpretive Learner that performs efficient multi-clause learning, implemented in Prolog. Louise is efficient enough to learn programs that are too large to be learned with the current state-of-the-art MIL system, Metagol. Louise learns by first constructing the most general program in the hypothesis space of a MIL problem and then reducing this "Top program" by Plotkin's program reduction algorithm. In this extended abstract we describe Louise's learning approach and experimentally demonstrate that Louise can learn programs that are too large to be learned by our implementation of Metagol, Thelma

Muggleton, Stephen H.

Patsantzis, Stassa

English

White Rose Research Online

This is a repository copy of Louise: A Meta-Interpretive Learner for Efficient Multi-clause Learning of Large Programs.White Rose Research Online URL for this paper:http://eprints.whiterose.ac.uk/157823/Conference or Workshop Item:Patsantzis, Stassa and Muggleton, Stephen H. (2019) Louise: A Meta-Interpretive Learner for Efficient Multi-clause Learning of Large Programs. In: The 29th ILP conference, 03-05 Sep 2019, Plovdiv, Bulgaria. eprints@whiterose.ac.ukhttps://eprints.whiterose.ac.uk/Reuse Items deposited in White Rose Research Online are protected by copyright, with all rights reserved unless indicated otherwise. They may be downloaded and/or printed for private study, or other acts as permitted by national copyright laws. The publisher or other rights holders may allow further reproduction and re-use of the full text version. This is indicated by the licence information on the White Rose Research Online record for the item. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. The 29th ILP conference, 3–5 Sep 2019, Plovdiv, Bulgaria.Louise: A Meta-Interpretive Learner for EfficientMulti-clause Learning of Large ProgramsStassa Patsantzis and Stephen H. Muggletone.patsantzis17@imperial.ac.uk, s.muggleton@imperial.ac.uk.Imperial College LondonAbstract. We present Louise, a new Meta-Interpretive Learner thatperforms efficient multi-clause learning, implemented in Prolog. Louiseis efficient enough to learn programs that are too large to be learnedwith the current state-of-the-art MIL system, Metagol. Louise learns byfirst constructing the most general program in the hypothesis space of aMIL problem and then reducing this “Top program” by Plotkin’s pro-gram reduction algorithm. In this extended abstract we describe Louise’slearning approach and experimentally demonstrate that Louise can learnprograms that are too large to be learned by our implementation ofMetagol, Thelma.1 IntroductionIn Meta-Interpretive Learning (MIL) [3, 2] a learner searches a space of hypothe-ses consisting of sets of first order definite datalog clauses. This search is expen-sive and the state-of-the-art MIL learner, Metagol [1] can not effectively learntheories larger than a handful of clauses in size1. Our new MIL-learner, Louise2,implemented in Prolog, avoids a classical search of the space of hypotheses andinstead learns by constructing the most general program that explains the train-ing examples given the background knowledge, and then logically reducing this“Top program” to remove redundant clauses. The result is an efficient learnerthat can learn hypotheses that are too large to be learned by Metagol.blood relative/2 relative/2Time Clauses Time ClausesThelma 76.509sec. 11 2hrs (Timed out) 5 (Timed out)Louise 0.094sec. 11 46.169sec. 29Table 1: Experiment results1 In practice this means 5 to a dozen clauses, depending on the problem.2 https://github.com/stassa/louise12 The 29th ILP conference, 3–5 Sep 2019, Plovdiv, Bulgaria.Top program constructionGeneralisation of positive examplesm(chain, grandfather, father, father).m(chain, grandfather, father,mother).m(chain, grandfather, father, parent).m(chain, grandfather, parent, father).m(chain, grandfather, parent,mother).m(chain, grandfather, parent, parent).Specialisation by negative examplesm(chain, grandfather, father, father).m(chain, grandfather, father,mother).m(chain, grandfather, father, parent).Hypothesis constructionTop proram unfoldingm(grandfather,A,B) : −m(father,A,C),m(father, C,B).m(grandfather,A,B) : −m(father,A,C),m(mother, C,B).m(grandfather,A,B) : −m(father,A,C),m(parent, C,B).Top proram reductionm(grandfather,A,B) : −m(father,A,C),m(parent, C,B)Reduction excapsulationgrandfather(A,B) : −father(A,C), parent(C,B).Table 2: Top program construction and reduction for “grandfather”.2 FrameworkAlgorithm 1 Learning by Top program construction and reductionGiven: A MIL Problem, T = {E,B,M,H}.Return: A hypothesis, H ∈ H.1: Encapsulate {E,B,M} and expand M.2: Construct the Top prorgam for T , S.3: Reduce S to S ′.4: Excapsulate S ′ as H.5: Return HLouise’s learning procedure is described in Algorithm 1. Construction of theTop program is defined in Algorithm 2. Reduction of the Top program is achievedby application of Gordon Plotkin’s program reduction algorithm, defined inPlotkin’s doctoral thesis, [4], as Theorem 3.3.1.2. Louise’s learning procedureis illustrated in table 2 with an example for the “grandfather” relation.A MIL problem T = {E,B,M,H} consists of ground definite clause ex-amples, E = {E+, E−}, definite clause definitions of background knowledgepredicates, B, second order definite clause metarules,M, and the correspondingA late breaking abstract published under a CC BY licence. 3Algorithm 2 Top program constructionGiven: An encapsulated MIL problem, T = {E+, E−, B,M,H}.Return: A set S of metasubstitutions µ/M in all solutions of T .1: Generalise(E+, B,M) → S+2: Specialise(E−, B,M,S+) → S−3: Return S = S−4: procedure Generalise(E+, B,M)5: Initialise: Let S+ = ∅6: for e+ ∈ E+ do7: for M ∈ M do8: Set S+ = S+ ∪ {µ/M : µM ∪B ∪M |= e+}9: end for10: end for11: Return S+12: end procedure13: procedure Specialise(E−, B,M,S+)14: Initialise: Let S− = S+15: for µ/M ∈ S− do16: if ∃e− ∈ E−: µM ∪B ∪M |= e− then17: Set S− = S− \ {µ/M}18: end if19: end for20: Return S−21: end proceduresearch space of definite datalog hypotheses, H. Louise begins learning by encap-sulating a MIL problem. Briefly, to encapsulate a literal means to replace it bya new literal of an encapsulation predicate (m, in Louise) whose arguments arethe predicate symbol and arguments of the original literal. Encapsulation allowsa compact representation of the Top program as a set of first order atoms repre-senting metasubstitutions of the existentially quantified variables in a metarule.In algorithm 2 a metasubstitution µ of the existentially quantified variables ofa metarule M ∈ M is represented as µ/M . The set of metasubstitutions inthe Top program is unfolded to produce a set of definite clause instances of themetarules inM before program reduction. The reduced Top program is then ex-acpsulated to form the learned hypothesis. Excapsulation is the opposite processof encapsulation.3 ExperimentsWe compare the performance of Louise to Thelma3, our Prolog implementationof Metagol, on two kinship problems. The target theory for blood relative/23 https://github.com/stassa/thelma4 The 29th ILP conference, 3–5 Sep 2019, Plovdiv, Bulgaria.consists of 16 clauses of the Identity metarule, P (X,Y )← Q(X,Y ). Each clausemaps one of 16 definitions of kinship relations to blood relative/2. The targettheory for relative/2 cannot be expressed in the background knowledge andmetarules we allow for the experiment, forcing the two systems to learn a largerhypothesis. We did not know what this larger hypothesis would be in advance ofour experiments. We set a time limit of 2 hours for both experiments. Results ofthe experiments are listed in table 1. Both learners compress the target theoryfor blood relative/2 to 11 clauses. However, Thelma takes a significantly longertime to learn the shorter theory than Louise and times out on the longer theory.Louise learns both theories in under a minute.4 Conclusions and future workUnlike Metagol and Thelma, Louise is not yet capable of predicate inventionbecause of the left-recursive nature of its representation of metarules. However,Louise can learn some programs for which predicate invention is normally re-quired by performing episodic learning and unfolding metarules onto each otherto produce extended metarules.In future work we intend to implement predicate invention by use of a TP op-erator for bottom-up evaluation, to avoid issues with left-recursion in metarules.We also intend to test Louise on larger problems, in particular, machine visionproblems.References1. Andrew Cropper and Stephen H. Muggleton. Metagol system, 2016.2. Stephen Muggleton and Dianhuan Lin. Meta-Interpretive Learning of Higher-OrderDyadic Datalog : Predicate Invention Revisited. Machine Learning, 100(1):49–73,2015.3. Stephen H. Muggleton, Dianhuan Lin, Niels Pahlavi, and Alireza Tamaddoni-Nezhad. Meta-interpretive learning: Application to grammatical inference. MachineLearning, 94(1):25–49, 2014.4. Gordon Plotkin. Automatic Methods of Inductive Inference. PhD thesis, The Uni-versity of Edinburgh, 1972.

Louise: A Meta-Interpretive Learner for Efficient Multi-clause Learning of Large Programs

https://eprints.whiterose.ac.uk/157823/1/Louise__A_Meta_Interpretive_Learner_for_efficient_multi_clause_learning_of_large_programs1.pdf

Louise: A Meta-Interpretive Learner for Efficient Multi-clause Learning of Large Programs

Abstract

Similar works

Full text

Available Versions

White Rose Research Online