MILCS: A mutual information learning classifier system

This paper introduces a new variety of learning classifier system (LCS), called MILCS, which utilizes mutual information as fitness feedback. Unlike most LCSs, MILCS is specifically designed for supervised learning. MILCS's design draws on an analogy to the structural learning approach of cascade correlation networks. We present preliminary results, and contrast them to results from XCS. We discuss the explanatory power of the resulting rule sets, and introduce a new technique for visualizing explanatory power. Final comments include future directions for this research, including investigations in neural networks and other systems. Copyright 2007 ACM

Learning programs by learning from failures

We describe an inductive logic programming (ILP) approach called learning from failures. In this approach, an ILP system (the learner) decomposes the learning problem into three separate stages: generate, test, and constrain. In the generate stage, the learner generates a hypothesis (a logic program) that satisfies a set of hypothesis constraints (constraints on the syntactic form of hypotheses). In the test stage, the learner tests the hypothesis against training examples. A hypothesis fails when it does not entail all the positive examples or entails a negative example. If a hypothesis fails, then, in the constrain stage, the learner learns constraints from the failed hypothesis to prune the hypothesis space, i.e. to constrain subsequent hypothesis generation. For instance, if a hypothesis is too general (entails a negative example), the constraints prune generalisations of the hypothesis. If a hypothesis is too specific (does not entail all the positive examples), the constraints prune specialisations of the hypothesis. This loop repeats until either (i) the learner finds a hypothesis that entails all the positive and none of the negative examples, or (ii) there are no more hypotheses to test. We introduce Popper, an ILP system that implements this approach by combining answer set programming and Prolog. Popper supports infinite problem domains, reasoning about lists and numbers, learning textually minimal programs, and learning recursive programs. Our experimental results on three domains (toy game problems, robot strategies, and list transformations) show that (i) constraints drastically improve learning performance, and (ii) Popper can outperform existing ILP systems, both in terms of predictive accuracies and learning times.Comment: Accepted for the machine learning journa

Logical Reduction of Metarules

International audienceMany forms of inductive logic programming (ILP) use metarules, second-order Horn clauses, to define the structure of learnable programs and thus the hypothesis space. Deciding which metarules to use for a given learning task is a major open problem and is a trade-off between efficiency and expressivity: the hypothesis space grows given more metarules, so we wish to use fewer metarules, but if we use too few metarules then we lose expressivity. In this paper, we study whether fragments of metarules can be logically reduced to minimal finite subsets. We consider two traditional forms of logical reduction: subsumption and entailment. We also consider a new reduction technique called derivation reduction, which is based on SLD-resolution. We compute reduced sets of metarules for fragments relevant to ILP and theoretically show whether these reduced sets are reductions for more general infinite fragments. We experimentally compare learning with reduced sets of metarules on three domains: Michalski trains, string transformations, and game rules. In general, derivation reduced sets of metarules outperform subsumption and entailment reduced sets, both in terms of predictive accuracies and learning times

Analytical learning and term-rewriting systems

Analytical learning is a set of machine learning techniques for revising the representation of a theory based on a small set of examples of that theory. When the representation of the theory is correct and complete but perhaps inefficient, an important objective of such analysis is to improve the computational efficiency of the representation. Several algorithms with this purpose have been suggested, most of which are closely tied to a first order logical language and are variants of goal regression, such as the familiar explanation based generalization (EBG) procedure. But because predicate calculus is a poor representation for some domains, these learning algorithms are extended to apply to other computational models. It is shown that the goal regression technique applies to a large family of programming languages, all based on a kind of term rewriting system. Included in this family are three language families of importance to artificial intelligence: logic programming, such as Prolog; lambda calculus, such as LISP; and combinatorial based languages, such as FP. A new analytical learning algorithm, AL-2, is exhibited that learns from success but is otherwise quite different from EBG. These results suggest that term rewriting systems are a good framework for analytical learning research in general, and that further research should be directed toward developing new techniques

In Search of a Consistent World View : Induction as Extension

In this thesis, I develop an account of classificatory induction that gives, for any observation report, a theory that contains all inductive consequences of the observation report. Such a theory is called the /maximal plausible generalisation/ of the observation report, and it is shown to be consistent, unique for each observation report, and to capture nicely the intuitive notion of inductive consequence by being the most informative generalisation that is still plausible. In the course of defining the maximal plausible generalisation, I present the conditions of entailment, consistency and plausibility, which any relation of inductive consequence should observe. These conditions also hold for the maximal plausible generalisation.Tässä tutkielmassa annan kuvauksen luokittelevasta induktiosta, joka määrittelee mille tahansa havaintoraportille kaikki sen induktiiviset seuraukset. Tätä tulosta kutsun havaintoraportin /laajimmaksi uskottavaksi yleistykseksi/. Todistan laajimman uskottavan yleistyksen olevan ristiriidaton ja yksiselitteinen jokaiselle havaintoraportille sekä kuvaavan hyvin induktiivisen seurauksen intuitiivista käsitettä, koska se on informatiivisin yleistys joka on vielä uskottava. Laajinta uskottavaa yleistystä määriteltäessä esittelen myös seuraamuksen, ristiriidattomuuden ja uskottavuuden ehdot, joiden tulee päteä mille tahansa induktiiviselle seurausrelaatiolle. Todistan laajimman uskottavan yleistyksen täyttävän nämä ehdot. Laajin uskottava yleistys on määritelty lauseiden suositummuusrelaation kautta, ja tälle suositummuusrelaatiolle määrittelen myös ehtoja, jotka vastaavat sitä, miten lauseita suositaan induktiivisessa päättelyssä. Todistan, että yksi näistä ehdoista (symmetria negaation suhteen) aiheuttaa, ettei mikään uskottava yleistys sisällä havaintoraporttiin liittymättömiä lauseita. Lopuksi esittelen konkreettisen esimerkin suositummuusrelaatiosta, joka täyttää asettamani ehdot. Tämä antaa konkreettisen määritelmän lausejoukon induktiiviselle seuraukselle

Protocol-based verification of message-passing parallel programs

© 2015 ACM.We present ParTypes, a type-based methodology for the verification of Message Passing Interface (MPI) programs written in the C programming language. The aim is to statically verify programs against protocol specifications, enforcing properties such as fidelity and absence of deadlocks. We develop a protocol language based on a dependent type system for message-passing parallel programs, which includes various communication operators, such as point-to-point messages, broadcast, reduce, array scatter and gather. For the verification of a program against a given protocol, the protocol is first translated into a representation read by VCC, a software verifier for C. We successfully verified several MPI programs in a running time that is independent of the number of processes or other input parameters. This contrasts with alternative techniques, notably model checking and runtime verification, that suffer from the state-explosion problem or that otherwise depend on parameters to the program itself. We experimentally evaluated our approach against state-of-the-art tools for MPI to conclude that our approach offers a scalable solution