Learning programs by learning from failures

We describe an inductive logic programming (ILP) approach called learning from failures. In this approach, an ILP system (the learner) decomposes the learning problem into three separate stages: generate, test, and constrain. In the generate stage, the learner generates a hypothesis (a logic program) that satisfies a set of hypothesis constraints (constraints on the syntactic form of hypotheses). In the test stage, the learner tests the hypothesis against training examples. A hypothesis fails when it does not entail all the positive examples or entails a negative example. If a hypothesis fails, then, in the constrain stage, the learner learns constraints from the failed hypothesis to prune the hypothesis space, i.e. to constrain subsequent hypothesis generation. For instance, if a hypothesis is too general (entails a negative example), the constraints prune generalisations of the hypothesis. If a hypothesis is too specific (does not entail all the positive examples), the constraints prune specialisations of the hypothesis. This loop repeats until either (i) the learner finds a hypothesis that entails all the positive and none of the negative examples, or (ii) there are no more hypotheses to test. We introduce Popper, an ILP system that implements this approach by combining answer set programming and Prolog. Popper supports infinite problem domains, reasoning about lists and numbers, learning textually minimal programs, and learning recursive programs. Our experimental results on three domains (toy game problems, robot strategies, and list transformations) show that (i) constraints drastically improve learning performance, and (ii) Popper can outperform existing ILP systems, both in terms of predictive accuracies and learning times.Comment: Accepted for the machine learning journa

Introducing Inventiveness into the Patent System: Submission to the Review of the National Innovation System

Because of the potential impact of the patent system on innovation diffusion, particularly on continuous and/or incremental innovation, patent policy should be of central importance to the review of the national innovation system. Substantial empirical evidence shows that most industrial innovations are not induced by the patent system. Even in very large markets, such as the USA, only a minority of patents are likely to be induced by the patent system. To the extent that patents do induce innovations, it is the inventiveness of the innovation which gives rise to possible social benefits (externalities, mainly in the form of knowledge spillovers) which may offset the costs of a patent system and thus give rise to a net economic benefit. On the basis of this evidence about the inducement effect of the patent system, and evidence on the current very low inventiveness standard for patent grant, policy proposals are put forward to re-introduce inventiveness into the patent system, thus making it potentially welfare-enhancing. These proposed changes would also have a major impact in ameliorating the negative impact of the patent system on continuous/incremental innovation