Numerical reasoning with an ILP system capable of lazy evaluation and customised search

Using problem-speci®c background knowledge, computer programs developed within theframework of Inductive Logic Programming (ILP) have been used to construct restricted®rst-order logic solutions to scienti®c problems. However, their approach to the analysis ofdata with substantial numerical content has been largely limited to constructing clauses that:(a) provide qualitative descriptions (high'', low'' etc.) of the values of response variables;and (b) contain simple inequalities restricting the ranges of predictor variables. This has precludedthe application of such techniques to scienti®c and engineering problems requiring amore sophisticated approach. A number of specialised methods have been suggested to remedythis. In contrast, we have chosen to take advantage of the fact that the existing theoreticalframework for ILP places very few restrictions of the nature of the background knowledge.We describe two issues of implementation that make it possible to use background predicatesthat implement well-established statistical and numerical analysis procedures. Any improvementsin analytical sophistication that result are evaluated empirically using arti®cial andreal-life data. Experiments utilising arti®cial data are concerned with extracting constraintsfor response variables in the text-book problem of balancing a pole on a cart. They illustratethe use of clausal de®nitions of arithmetic and trigonometric functions, inequalities, multiplelinear regression, and numerical derivatives. A non-trivial problem concerning the predictionof mutagenic activity of nitroaromatic molecules is also examined. In this case, expert chemistshave been unable to devise a model for explaining the data. The result demonstrates the combineduse by an ILP program of logical and numerical capabilities to achieve an analysis thatincludes linear modelling, clustering and classi®cation. In all experiments, the predictions obtainedcompare favourably against benchmarks set by more traditional methods of quantitativemethods, namely, regression and neural-network

Omnivariate rule induction using a novel pairwise statistical test

Rule learning algorithms, for example, RIPPER, induces univariate rules, that is, a propositional condition in a rule uses only one feature. In this paper, we propose an omnivariate induction of rules where under each condition, both a univariate and a multivariate condition are trained, and the best is chosen according to a novel statistical test. This paper has three main contributions: First, we propose a novel statistical test, the combined 5 x 2 cv t test, to compare two classifiers, which is a variant of the 5 x 2 cv t test and give the connections to other tests as 5 x 2 cv F test and k-fold paired t test. Second, we propose a multivariate version of RIPPER, where support vector machine with linear kernel is used to find multivariate linear conditions. Third, we propose an omnivariate version of RIPPER, where the model selection is done via the combined 5 x 2 cv t test. Our results indicate that 1) the combined 5 x 2 cv t test has higher power (lower type II error), lower type I error, and higher replicability compared to the 5 x 2 cv t test, 2) omnivariate rules are better in that they choose whichever condition is more accurate, selecting the right model automatically and separately for each condition in a rule.Publisher's VersionAuthor Post Prin

A workbench to develop ILP systems

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

Object-oriented data mining

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Automated Construction of Relational Attributes ACORA: A Progress Report

Data mining research has not only development a large number of algorithms, but also enhanced our knowledge and understanding of their applicability and performance. However, the application of data mining technology in business environments is still no very common, despite the fact that organizations have access to large amounts of data and make decisions that could profit from data mining on a daily basis. One of the reasons is the mismatch between data representation for data storage and data analysis. Data are most commonly stored in multi-table relational databases whereas data mining methods require that the data be represented as a simple feature vector. This work presents a general framework for feature construction from multiple relational tables for data mining applications. The second part describes our prototype implementation ACORA (Automated Construction of Relational Features).Information Systems Working Papers Serie

Structural Logistic Regression for Link Analysis

We present Structural Logistic Regression, an extension of logistic regression to modeling relational data. It is an integrated approach to building regression models from data stored in relational databases in which potential predictors, both boolean and real-valued, are generated by structured search in the space of queries to the database, and then tested with statistical information criteria for inclusion in a logistic regression. Using statistics and relational representation allows modeling in noisy domains with complex structure. Link prediction is a task of high interest with exactly such characteristics. Be it in the domain of scientific citations, social networks or hypertext, the underlying data are extremely noisy and the features useful for prediction are not readily available in a flat file format. We propose the application of Structural Logistic Regression to building link prediction models, and present experimental results for the task of predicting citations made in scientific literature using relational data taken from the CiteSeer search engine. This data includes the citation graph, authorship and publication venues of papers, as well as their word content

A sequence-length sensitive approach to learning biological grammars using inductive logic programming.

This thesis aims to investigate if the ideas behind compression principles, such as the Minimum Description Length, can help us to improve the process of learning biological grammars from protein sequences using Inductive Logic Programming (ILP). Contrary to most traditional ILP learning problems, biological sequences often have a high variation in their length. This variation in length is an important feature of biological sequences which should not be ignored by ILP systems. However we have identified that some ILP systems do not take into account the length of examples when evaluating their proposed hypotheses. During the learning process, many ILP systems use clause evaluation functions to assign a score to induced hypotheses, estimating their quality and effectively influencing the search. Traditionally, clause evaluation functions do not take into account the length of the examples which are covered by the clause. We propose L-modification, a way of modifying existing clause evaluation functions so that they take into account the length of the examples which they learn from. An empirical study was undertaken to investigate if significant improvements can be achieved by applying L-modification to a standard clause evaluation function. Furthermore, we generally investigated how ILP systems cope with the length of examples in training data. We show that our L-modified clause evaluation function outperforms our benchmark function in every experiment we conducted and thus we prove that L-modification is a useful concept. We also show that the length of the examples in the training data used by ILP systems does have an undeniable impact on the results

Specific-to-General Learning for Temporal Events with Application to Learning Event Definitions from Video

We develop, analyze, and evaluate a novel, supervised, specific-to-general learner for a simple temporal logic and use the resulting algorithm to learn visual event definitions from video sequences. First, we introduce a simple, propositional, temporal, event-description language called AMA that is sufficiently expressive to represent many events yet sufficiently restrictive to support learning. We then give algorithms, along with lower and upper complexity bounds, for the subsumption and generalization problems for AMA formulas. We present a positive-examples--only specific-to-general learning method based on these algorithms. We also present a polynomial-time--computable ``syntactic'' subsumption test that implies semantic subsumption without being equivalent to it. A generalization algorithm based on syntactic subsumption can be used in place of semantic generalization to improve the asymptotic complexity of the resulting learning algorithm. Finally, we apply this algorithm to the task of learning relational event definitions from video and show that it yields definitions that are competitive with hand-coded ones