EASe : integrating search with learned episodes

Weak methods are insufficient to solve complex problems. Constrained weak methods, like hill-climbing, search too little of the problem space. Unconstrained weak methods, like breadth-first search, are intractable. Fortunately, through the integration of multiple weak methods more powerful problem solvers can be created. We demonstrate that augmenting a weak constrained search method with episodes provides a tractable method for solving a large class of problems. We demonstrate that these episodes can be generated using an unconstrained weak method while solving simple problems from a domain. We provide an analytical model of our approach and empirical results from the logic synthesis domain of VLSI design as well as the classic tile-sliding domain

Forgetting Exceptions is Harmful in Language Learning

We show that in language learning, contrary to received wisdom, keeping exceptional training instances in memory can be beneficial for generalization accuracy. We investigate this phenomenon empirically on a selection of benchmark natural language processing tasks: grapheme-to-phoneme conversion, part-of-speech tagging, prepositional-phrase attachment, and base noun phrase chunking. In a first series of experiments we combine memory-based learning with training set editing techniques, in which instances are edited based on their typicality and class prediction strength. Results show that editing exceptional instances (with low typicality or low class prediction strength) tends to harm generalization accuracy. In a second series of experiments we compare memory-based learning and decision-tree learning methods on the same selection of tasks, and find that decision-tree learning often performs worse than memory-based learning. Moreover, the decrease in performance can be linked to the degree of abstraction from exceptions (i.e., pruning or eagerness). We provide explanations for both results in terms of the properties of the natural language processing tasks and the learning algorithms.Comment: 31 pages, 7 figures, 10 tables. uses 11pt, fullname, a4wide tex styles. Pre-print version of article to appear in Machine Learning 11:1-3, Special Issue on Natural Language Learning. Figures on page 22 slightly compressed to avoid page overloa

Contributions to Time-bounded Problem Solving Using Knowledge-based Techniques

Time-bounded computations represent major challenge for knowledge-based techniques. Being primarily non-algorithmic in nature, such techniques suffer from obvious open-endedness in the sense that demands on time and other resources for a particular task cannot be predicted in advance. Consequently, efficiency of traditional knowledge-based techniques in solving time-bounded problems is not at all guaranteed. Artificial Intelligence researchers working in real-time problem solving have generally tried to avoid this difficulty by improving the speed of computation (through code optimisation or dedicated hardware) or using heuristics. However, most of these shortcuts are likely to be inappropriate or unsuitable in complicated real-time applications. Consequently, there is a need of more systematic and/or general measures. We propose a two-fold improvement over traditional knowledge-based techniques for tackling this problem. Firstly, that a cache-based architecture should be used in choosing the best alternative approach (when there are two or more) compatible to the time constraints. This cache differs from traditional caches, used in other branches of computer science, in the sense that it can hold not just "ready to use" values but also knowledge suggesting which AI technique will be most suitable to meet a temporal demand in a given context. The second improvement is in processing the cached knowledge itself. We propose a technique which can be called "knowledge interpolation" and which can be applied to different forms of knowledge (such as symbolic values, rules, cases) when the keys used for cache access do not make exact matches with the labels for any cell of the cache. The research reported in this thesis comprises development of cache-based architecture and interpolation techniques, studies of their requisites and representational issues and their complementary roles in achieving time-bounded performance. Ground operations control of an airport and allocating resources for short-wave radio communications are two domains in which our proposed methods are studied

A study of instance-based algorithms for supervised learning tasks : mathematical, empirical, and psychological evaluations

This dissertation introduces a framework for specifying instance-based algorithms that can solve supervised learning tasks. These algorithms input a sequence of instances and yield a partial concept description, which is represented by a set of stored instances and associated information. This description can be used to predict values for subsequently presented instances. The thesis of this framework is that extensional concept descriptions and lazy generalization strategies can support efficient supervised learning behavior.The instance-based learning framework consists of three components. The pre-processor component transforms an instance into a more palatable form for the performance component, which computes the instance's similarity with a set of stored instances and yields a prediction for its target value(s). Therefore, the similarity and prediction functions impose generalizations on the stored instances to inductively derive predictions. The learning component assesses the accuracy of these prediction(s) and updates partial concept descriptions to improve their predictive accuracy.This framework is evaluated in four ways. First, its generality is evaluated by mathematically determining the classes of symbolic concepts and numeric functions that can be closely approximated by IB_1, a simple algorithm specified by this framework. Second, this framework is empirically evaluated for its ability to specify algorithms that improve IB_1's learning efficiency. Significant efficiency improvements are obtained by instance-based algorithms that reduce storage requirements, tolerate noisy data, and learn domain-specific similarity functions respectively. Alternative component definitions for these algorithms are empirically analyzed in a set of five high-level parameter studies. Third, this framework is evaluated for its ability to specify psychologically plausible process models for categorization tasks. Results from subject experiments indicate a positive correlation between a models' ability to utilize attribute correlation information and its ability to explain psychological phenomena. Finally, this framework is evaluated for its ability to explain and relate a dozen prominent instance-based learning systems. The survey shows that this framework requires only slight modifications to fit these highly diverse systems. Relationships with edited nearest neighbor algorithms, case-based reasoners, and artificial neural networks are also described