Learning functional descriptions from examples

The task of inductive learning from examples places constraints on the representation of training instances and concepts. These constraints are different from, and often incompatible with, the constraints placed on the representation by the performance task. This incompatibility is severe when learning functional concepts and explains why previous researchers have found it so difficult to construct good representations for inductive learningthey were trying to achieve a compromise between these two sets of constraints. This thesis addresses this problem, and takes a different approach. Rather than designing a compromise representation we employ two different representations: one for learning and one for performance. The system developed learns concepts in chess and checkers. Training instances are presented in the "performance representation" as simple board positions, then converted to the "learning representation" via a search process that builds an explanation of the outcome of the position. Inductive generalization is performed over these explanations to form descriptions of the concepts in terms of the moves and goals involved. Finally the concepts are translated back into the "performance representation" to support efficient recognition of future instances. The advantages of this "two representation" approach are (a) many fewer training instances are required to learn the concept, (b) the biases of the learning program are very simple, and (c) the learning system requires virtually no "vocabulary engineering" to learn concepts in a new domain

Regular mosaic pattern development: A study of the interplay between lateral inhibition, apoptosis and differential adhesion

<p>Abstract</p> <p>Background</p> <p>A significant body of literature is devoted to modeling developmental mechanisms that create patterns within groups of initially equivalent embryonic cells. Although it is clear that these mechanisms do not function in isolation, the timing of and interactions between these mechanisms during embryogenesis is not well known. In this work, a computational approach was taken to understand how lateral inhibition, differential adhesion and programmed cell death can interact to create a mosaic pattern of biologically realistic primary and secondary cells, such as that formed by sensory (primary) and supporting (secondary) cells of the developing chick inner ear epithelium.</p> <p>Results</p> <p>Four different models that interlaced cellular patterning mechanisms in a variety of ways were examined and their output compared to the mosaic of sensory and supporting cells that develops in the chick inner ear sensory epithelium. The results show that: 1) no single patterning mechanism can create a 2-dimensional mosaic pattern of the regularity seen in the chick inner ear; 2) cell death was essential to generate the most regular mosaics, even through extensive cell death has not been reported for the developing basilar papilla; 3) a model that includes an iterative loop of lateral inhibition, programmed cell death and cell rearrangements driven by differential adhesion created mosaics of primary and secondary cells that are more regular than the basilar papilla; 4) this same model was much more robust to changes in homo- and heterotypic cell-cell adhesive differences than models that considered either fewer patterning mechanisms or single rather than iterative use of each mechanism.</p> <p>Conclusion</p> <p>Patterning the embryo requires collaboration between multiple mechanisms that operate iteratively. Interlacing these mechanisms into feedback loops not only refines the output patterns, but also increases the robustness of patterning to varying initial cell states.</p

Correct abstraction in counter-planning : a knowledge compilation approach

Knowledge compilation improves search-intensive problem-solvers that are easily specified but inefficient. One promising approach improves efficiency by constructing a database of problem-instance/best-action pairs that replace problem-solving search with efficient lookup. The database is constructed by reverse enumeration- expanding the complete search space backwards, from the terminal problem instances. This approach has been used successfully in counter-planning to construct perfect problem-solvers for sub domains of chess and checkers. However, the approach is limited to small problems because both the space needed to store the database and the time needed to generate the database grow exponentially with problem size. This thesis addresses these problems through two mechanisms. First, the space needed is reduced through an abstraction mechanism that is especially suited to counter-planning domains. The search space is abstracted by representing problem states as equivalence classes with respect to the goal achieved and the operators as equivalence classes with respect to how they influence the goals. Second, the time needed is reduced through a hueristic best-first control of the reverse enumeration. Since with larger problems it may be impractical to run the compiler to completion, the search is organized to optimize the tradeoff between the time spent compiling a domain and the coverage achieved over that domain. These two mechanisms are implemented in a system that has been applied to problems in chess and checkers. Empirical results demonstrate both the strengths and weaknesses of the approach. In most problems and 80/20 rule was demonstrated, where a small number of patterns were identified early that covered most of the domain, justifying the use of best-first search. In addition, the method was able to automatically generate a set of abstract rules that had previously required two person-months to hand engineer

Improving problem solving performance by example guided reformulation of knowledge

This paper introduces a method that improves the performance of a problem solver by reformulating its domain theory into one in which functionally relevant features are explicit in the syntax. This method, in contrast to previous reformulation methods, employs sets of training examples to constrain and direct the reformulation process. The use of examples offers two advantages over purely deductive approaches: First, the examples identify the exact part of the domain theory to be reformulated. Second, a proof with examples is much simpler to construct than a general proof because it is fully instantiated. The method exploits the fact that what is relevant to a goal is syntactically explicit in successful solutions to that goal. The method first takes as input a set of training examples that "exercise" an important part of the domain theory and then applies the problem solver to explain the examples in terms of a relevant goal. Next, the set of explanations is "clustered" into cases and then generalized using the induction over explanations method, forming a set of general explanations. Finally, these general explanations are reformulated into new domain theory rules. We illustrate the method in the domain of chess. We reformulate a simple declarative encoding of legal-move to produce a new domain theory that can generate the legal moves in a tenth of the time required by the original theory. We also show how the reformulated theory can more efficiently describe the important knight-fork feature

A computational study of VEGF production by patterned retinal epithelial cell colonies as a model for neovascular macular degeneration

Background: The configuration of necrotic areas within the retinal pigmented epithelium is an important element in the progression of age-related macular degeneration (AMD). In the exudative (wet) and non-exudative (dry) forms of the disease, retinal pigment epithelial (RPE) cells respond to adjacent atrophied regions by secreting vascular endothelial growth factor (VEGF) that in turn recruits new blood vessels which lead to a further reduction in retinal function and vision. In vitro models exist for studying VEGF expression in wet AMD (Vargis et al., Biomaterials 35(13):3999–4004, 2014), but are limited in the patterns of necrotic and intact RPE epithelium they can produce and in their ability to finely resolve VEGF expression dynamics. Results: In this work, an in silico hybrid agent-based model was developed and validated using the results of this cell culture model of VEGF expression in AMD. The computational model was used to extend the cell culture investigation to explore the dynamics of VEGF expression in different sized patches of RPE cells and the role of negative feedback in VEGF expression. Results of the simulation and the cell culture studies were in excellent qualitative agreement, and close quantitative agreement. Conclusions: The model indicated that the configuration of necrotic and RPE cell-containing regions have a major impact on VEGF expression dynamics and made precise predictions of VEGF expression dynamics by groups of RPE cells of various sizes and configurations. Coupled with biological studies, this model may give insights into key molecular mechanisms of AMD progression and open routes to more effective treatments

Exploiting Self-Organization in Bioengineered Systems: A Computational Approach

The productivity of bioengineered cell factories is limited by inefficiencies in nutrient delivery and waste and product removal. Current solution approaches explore changes in the physical configurations of the bioreactors. This work investigates the possibilities of exploiting self-organizing vascular networks to support producer cells within the factory. A computational model simulates de novo vascular development of endothelial-like cells and the resultant network functioning to deliver nutrients and extract product and waste from the cell culture. Microbial factories with vascular networks are evaluated for their scalability, robustness, and productivity compared to the cell factories without a vascular network. Initial studies demonstrate that at least an order of magnitude increase in production is possible, the system can be scaled up, and the self-organization of an efficient vascular network is robust. The work suggests that bioengineered multicellularity may offer efficiency improvements difficult to achieve with physical engineering approaches

A study of explanation-based methods for inductive learning

This paper formalizes a new learning from examples problem: identifying a correct concept definition from positive examples such that the concept is some specialization of a target concept defined by a domain theory. This paper describes an empirical study that evaluates three methods for solving this problem: explanation based generalization (EBG), multiple example explanation based generalization (mEBG), and a new method, induction over explanations (IOE). The study demonstrates that the two existing methods (EBG and mEBG) exhibit two shortcomings: (a) the methods rarely identify the correct definition, and (b) the methods are brittle-their success depends greatly on the choice of encoding of the domain theory rules. The study demonstrates that the new method, IOE, does not exhibit these shortcomings. The IOE method applies the domain theory to construct explanations from multiple training examples as in mEBG, but forms the concept definition by employing a similarity-based generalization policy over the explanations. The method has the advantage that an explicit domain theory can be exploited to aid the learning process, the dependence on the initial encoding of the domain theory is significantly reduced, and the correct concepts can be learned from few examples. The study evaluates the methods in an implemented system, called Wyl2, learning a variety of concepts in chess including "skewer" and "knight-fork."Key words: Learning from examples, induction over explanations, explanation based learning, inductive learning, knowledge compilation, evaluation of learning method

Induction over explanations : a method that exploits domain knowledge to learn from examples

We introduce five criteria by which to judge the suitability of a method for solving the problem of learning concepts from examples: correctness (the correct concept should be identified), performance efficiency (the learned definition should be efficient to apply to the performance task), flexibility (the method should be able to learn a variety of different concepts), ease of engineering (the method should be easy to implement in new domains) and learning efficiency (the method should learn from few examples efficiently). We analyze two existing methods for learning from examples, similarity-based learning (SBL) and explanation-based learning (EBL), and find them inappropriate for solving an important sub-problem: learning functional concepts from examples. In SBL, the performance efficiency goal is incompatible with the other goals, because the representation best for performance is ineffective for learning. In EBL, it is difficult to satisfy the flexibility or correctness goals, because the concepts are identified from a single example and an inflexible generalization policy. "We introduce a new method, called induction over explanations (IOE), that overcomes these difficulties. The method applies a domain theory to construct explanations from the training examples as in EBL, but forms the concept definition by employing an SBL generalization policy over the explanations. The concept definition is then compiled into a form efficient for the performance task. The method has the advantage that an explicit domain theory can be exploited to aid the learning process, the vocabulary engineering of representations is significantly reduced, and the correct concepts can be learned from few examples. We illustrate the method in an implemented system, called Wyl2, that learns concepts in a variety of domains including the concepts "skewer" and "knight-fork" in chess.Key words: Learning from examples, induction over explanations, explanation based learning, similarity based learning, inductive learning, evaluation of learning methods

Two short papers on machine learning

This technical report reprints two articles that appeared in Proceedings of the Third International Machine Learning Workshop at Skytop, Pennsylvania, June 24-26, 1985. The first paper, The EG Project: Recent Progress, summarizes work on the EG project, which is investigating the role of active experimentation in aiding machine learning programs. The second paper, Exploiting Functional Vocabularies to Learn Structural Descriptions describes work on the problem of developing computer programs that automatically construct their own repre­sentational vocabulary