Using Analogy to Acquire Commonsense Knowledge from Human Contributors

The goal of the work reported here is to capture the commonsense knowledge of non-expert human contributors. Achieving this goal will enable more intelligent human-computer interfaces and pave the way for computers to reason about our world. In the domain of natural language processing, it will provide the world knowledge much needed for semantic processing of natural language. To acquire knowledge from contributors not trained in knowledge engineering, I take the following four steps: (i) develop a knowledge representation (KR) model for simple assertions in natural language, (ii) introduce cumulative analogy, a class of nearest-neighbor based analogical reasoning algorithms over this representation, (iii) argue that cumulative analogy is well suited for knowledge acquisition (KA) based on a theoretical analysis of effectiveness of KA with this approach, and (iv) test the KR model and the effectiveness of the cumulative analogy algorithms empirically. To investigate effectiveness of cumulative analogy for KA empirically, Learner, an open source system for KA by cumulative analogy has been implemented, deployed, and evaluated. (The site "1001 Questions," is available at http://teach-computers.org/learner.html). Learner acquires assertion-level knowledge by constructing shallow semantic analogies between a KA topic and its nearest neighbors and posing these analogies as natural language questions to human contributors. Suppose, for example, that based on the knowledge about "newspapers" already present in the knowledge base, Learner judges "newspaper" to be similar to "book" and "magazine." Further suppose that assertions "books contain information" and "magazines contain information" are also already in the knowledge base. Then Learner will use cumulative analogy from the similar topics to ask humans whether "newspapers contain information." Because similarity between topics is computed based on what is already known about them, Learner exhibits bootstrapping behavior --- the quality of its questions improves as it gathers more knowledge. By summing evidence for and against posing any given question, Learner also exhibits noise tolerance, limiting the effect of incorrect similarities. The KA power of shallow semantic analogy from nearest neighbors is one of the main findings of this thesis. I perform an analysis of commonsense knowledge collected by another research effort that did not rely on analogical reasoning and demonstrate that indeed there is sufficient amount of correlation in the knowledge base to motivate using cumulative analogy from nearest neighbors as a KA method. Empirically, evaluating the percentages of questions answered affirmatively, negatively and judged to be nonsensical in the cumulative analogy case compares favorably with the baseline, no-similarity case that relies on random objects rather than nearest neighbors. Of the questions generated by cumulative analogy, contributors answered 45% affirmatively, 28% negatively and marked 13% as nonsensical; in the control, no-similarity case 8% of questions were answered affirmatively, 60% negatively and 26% were marked as nonsensical

Recognition and classification by exploration

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science; and, Thesis (B.S.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998.Includes bibliographical references (p. 66).by Timothy A. Chklovski.B.S.M.Eng

Creating Large Annotated Data Collections with Web Users' Help

This paper discusses creating annotated data collections

The SENSEVAL-3 English Lexical Sample Task

This paper presents the task definition, resources, participating systems, and comparative results for the English lexical sample task, which was organized as part of the SENSEVAL-3 evaluation exercise

The SENSEVAL-3 Multilingual English-Hindi Lexical Sample Task

This paper describes the English-Hindi Multilingual lexical sample task in SENSEVAL-3

An Evaluation Exercise for Romanian Word Sense Disambiguation

This paper discusses an evaluation exercise for Romanian word sense disambiguation

LEARNER: A System for Acquiring Commonsense Knowledge by Analogy

One of the long-term goals of Artificial Intelligence is construction of a machine that is capable of reasoning about the everyday world the way humans are. In this paper, I first argue that construction of a large collection of statements about everyday world (a repository of commonsense knowledge) is a valuable step towards this long-term goal. Then, I point out that volunteer contributors over the Internet — a frequently overlooked source of knowledge — can be tapped to construct such a knowledge repository. To operationalize construction of a large commonsense knowledge repository by volunteer contributors, I then introduce cumulative analogy, a class of analogy-based reasoning algorithms that leverage existing knowledge to pose knowledge acquisition questions to the volunteer contributors. The algorithms have been implemented and deployed as the Learner system. To date, about 3,400 volunteer contributors have interacted with the system over the course of 11 months, increasing a starting collection of 47,147 statements by 362 % to a total of 217,971. The deployed system and the growing collection of knowledge it acquired are publicly available fro

Building a Sense Tagged Corpus with Open Mind Word Expert

Open Mind Word Expert is an implemented active learning system for collecting word sense tagging from the general public over the Web. It is available a