Relaxed Precision and Recall for Ontology Matching

ehrig2005aInternational audienceIn order to evaluate the performance of ontology matching algorithms it is necessary to confront them with test ontologies and to compare the results. The most prominent criteria are precision and recall originating from information retrieval. However, it can happen that an alignment be very close to the expected result and another quite remote from it, and they both share the same precision and recall. This is due to the inability of precision and recall to measure the closeness of the results. To overcome this problem, we present a framework for generalizing precision and recall. This framework is instantiated by three different measures and we show in a motivating example that the proposed measures are prone to solve the problem of rigidity of classical precision and recall

Generalizing precision and recall for evaluating ontology matching

ehrig2005bInternational audienceWe observe that the precision and recall measures are not able to discriminate between very bad and slightly out of target alignments. We propose to generalise these measures by determining the distance between the obtained alignment and the expected one. This generalisation is done so that precision and recall results are at least preserved. In addition, the measures keep some tolerance to errors, i.e., accounting for some correspondences that are close to the target instead of out of target

Specification of a benchmarking methodology for alignment techniques

euzenat2004lThis document considers potential strategies for evaluating ontology alignment algorithms. It identifies various goals for such an evaluation. In the context of the Knowledge web network of excellence, the most important objective is the improvement of existing methods. We examine general evaluation strategies as well as efforts that have already been undergone in the specific field of ontology alignment. We then put forward some methodological and practical guidelines for running such an evaluation

Description of alignment implementation and benchmarking results

stuckenschmidt2005aThis deliverable presents the evaluation campaign carried out in 2005 and the improvement participants to these campaign and others have to their systems. We draw lessons from this work and proposes improvements for future campaigns

Relaxed Precision and Recall for Ontology Matching

In order to evaluate the performance of ontology matching algorithms it is necessary to confront them with test ontologies and to compare the results. The most prominent criteria are precision and recall originating from information retrieval. However, it can happen that an alignment be very close to the expected result and another quite remote from it, and they both share the same precision and recall. This is due to the inability of precision and recall to measure the closeness of the results. To overcome this problem, we present a framework for generalizing precision and recall. This framework is instantiated by three different measures and we show in a motivating example that the proposed measures are prone to solve the problem of rigidity of classical precision and recall

Ontology Mapping by Axioms (OMA)

this paper, we determine mappings based on the similarity of the features of individual ontological entities. We show that mappings can be derived automatically and on the fly by encoding similarities into logical axioms and processing them with an inference engine. The advantages of this approach are obvious. Firstly, the axioms can easily be reused for mappings of arbitrary ontologies, no additional modeling effort is required. Secondly, the inference engine is the only mandatory technological infrastructure which means that no additional implementation effort is neede

Y.: Foam - framework for ontology alignment and mapping; results of the ontology alignment initiative

This paper briefly introduces the system FOAM and its underlying techniques. We then discuss the results returned from the evaluation. They were very promising and at the same time clarifying. Concisely: labels are very important; structure helps in cases where labels do not work; dictionaries may provide additional evidence; ontology management systems need to deal with OWL-Full. The results of this paper will also be very interesting for other participants, showing specific strengths and weaknesses of our approach. 1. PRESENTATION OF THE SYSTEM 1.1 State, purpose, general statement In recent years, we have seen a range of research work on methods proposing alignments [1; 2]. When we tried to apply these methods to some of the real-world scenarios we address in other research contributions [3], we found that existing alignment methods did not suit the given requirements: • high quality results; • efficiency; • optional user-interaction; • flexibility with respect to use cases; • and easy adjusting and parameterizing. We wanted to provide the end-user with a tool taking ontologies as input and returning alignments (with explanations) as output meeting these requirements. 1.2 Specific techniques used We have observed that alignment methods like QOM [4] or PROMPT [2] may be mapped onto a generic alignment process (Figure 1). Here we will only mention the six major steps to clarify the underlying approach for the FOAM tool. We refer to [4] for a detailed description. 1. Feature Engineering, i.e. select excerpts of the overall ontology definition to describe a specific. This includes individual features, e.g. labels, structural features, e.g. subsumption, but also more complex features as used in OWL, e.g. restrictions. 2. Search Step Selection, i.e. choose two entities from the two ontologies to compare (e1,e2). 3. Similarity Assessment, i.e. indicate a similarity for a given description (feature) of two entities (e.g., sim superConcept(e 1,e 2)=1.0)

Ontology-Focused Crawling of Web Documents

The Web, the largest unstructured database of the world, has greatly improved access to documents. However, documents on the Web are largely disorganized. Due to the distributed nature of the World Wide Web it is difficult to use it as a tool for information and knowledge management. Therefore, users doing the difficult task of exploring the Web have to be supported by intelligent means