Executing Evaluations over Semantic Technologies using the SEALS Platform

The SEALS European project aims to develop an infrastructure for the evaluation of semantic technologies. This paper presents in detail the approach followed to automate the execution of these evaluations in the infrastructure. To materialize this approach, we have dened the entities managed by the infrastructure and their life cycle, the process followed to execute evaluations, the management of the computing resources that form the execution infrastructure, and how tools can be integrated with the infrastructure

Design of the second evaluation campaign

wrigley2011aThis deliverable is concerned with the implementation of the second evaluation campaign based upon the methodology and design recommendations made in SEALS Deliverable D3.1. This deliverable covers the initial preparation of the second SEALS Evaluation Campaign and describes the tasks that have been performed during the Initiation and Involvement phases. Furthermore, the deliverable describes the steps to be taken over the next few months and the actors who are responsible for those steps

Automating OAEI Campaigns (First Report)

trojahn2010cInternational audienceThis paper reports the first effort into integrating OAEI and SEALS evaluation campaigns. The SEALS project aims at providing standardized resources (software components, data sets, etc.) for automatically executing evaluations of typical semantic web tools, including ontology matching tools. A first version of the software infrastructure is based on the use of a web service interface wrapping the functionality of a matching tool to be evaluated. In this setting, the evaluation results can visualized and manipulated immediately in a direct feedback cycle. We describe how parts of the OAEI 2010 evaluation campaign have been integrated into this software infrastructure. In particular, we discuss technical and organizational aspects related to the use of the new technology for both participants and organizers of the OAEI

Ontology Alignment using Biologically-inspired Optimisation Algorithms

It is investigated how biologically-inspired optimisation methods can be used to compute alignments between ontologies. Independent of particular similarity metrics, the developed techniques demonstrate anytime behaviour and high scalability. Due to the inherent parallelisability of these population-based algorithms it is possible to exploit dynamically scalable cloud infrastructures - a step towards the provisioning of Alignment-as-a-Service solutions for future semantic applications

A web-based evaluation service for ontology matching

euzenat2010cInternational audienceEvaluation of semantic web technologies at large scale, including ontology matching, is an important topic of semantic web research. This paper presents a web-based evaluation service for automatically executing the evaluation of ontology matching systems. This service is based on the use of a web service interface wrapping the functionality of a matching tool to be evaluated and allows developers to launch evaluations of their tool at any time on their own. Furthermore, the service can be used to visualise and manipulate the evaluation results. The approach allows the execution of the tool on the machine of the tool developer without the need for a runtime environment

A Lime-Flavored REST API for Alignment Services

A practical alignment service should be flexible enough to handle the varied alignment scenarios that arise in the real world, while minimizing the need for manual configuration. MAPLE, an orchestration framework for ontology alignment, supports this goal by coordinating a few loosely coupled actors, which communicate and cooperate to solve a matching task using explicit metadata about the input ontologies, other available resources and the task itself. The alignment task is thus summarized by a report listing its characteristics and suggesting alignment strategies. The schema of the report is based on several metadata vocabularies, among which the Lime module of the OntoLex-Lemon model is particularly important, summarizing the lexical content of the input ontologies and describing external language resources that may be exploited for performing the alignment. In this paper, we propose a REST API that enables the participation of downstream alignment services in the process orchestrated by MAPLE, helping them self-adapt in order to handle heterogeneous alignment tasks and scenarios. The realization of this alignment orchestration effort has been performed through two main phases: we first described its API as an OpenAPI specification (a la API-first), which we then exploited to generate server stubs and compliant client libraries. Finally, we switched our focus to the integration of existing alignment systems, with one fully integrated system and an additional one being worked on, in the effort to propose the API as a valuable addendum to any system being developed