Initial specification of the evaluation tasks "Use cases to bridge validation and benchmarking" PROMISE Deliverable 2.1

Evaluation of multimedia and multilingual information access systems needs to be performed from a usage oriented perspective. This document outlines use cases from the three use case domains of the PROMISE project and gives some initial pointers to how their respective characteristics can be extrapolated to determine and guide evaluation activities, both with respect to benchmarking and to validation of the usage hypotheses. The use cases will be developed further during the course of the evaluation activities and workshops projected to occur in coming CLEF conferences

Off-line answer extraction for Question Answering

In deze tijd neemt de hoeveelheid digitale informatie enorm toe. De ontwikkeling van software om deze groeiende berg van informatie te doorzoeken wordt daarbij steeds belangrijker. Een bekend voorbeeld hiervan is de zoekmachine Google. Op basis van sleutelwoorden worden links naar relevante documenten getoond. Maar in sommige situaties willen gebruikers op een concrete vraag een direct antwoord in een paar zinnen. In dergelijke gevallen is een Question Answering (QA) systeem een uitkomst. Zowel vanuit de academische wereld als vanuit het bedrijfsleven is er daarom aandacht voor Question Answering systemen. Elk jaar doen er tientallen academische, commerci¨ele en overheidsinstanties mee aan het Question Answering onderdeel van TREC. Voor niet-engelstalige systemen is er iets soortgelijks, de Cross Language Evaluation Forum (CLEF). Zie verder: Samenvatting

Passage retrieval in legal texts

[EN] Legal texts usually comprise many kinds of texts, such as contracts, patents and treaties. These texts usually include a huge quantity of unstructured information written in natural language. Thanks to automatic analysis and Information Retrieval (IR) techniques, it is possible to filter out information that is not relevant and, therefore, to reduce the amount of documents that users need to browse to find the information they are looking for. In this paper we adapted the JIRS passage retrieval system to work with three kinds of legal texts: treaties, patents and contracts, studying the issues related with the processing of this kind of information. In particular, we studied how a passage retrieval system might be linked up to automated analysis based on logic and algebraic programming for the detection of conflicts in contracts. In our set-up, a contract is translated into formal clauses, which are analysed by means of a model checking tool; then, the passage retrieval system is used to extract conflicting sentences from the original contract text. © 2011 Elsevier Inc. All rights reserved.We thank the MICINN (Plan I+D+i) TEXT-ENTERPRISE 2.0: (TIN2009-13391-C04-03) research project. The work of the second author has been possible thanks to a scholarship funded by Maat Gknowledge in the framework of the project with the Universidad Politécnica de Valencia Módulo de servicios semánticos de la plataforma GRosso, P.; Correa García, S.; Buscaldi, D. (2011). Passage retrieval in legal texts. Journal of Logic and Algebraic Programming. 80(3-5):139-153. doi:10.1016/j.jlap.2011.02.001S139153803-