Open- vs. Restricted-Domain QA Systems in the Biomedical Field

Question Answering Systems (hereinafter QA systems) stand as a new alternative for Information Retrieval Systems. We conducted a study to evaluate the efficiency of QA systems as terminological sources for physicians, specialized translators, and users in general. To this end we analysed the performance of two open-domain and two restricted domain QA systems. The research entailed a collection of one hundred fifty definitional questions from WebMed. We studied the sources that QA systems used to retrieve the answers, and later applied a range of evaluation measures to mark the quality of answers. Through analysing the results obtained by asking the 150 questions in the QA systems MedQA, START, QuALiM and HONqa, it was possible to evaluate the systems’ operation through applying specific metrics. Despite the limitations demonstrated by these systems, as they are not accessible to everyone and they are not always completely developed, it has been confirmed that these four QA systems are valid and useful for obtaining definitional medical information in that they offer coherent and precise answers. The results are encouraging because they present this type of tool as a new possibility for gathering precise, reliable and specific information in a short period of time

Open- vs. Restricted-Domain QA Systems in the Biomedical Field

Question Answering Systems (hereinafter QA systems) stand as a new alternative for Information Retrieval Systems. We conducted a study to evaluate the efficiency of QA systems as terminological sources for physicians, specialized translators, and users in general. To this end we analysed the performance of two open-domain and two restricted domain QA systems. The research entailed a collection of one hundred fifty definitional questions from WebMed. We studied the sources that QA systems used to retrieve the answers, and later applied a range of evaluation measures to mark the quality of answers. Through analysing the results obtained by asking the 150 questions in the QA systems MedQA, START, QuALiM and HONqa, it was possible to evaluate the systems’ operation through applying specific metrics. Despite the limitations demonstrated by these systems, as they are not accessible to everyone and they are not always completely developed, it has been confirmed that these four QA systems are valid and useful for obtaining definitional medical information in that they offer coherent and precise answers. The results are encouraging because they present this type of tool as a new possibility for gathering precise, reliable and specific information in a short period of time

IBM’s PIQUANT II in TREC2004

PIQUANT II, the system we used for TREC 2004, is a completely reengineered system whose core functionalities for answering factoid and list questions remain largely unchanged from previous. We continue to address these questions using our multi-strategy and multi-source approach

Language Resources Used in Multi-Lingual Question Answering Systems

Purpose – In the field of information retrieval, some multi-lingual tools are being created to help the users to overcome the language barriers. Nevertheless, these tools are not developed completely and it is necessary to investigate more for their improvement and application. One of their main problems is the choice of the linguistic resources to offer better coverage and to solve the translation problems in the context of the multi-lingual information retrieval. This paper aims to address this issue. Design/methodology/approach – This research is focused on the analysis of resources used by the multi-lingual question-answering systems, which respond to users' queries with short answers, rather than just offering a list of documents related to the search. An analysis of the main publications about the multi-lingual QA systems was carried out, with the aim of identifying the typology, the advantages and disadvantages, and the real use and trend of each of the linguistic resources and tools used in this new kind of system. Findings – Five of the resources most used in the cross-languages QA systems were identified and studied: databases, dictionaries, corpora, ontologies and thesauri. The three most popular traditional resources (automatic translators, dictionaries, and corpora) are gradually leaving a widening gap for others – such as ontologies and the free encyclopaedia Wikipedia. Originality/value – The perspective offered by the translation discipline can improve the effectiveness of QA system

Evaluación de los sistemas QA de dominio abierto frente a los de dominio especializado en el ámbito biomédico

Los sistemas QA se presentan como una alternativa a los sistemas tradicionales de recuperación de información tratando de ofrecer respuestas precisas a preguntas factuales. Hemos realizado un estudio para evaluar la eficiencia de estos sistemas como fuentes terminológicas para los especialistas y para usuarios en general. Con este fin, se ha evaluado el funcionamiento de cuatro sistemas QA, dos especializados en el dominio biomédico (MedQA y HONqa) y dos de dominio general (START y QuALiM). El estudio ha utilizado una colección de 150 preguntas biomédicas definicionales (What is…?), obtenidas del sitio web médico WebMD. Para determinar el funcionamiento, se han evaluado las respuestas ofrecidas utilizando una serie de medidas específicas (precisión, MRR, TRR, FHS). El estudio permite confirmar que los cuatro sistemas son útiles para la recuperación de información definicional en este ámbito, ya que han proporcionado respuestas coherentes y precisas con un grado de aceptabilidad adecuado.Grupo de Investigación: Acceso y evaluación de la información científica (HUM466)

Language Resources Used in Multi-Lingual Question Answering Systems

Purpose – In the field of information retrieval, some multi-lingual tools are being created to help the users to overcome the language barriers. Nevertheless, these tools are not developed completely and it is necessary to investigate more for their improvement and application. One of their main problems is the choice of the linguistic resources to offer better coverage and to solve the translation problems in the context of the multi-lingual information retrieval. This paper aims to address this issue. Design/methodology/approach – This research is focused on the analysis of resources used by the multi-lingual question-answering systems, which respond to users' queries with short answers, rather than just offering a list of documents related to the search. An analysis of the main publications about the multi-lingual QA systems was carried out, with the aim of identifying the typology, the advantages and disadvantages, and the real use and trend of each of the linguistic resources and tools used in this new kind of system. Findings – Five of the resources most used in the cross-languages QA systems were identified and studied: databases, dictionaries, corpora, ontologies and thesauri. The three most popular traditional resources (automatic translators, dictionaries, and corpora) are gradually leaving a widening gap for others – such as ontologies and the free encyclopaedia Wikipedia. Originality/value – The perspective offered by the translation discipline can improve the effectiveness of QA system

Columbia University at DUC 2004

We describe our participation in tasks 2, 4 and 5 of the DUC 2004 evaluation. For each task, we present the system (s) used, focusing on novel and newly developed aspects. We also analyze the results of the human and automatic evaluations

Soft matching for question answering

Ph.DDOCTOR OF PHILOSOPH

Satisfacción de usuarios del ámbito de la traducción en el uso de sistemas de búsqueda multilingüe de respuestas como recurso de información terminológica

The present work focuses exclusively on the evaluation of multilingual answer search systems since it allows the user to access terminological information not available in their language, and in the user-centered evaluation to understand the needs of the user and identify the dimensions and factors in the development of an information system in order to improve its acceptance