DI@UE in CLEF2012: question answering approach to the multiple choice QA4MRE challenge

In the 2012 edition of CLEF, the DI@UE team has signed up for Question Answering for Machine Reading Evaluation (QA4MRE) main task. For each question, our system tries to guess which of the five hypotheses is the more plausible response, taking into account the reading test content and the documents from the background collection on the question topic. For each question, the system applies Named Entity Recognition, Question Classification, Document and Passage Retrieval. The criteria used in the first run is to choose the answer with the smallest distance between question and answer key elements. The system applies a specific treatment for certain factual questions, with the categories Quantity, When, Where, What, and Who, whose responses are usually short and likely to be detected in the text. For the second run, the system tries to solve each question according to its category. Textual patterns used for answer validation and Web answer projection are defined according to the question category. The system answered to all 160 questions, having found 50 right candidate answers

Adaptation of LIMSI's QALC for QA4MRE.

International audienceIn this paper, we present LIMSI participation to one of the pilot tasks of QA4MRE at CLEF 2012: Machine Reading of Biomedical Texts about Alzheimer. For this exercise, we adapted an existing question answering (QA) system, QALC, by searching answers in the reading document. This basic version was used for the evaluation and obtains 0.2, which was increased to 0.325 after basic corrections. We developed then different methods for choosing an answer, based on the expected answer type and the question plus answer rewritten to form hypothesis compared with candidates sentences. We also conducted studies on relation extraction by using an existing system. The last version of our system obtains 0.375

An Approach to the Main Task of QA4MRE-2013

This article describes the participation of a group from the University of Évora in the CLEF2013 QA4MRE main task. Our system has a superficial text analysis based approach. The methodology starts with the preprocessing of background collection documents, whose texts are lemmatized and then indexed. Named entities and numerical expressions are sought in questions and their candidate answers. Then the lemmatizer is applied and stop words are removed. Answer patterns are formed for each question+answer pair, with a search query for document retrieval. Original search terms are expanded with synonyms and hyperonyms. Finally, the texts retrieved for each candidate response are segmented and scored for answer selection. Considering only the main questions, the system best result was obtained in the third run, having answered to 206 questions, with 0.24 c@1 and 51 correct answers. When evaluating main and auxiliary questions, the final run continued to have our better results, being answered 245 questions, with 64 right answers and 0.26 for c@1. The use of hypernyms proved to be an improvement factor in the third run, which results had a 12% increase of correct answers and a 0.02 gain in c@1

Selecting answers with structured lexical expansion and discourse relations: LIMSI's participation at QA4MRE 2013

International audiencen this paper, we present the LIMSI’s participation to QA4MRE2013. We decided to test two kinds of methods. The first one focuses on complex questions, such as causal questions, and exploits discourse relations. Relation recognition shows promising results, however it has to be improved to have an impact on answer selection. The second method is based on semantic variations. We explored the English Wiktionary to find reformulations of words in the definitions, and used these reformulations to index the documents and select passages in the Entrance exams task

Arabic QA4MRE at CLEF 2012: Arabic Question Answering for Machine Reading Evaluation

This paper presents the work carried out at ANLP Research Group for the CLEF-QA4MRE 2012 competition. This year, the Arabic language was introduced for the first time on QA4MRE lab at CLEF whose intention was to ask questions which require a deep knowledge of individual short texts and in which systems were required to choose one answer from multiple answer choices, by analyzing the corresponding test document in conjunction with background collections. In our participation, we have proposed an approach which can answer questions with multiple answer choices from short Arabic texts. This approach is constituted essentially of shallow information retrieval methods. The evaluation results of the running submitted has given the following scores: accuracy calculated overall all questions is 0.19 (i.e., 31 correct questions answered correctly among 160), while overall c@1 measure is also 0.19. The overall results obtained are not enough satisfactory comparing to the top works realized last year in QA4MRE lab. But as a first step at the roadmap of the evolution of the QA to Machine Reading (MR) systems in Arabic language and with the lack of researches investigated in the MR and deep knowledge reasoning in Arabic language, it is an encouraging step. Our proposed approach with its shallow criterion has succeeded to obtain the goal fixed at the beginning which is: select answers to questions from short texts without required enough external knowledge and complex inference.Trigui, O.; Hadrich Belguith, L.; Rosso, P.; Ben Amor, H.; Gafsaoui, B. (2012). Arabic QA4MRE at CLEF 2012: Arabic Question Answering for Machine Reading Evaluation. CELCT. http://hdl.handle.net/10251/46315

The C@merata Task at MediaEval 2015: Natural Language Queries on Classical Music Scores

This was the second year of the C@merata task [16,1] which relates natural language processing to music inform ation retrieval. Participants each build a system which takes as input a query and a music score and produces as output one or more ma tching passages in the score. This year, questions were mo re difficult and scores were more complex. Participants were the same as last year and once again CLAS was the best with a Beat F-Score of 0.620

A book-oriented chatbot

The automatic answer to questions in natural language is an area that has been studied for many years. However, based on the existing question answering systems, the percentage of correct answers over a set of questions, generated from a dataset, we can see that the performance it is still far away from to 100%, which is many times the value achieved when the questions are tested by humans. This work addresses the idea of a book-oriented Chatbot, more precisely a question answering system directed to answer to questions in which the dataset is one or more books. This way, we intend to adopt a new system, incorporating two existent projects, the OpenBookQA and the Question-Generation. We have used two Domain Specific Datasets that were not studied in both project, that were the QA4MRE and RACE. To these we have applied the main approach: enrich them with automatic generated questions. We have run many experiments, training neural network models. This way, we intended to study the impact of those questions and obtain good accuracy results for both datasets. The obtained results suggest that having a significant representation of generated questions in a dataset, leads to a higher test accuracy results of correct answers. Becoming clear that, enrich a dataset, based on a book, with generated questions about that book, is giving to the dataset the content of the book. This dissertation presents promising results, through the datasets with automatic generated questions.A resposta automática a perguntas em língua natural é um tema estudado há largos anos. Tendo por base os sistemas existentes de resposta a perguntas, quando comparamos a percentagem de respostas correctas sobre um conjunto de perguntas, geradas a partir de um conjunto de dados, conseguimos ver que o desempenho está ainda longe de 100%, que muitas vezes é o valor alcançado quando as perguntas são testadas por humanos. Este trabalho aborda a ideia de um agente conversacional orientado para livros, mais propriamente um sistema de resposta a perguntas direccionado para responder a perguntas cujo conjunto de dados seja um ou mais livros. Deste modo, pretendemos adoptar um novo sistema, incorporando dois projectos existentes, o OpenBookQA e o Question-Generation. Utilizámos dois conjuntos de dados de domínio específico, sem terem sido ainda estudados nos dois projectos, que foram o QA4MRE e o RACE. A estes aplicámos a abordagem principal: enriquecê-los com perguntas geradas automaticamente. Corremos uma série de experiências, treinando modelos de redes neuronais. Deste modo, pretendemos estudar o impacto das perguntas geradas e obter bons resultados de precisão de respostas correctas para os dois conjuntos de dados. Os resultados obtidos sugerem que ter uma quantidade significativa de perguntas geradas num conjunto de dados, conduz a maior precisão de respostas correctas. Tornando claro que, enriquecer um dataset, sobre um livro, com perguntas geradas sobre esse mesmo livro, é dar ao dataset o contéudo do livro. Esta dissertação apresenta resultados promissores, a partir de conjuntos de dados com perguntas geradas automaticamente

Relating Natural Language Text to Musical Passages

There is a vast body of musicological literature containing detailed analyses of musical works. These texts make frequent references to musical passages in scores by means of natural language phrases. Our long- term aim is to investigate whether these phrases can be linked automatically to the musical passages to which they refer. As a first step, we have organised for two years running a shared evaluation in which participants must develop software to identify passages in a MusicXML score based on a short noun phrase in English. In this paper, we present the rationale for this work, discuss the kind of references to musical passages which can occur in actual scholarly texts, describe the first two years of the evaluation and finally appraise the results to establish what progress we have made

Réponse à des tests de compréhension.

National audienceDans cet article, nous présentons une adaptation d’un système de questions-réponses existant pour une tâche de réponse à des questions de compréhension de textes. La méthode proposée pour sélectionner les réponses correctes repose sur la reconnaissance d’implication textuelle entre les hypothèses et les textes. Les spécificités de cette méthode sont la génération d’hypothèses par réécriture syntaxique, et l’évaluation de plusieurs critères de distance,adaptés pour gérer des variantes de termes