Comunicación presentada en Cross-Language Evaluation Forum (CLEF 2008), Aarhus, Denmark, September 17-19, 2008.In QA@CLEF 2008, we participate in monolingual (Spanish) and multilingual (English - Spanish) tasks. Specifically, in this paper, we will tackle with the English - Spanish QA task. In this edition we will deal with two main problems: an heterogeneous document collection (news articles and Wikipedia) and a large number of topic-related questions, which make somewhat difficult our participation. We want to highlight in the translation module in our system two possible mechanisms: one based on logic forms, and the other, on machine translation techniques. In addition, it has also been used a system of anaphora resolution that it is described below and a QA System, AliQAn (also used this year in the monolingual task)

Ferrández, Antonio

Gómez, José M.

Martínez-Barco, Patricio

Muñoz Terol, Rafael

Pardiño Juan, María

Peral, Jesús

Puchol Blasco, Marcel

Roger Calzetti, Sandra Emilce

Vila Rodríguez, Katia

Repositorio Institucional de la Universidad de Alicante

AliQAn, Spanish QA System at multilingualQA@CLEF-2008∗R. Mun˜oz-Terol, M.Puchol-Blasco, M. Pardin˜o, J.M. Go´mez, S.Roger, K. Vila,A. Ferra´ndez, J. Peral, P. Mart´ınez-BarcoGrupo de Investigacio´n en Procesamiento del Lenguaje Natural y Sistemas de Informacio´nNatural Language Processing and Information Systems Group Department of Softwareand Computing SystemsUniversity of Alicante, Spainrafamt,marcel,maria,jmgomez,sroger,kvila,antonio,jperal,patricio@dlsi.ua.esAbstractIn QA@CLEF 2008, we participate in monolingual (Spanish) and multilingual (English- Spanish) tasks. Specifically, in this paper, we will tackle with the English - Spanish QAtask. In this edition we will deal with two main problems: an heterogeneous documentcollection (news articles and Wikipedia) and a large number of topic-related questions, whichmake somewhat difficult our participation. We want to highlight in the translation modulein our system two possible mechanisms: one based on logic forms, and the other, on machinetranslation techniques. In addition, it has also been used a system of anaphora resolutionthat it is described below and a QA System, AliQAn (also used this year in the monolingualtask).1 IntroductionFor our participation in this task we have used a monolingual QA system: AliQAn; a translationsystem based on logic forms and an anaphora resolution system to address the interrelated ques-tions and answers.The system AliQAn was widely described by Roger et al. [8] after their participation in thecompetition of the Spanish CLEF 2005. It is important to emphasize that this system has beenadapted to work in this year’s task monolingual too, since there are a great number of changesmade in Roger et al. [9].The main problems that contributed to make difficult our participation were the processing ofWikipedia (this corpus is different from those used to work with AliQAn) and the large numberof questions related.In this research work, our QA system applies two different methods to perform the translationof the questions from one language to another. The first one consists in applying natural languageprocessing techniques based on the formal representation of questions, by using logic forms [10]and, the second one, simply consists in using machine translation techniques. So, the main goalconsists in comparing both methods in the framework of the efficiency of the question answeringsystem to solve the question answering problem.The anaphora resolution problem has been introduced in the last two years in QA task, in-creasing the number of questions with anaphoric relationships since last year. As a consequence, a∗This paper has been partially supported by the Spanish government, project TIN-2006-15265-C06-01 andproject GV06/028, and by the framework of the project QALL-ME, which is a 6th Framework Research Programmeof the European Union (EU), contract number: FP6-IST-033860, by the Generalitat Valenciana throught theresearch grants BFPI/2008/093 and BFPI06/182 and by the University of Comahue under the project 04/E062.1spanish rule-based anaphora resolution system for QA has been implemented to solve this problem.In our system, three kinds of anaphora can be solved: pronoun anaphora, definite descriptions,and zero anaphora. The results obtained by our anaphora resolution system are around 40%precision, using the questions translated from English into Spanish.2 BackgroundPrevious research works such as Bowden et al. [1] directly apply machine translation methods totranslate the questions. On the contrary, and in order to perform the translation of the questionsfrom one language to other one, the research work developed by Ferra´ndez and Ferra´ndez [4] andthe PhD thesis defended by Ferra´ndez [3] exhibits that the use of machine translation techniquesto perform the translation of questions from one language to another decreases the efficiency ofthe multilingual question answering system versus other approaches based on natural languageprocessing techniques.3 Description of the System3.1 English-Spanish Translation based on Logic Forms and lexical re-sourcesThe translation of questions from English into Spanish is performed by inferring the logic formof the questions and using lexical resources to translate the predicates of the logic form. Thetechnique applied to infer the logic form of the questions is the one developed by Terol et al. [10].Consequently, this translation technique is performed as follows:• The predicates whose types are corresponded to noun or verb are translated using the Eu-roWordNet [11] lexical resource. The connection between the synsets of the English andSpanish WordNets is performed in a similar way as treated by Ferrandez et al. [5] usingthe Inter-Lingual-Index (ILI). Each one of the synsets that are mapped from English intoSpanish towards ILI contain a set of concepts. The processing consists in counting the occur-rences of the different concepts that appear in the mapping process and, finally, the conceptwith the highest number of occurrences is chosen as the translation of the predicate.• The predicates whose type is instantiated to adjective, adverb or preposition, and the pred-icates treated in the previous step that are not translated by EuroWordNet are translatedapplying the English-Spanish Babylon dictionary 1. As it occurred in the previous step,the dictionary can return a set of different translations grouped in synsets (different fromWordNet synsets). Thus, the processing consists in counting the occurrences of the differenttranslations that the dictionary returns and, finally, the translation with the highest numberof occurrences is chosen as the translation of the predicate.• Finally, the rest of predicates and the ones than are not translated in the two previous stepsare definitely translated by using the Google Translation Toolkit 2.Once the predicates of the logic form are translated according to the recently detailed rules,the last translation task consists in translating the question as a result of applying some English-Spanish contrastive grammar rules to the sequence of translations of predicates of the logic form.The applied English-Spanish contrastive grammar rules are based on the ones derived from the1http://www.babylon.com2http://www.google.com/translate t2previous study developed by Fernandez et al. [2] and Martinez-Vazquez [6]. The applied English-Spanish contrastive grammar rules are detailed in Table 1, and an example of translation, applyingeach one of these rules, is explained in Table 2. Note that the rule number four is recurrent becausea complex nominal (NNC) can recurringly derive others.Rule Id. English Structure Spanish Translation1 JJ + NN Translation(NN) + Translation(JJ)2 JJ1 + JJ2 + NN Translation(JJ1) + Translation(NN)+ Translation(JJ2)3 NN1 + NN2 Translation(NN1) + Translation(NN2)4 NN + NNC Translation(NNC) + ”de” + Translation(NN)5 JJ + NN1 + NN2 Translation(NN2) + Translation(JJ)+ ”de” + Translation(NN1)Table 1: Applied English-Spanish contrastive grammar rulesRule Id. English Expression Spanish Translation1 red car coche rojo2 beautiful green eyes bonitos ojos verdes3 electronic database base de datos electrnica4 train station ticket office oficina de billete de estacin de tren5 multiple regression model modelo mltiple de regresinTable 2: Example of the applied English-Spanish contrastive grammar rulesFinally, the translation performed by the rest of predicates of the logic form, whose logic struc-ture does not match these English-Spanish contrastive grammar rules, consists in the concatenationof the sequence of translations of these predicates.3.2 Anaphora ResolutionAs last year, anaphora resolution problem has been introduced this year. Questions are groupedin topics. In those topics, the first question is anaphora-free, but the other questions may requireinformation from data contained in the first question, or in the first answer (it can only be thefirst question or the first answer for a specific topic, exactly as described in the guidelines for thisyear).From our analysis of Spanish examples used last year, we have discovered three types of possibleanaphora: pronoun anaphora, definite descriptions, and zero anaphora. These types of anaphorahave been analyzed and are the basis for our system. Our approach for anaphora resolution isrule-based. Basically, it is based on the papers referenced in Mitkov [7].The first step is to get the Part-of-Speech tagging for possible antecedents (the first question orthe first answer for an specific topic) and a question with a possible anaphora. Later, an anaphoradetection module is executed. This module can be considered as three different subsystems inwhich each subsystem corrsponds to: pronoun anaphora detection, definite description detectionand zero anaphora detection. In the first one, pronouns are taken as anaphora. In definitedescriptions, noun phrases are taken as a possible anaphora. And, in zero anaphora, the entirephrase can be taken as a possible anaphora.For all anaphora resolution modules, for each topic group, noun phrases are extracted fromthe first question-answer pair as possible antecedent. Then, when we have a pronoun anaphora,the possible anaphora is compared in gender and number with each possible antecedent. Indefinite descriptions and zero anaphora, for each noun phrase, a google search is launched joiningthe possible antecedent and the main words of the anaphora (or main words contained in thenoun phrases, in case of zero anaphora). Later, for each noun phrase in the possible anaphora,3the relations between it and the main words contained in the antecedents are extracted usingMultiWordNet 3 .In all previous cases, a specific weight is assigned to each possible anaphora resolution. Later,those weights are ordered, and the best case is selected as anaphora resolution for the relatedquestion. It is important to mention that if the noun phrase is in the answer, the greater weightis assigned to it, due to the analysis done in the last year corpus.A few manual experiments are performed using the last year Spanish corpus. The resultsobtained are around 56% precision. The results obtained using the corpus for this year, once thequestions are translated from English into Spanish, are worse, obtaining around 40% precision.4 Results and ConclusionsThe scores obtained in the application of the machine translation techniques are a bit better thanthe ones obtained in the application of the techniques based on logic forms. This can be due to thefact that the use of logic forms is a good method to perform the language-independent knowledgerepresentation, but this method must be improved to perform the translation of sentences fromone language to another. As a further work, the next research goal will be to improve the logicform processing methods in the translation process.We have had some problems with the anaphora resolution system, because it is a rule-basedsystem and it needs well-formed sentences to work. Due to the fact that translations offered bythe system are not well-formed, in most cases, the precision obtained by the anaphora resolutionsystem has decreased considerably, arriving at around 40% precision.Finally, we present the results obtained with our system in this task in Tables 3 and 4.Specifically, Tables 3 shows the number of right answers(R), wrong answers (W), inexact answers(X), unsupported answers (U) obtained by our system.Run ID #Questions #R #W #X #URun 1 (machine translation) 200 25 173 0 2Run 2 (logic forms) 200 18 176 3 3Table 3: Results obtained by our system at English-Spanish QA task 2008Table 4 shows the comparative of the values obtained for MRR, CSW and Accuracy with bothruns.Run ID # Questions Accuracy(%) CSW MRRRun 1 (machine translation) 200 12.5 0.01114 0.17797Run 2 (logic forms) 200 9.0 0.00626 0.11499Table 4: Results obtained by our system at English-Spanish QA task 2008References[1] Mitchell Bowden, Marian Olteanu, Pasin Suriyentrakorn, Thomas d’Silva, and Dan Moldovan.Multilingual question answering through intermediate translation: Lcc’s poweranswer atqa@clef 2007. In Working Notes for the CLEF 2007 Workshop, 2007.[2] F. Ferna´ndez and B. Montero-Fleta. La premodificacio´n nominal en el a´mbito de la informtica.Estudio contrastivo ingle´s-espan˜ol. Universidad de Valencia, 2003.3http://multiwordnet.itc.it/online/multiwordnet.php4[3] S. Ferra´ndez. Arquitectura multilingu¨e de sistemas de Bu´squeda de Respuestas basada en ILIy Wikipedia. PhD thesis, University of Alicante, 2008.[4] S. Ferra´ndez and A. Ferra´ndez. The negative effect of machine translation on crosslingualquestion answering. In Computational Linguistics and Intelligent Text Processing, pages 494–505, 2007.[5] S. Ferra´ndez, A. Ferra´ndez, S. Roger, P. Lo´pez-Moreno, and J. Peral. Brili, an english-spanishquestion answering system. In Proceedings of the International Multiconference on ComputerScience and Information Technology, pages 23–29, 2006.[6] M. Mart´ınez-Va´zquez. Grama´tica contrastiva ingle´s-espan˜ol. Servicio de publicaciones de laUniversidad de Huelva, 1996.[7] Ruslan Mitkov. Anaphora Resolution. Longman, London, 2002.[8] S. Roger, S. Ferra´ndez, A. Ferra´ndez, J. Peral, F. Llopis, A. Aguilar, and D. Toma´s. Aliqan,spanish qa system at clef-2005. In CLEF, pages 457–466, 2005.[9] S. Roger, K. Vila, A. Ferra´ndez, M. Pardin˜o, J.M. Go´mez, M. Puchol-Blasco, and J. Peral.Aliqan, spanish qa system at clef-2008. In CLEF, 2008.[10] R.M. Terol, P. Mart´ınez-Barco, and M. Palomar. A knowledge based method for the medicalquestion answering problem. In Computers in Biology and Medicine, volume 37, pages 1511– 1521, 2007.[11] P. Vossen. EuroWordNet General Document. Part A. Final Document. EuroWordNet (LE2-4003, LE4-8328), 2002.5

AliQAn, Spanish QA system at multilingual QA@CLEF-2008

AliQAn, Spanish QA system at multilingual QA@CLEF-2008

Abstract

Similar works

Full text

Available Versions

Repositorio Institucional de la Universidad de Alicante