Integrazioa hizkuntzaren prozesamenduan. Anotazio-eskemak eta elkarreragingarritasuna.Testuen prozesatze masiboa, datu handien teknikak erabiliz

179 p.Tesi-lan honetan hizkuntzaren prozesamenduko tresnen integrazioa landudugu, datu handien teknikei arreta berezia eskainiz. Tresnenintegrazioa, izatez, bi mailatan landu dugu: anotazio-eskemen mailaneta prozesuen mailan.Anotazio-eskemen mailako integrazioan tresnen artekoelkarreragingarritasuna lortzeko lehenbiziko pausoak aurkeztea izandugu helburu. Horrekin lotuta, bi anotazio-eskema aurkeztu ditugu:Anotazio-Amaraunen Arkitektura (AWA, Annotation Web Architecture) etaNLP Annotation Format (NAF). AWA tesi-lan honekin hasi aurretik sortuaizan zen, eta orain formalizazio-lan bat egin dugu berarekin,elkarreragingarritasunari arreta berezia jarriz. NAF, bere aldetik,eskema praktikoa eta sinplea izateko helburuekin sortu dugu. Bianotazio-eskema horietatik abiatuz, eskemarekiko independentea deneredu abstraktu bat diseinatu dugu. Abstrakzio horri esker,elkarreragingarritasunerantz jotzeko bidea zabaldu nahi izan dugu,eredu abstraktua edozein eskemarekin bateragarria dela argudiatuz.Bestalde, tresnen prozesu mailako integrazioa ere landudugu. Horretarako, analisi-kateak modu malguan eta deklaratiboaneraikitzeko azpiegitura bat diseinatu eta inplementatu dugu. Gainera,azpiegitura horretan oinarrituz eta datu handien teknikak aplikatuz,testu-dokumentuen bilduma erraldoiak modu banatuan eta eskalagarrianprozesatzeko arkitektura bat diseinatu eta inplementatu dugu. Sistemahori hainbat nodoz osatutako terminal talde batean ezarriz, baianalisi-kateko tresnak eta bai prozesatu beharreko dokumentuak,automatikoki, eskura dauden nodoetan zehar banatuko dira, sistemaosoaren ahalmenari ahalik eta etekin handiena ateraz

Task-based parser output combination : workflow and infrastructure

This dissertation introduces the method of task-based parser output combination as a device to enhance the reliability of automatically generated syntactic information for further processing tasks. Parsers, i.e. tools generating syntactic analyses, are usually based on reference data. Typically these are modern news texts. However, the data relevant for applications or tasks beyond parsing often differs from this standard domain, or only specific phenomena from the syntactic analysis are actually relevant for further processing. In these cases, the reliability of the parsing output might deviate essentially from the expected outcome on standard news text. Studies for several levels of analysis in natural language processing have shown that combining systems from the same analysis level outperforms the best involved single system. This is due to different error distributions of the involved systems which can be exploited, e.g. in a majority voting approach. In other words: for an effective combination, the involved systems have to be sufficiently different. In these combination studies, usually the complete analyses are combined and evaluated. However, to be able to combine the analyses completely, a full mapping of their structures and tagsets has to be found. The need for a full mapping either restricts the degree to which the participating systems are allowed to differ or it results in information loss. Moreover, the evaluation of the combined complete analyses does not reflect the reliability achieved in the analysis of the specific aspects needed to resolve a given task. This work presents an abstract workflow which can be instantiated based on the respective task and the available parsers. The approach focusses on the task-relevant aspects and aims at increasing the reliability of their analysis. Moreover, this focus allows a combination of more diverging systems, since no full mapping of the structures and tagsets from the single systems is needed. The usability of this method is also increased by focussing on the output of the parsers: It is not necessary for the users to reengineer the tools. Instead, off-the-shelf parsers and parsers for which no configuration options or sources are available to the users can be included. Based on this, the method is applicable to a broad range of applications. For instance, it can be applied to tasks from the growing field of Digital Humanities, where the focus is often on tasks different from syntactic analysis