Corpus-based automatic detection of example sentences for dictionaries for Estonian learners

Väitekirja elektrooniline versioon ei sisalda publikatsiooneNäitelause täidab sõnastikus kindlat eesmärki, aidates aru saada sõna tähendusest ja illustreerides sõna erinevaid kasutuskontekste. Näitelausete põhiallikas on mahukas tekstikorpus, kust aga käsitsi on näitelauset leida väga keeruline. Elektroonilise leksikograafia arenguga on Eestisse jõudnud mitmed töövahendid, mis aitavad automaatselt tuvastada eri sõnastike jaoks vajalikku infot, sealhulgas näitelauseid. Väitekirjas uuritakse, missugused parameetrid iseloomustavad Eesti Keele Instituudis koostatud sõnastike "Eesti keele sõnaraamat 2019", "Eesti keele põhisõnavara sõnastik 2014", "Eesti keele naabersõnad 2019" näitelauseid ning "Eesti keele A1−C1 õpikute korpuse 2018" lauseid. Uurimuse eesmärk on välja töötada meetod, mis võimaldab neid parameetreid arvestades korpusest automaatselt tuvastada eesti keele õppijatele sobivaid lauseid. Töö keskmes on reeglipõhine lähenemine, mida rakendatakse korpuspäringusüsteemi Sketch Engine integreeritud tööriista GDEX ehk Good Dictionary Examples näitel. Parameetrite häälestamiseks on osaliselt kasutatud ka masinõppe elemente. Sõnastiku näitelausete ja õpikulausete analüüs näitas, et hea eesti keele näitelause peab olema täislause ja vastama muuhulgas järgmistele parameetritele: on 4–20 sõnet pikk; ei sisalda sõnesid, mis on pikemad kui 20 tähemärki; ei alga teatud sõnaliikidega (nt sidesõnaga) ega tagasi viitavate sõnade (nt sellepärast) või sõnapaaridega (nt sellisel puhul); ei sisalda vulgaarseid ja halvustavaid sõnu, madala sagedusega sõnu jmt. Uurimuse tulemusena on loodud "Eesti keele õppekorpus 2018 (etSkELL)", mis sisaldab ainult välja töötatud parameetritele vastavaid lauseid. Õppekorpus on omakorda aluseks eesti keele õppekeskkonnale Sketch Engine for Estonian Language Learning ehk etSkELL ja veebilausetele Eesti Keele Instituudi keeleportaalis Sõnaveeb.The function of an example sentence in a dictionary is to help the reader understand the meaning of the headword and illustrate its contexts of use. Nowadays, the main source of example sentences is a large text corpus, where suitable sentences are hard to find. Luckily, e-lexicography has generated automatic tools to help detect various information for dictionaries, including example sentences. The dissertation examines certain parameters of the example sentences presented in the Dictionary of Estonian (2019), Basic Estonian Dictionary (2014), Estonian Collocations Dictionary (2019), and Estonian Coursebook Corpus (2018); all four were compiled at the Institute of the Estonian language. The aim of my study is to elaborate an automatic method using parameters which identify sentences suitable for learners of Estonian. To that end, a rule-based approach was applied to the example of Good Dictionary Examples (GDEX) integrated in the Sketch Engine corpus query tool. Machine learning elements were also adopted to fine-tune the parameters. According to the analysis of the example sentences used in the dictionaries and coursebook sentences, a good Estonian example sentence should be a full sentence meeting, inter alia, the following parameters: length 4–20 tokens; no tokens longer than 20 characters; never begins with certain parts of speech (e.g., conjunction) or an anaphoric word (e.g., sellepärast ‘this is why’) or word pair (e.g., sellisel puhul ‘in such a case’); and vulgar or disparaging words, rare words, etc., are excluded. The study resulted in the compilation of the Estonian Corpus for Learners 2018 (etSkELL), which contains no other sentences but those corresponding to the developed parameters. The corpus, in turn, serves as the basis for the corpus-based web tool Sketch Engine for Estonian Language Learning (etSkELL) and the web sentences in the language portal Sõnaveeb of the Institute of the Estonian Language.https://www.ester.ee/record=b530293

Annotation en rôles sémantiques du français en domaine spécifique

In this Natural Language Processing Ph. D. Thesis, we aim to perform semantic role labeling on French domain-specific texts. This task first disambiguates the sense of predicates in a given text and annotates its child chunks with semantic roles such as Agent, Patient or Destination. The task helps many applications in domains where annotated corpora exist, but is difficult to use otherwise. We first evaluate on the FrameNet corpus an existing method based on VerbNet, which explains why the method is domain-independant. We show that substantial improvements can be obtained. We first use syntactic information by handling the passive voice. Next, we use semantic informations by taking advantage of the selectional restrictions present in VerbNet. To apply this method to French, we first translate lexical resources. We first translate the WordNet lexical database. Next, we translate the VerbNet lexicon which is organized semantically using syntactic information. We obtain its translation, VerbeNet, by reusing two French verb lexicons (the Lexique-Grammaire and Les Verbes Français) and by manually modifying and reorganizing the resulting lexicon. Finally, once those building blocks are in place, we evaluate the feasibility of semantic role labeling of French and English in three specific domains. We study the pros and cons of using VerbNet and VerbeNet to annotate those domains before explaining our future work.Cette thèse de Traitement Automatique des Langues a pour objectif l'annotation automatique en rôles sémantiques du français en domaine spécifique. Cette tâche désambiguïse le sens des prédicats d'un texte et annote les syntagmes liés avec des rôles sémantiques tels qu'Agent, Patient ou Destination. Elle aide de nombreuses applications dans les domaines où des corpus annotés existent, mais est difficile à utiliser quand ce n'est pas le cas. Nous avons d'abord évalué sur le corpus FrameNet une méthode existante d'annotation basée uniquement sur VerbNet et donc indépendante du domaine considéré. Nous montrons que des améliorations conséquentes peuvent être obtenues à la fois d'un point de vue syntaxique avec la prise en compte de la voix passive et d'un point de vue sémantique en utilisant les restrictions de sélection indiquées dans VerbNet. Pour utiliser cette méthode en français, nous traduisons deux ressources lexicales anglaises. Nous commençons par la base de données lexicales WordNet. Nous traduisons ensuite le lexique VerbNet dans lequel les verbes sont regroupés sémantiquement grâce à leurs traits syntaxiques. La traduction, VerbeNet, a été obtenue en réutilisant deux lexiques verbaux du français (le Lexique-Grammaire et Les Verbes Français) puis en modifiant manuellement l'ensemble des informations obtenues. Enfin, une fois ces briques en place, nous évaluons la faisabilité de l'annotation en rôles sémantiques en anglais et en français dans trois domaines spécifiques. Nous évaluons quels sont les avantages et inconvénients de se baser sur VerbNet et VerbeNet pour annoter ces domaines, avant d'indiquer nos perspectives pour poursuivre ces travaux

Normalisation of imprecise temporal expressions extracted from text

Orientador : Prof. Dr. Marcos Didonet Del FabroCo-Orientador : Prof. Dr. Angus RobertsTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa: Curitiba, 05/04/2016Inclui referências : f. 95-105Resumo: Técnicas e sistemas de extração de informações são capazes de lidar com a crescente quantidade de dados não estruturados disponíveis hoje em dia. A informação temporal está entre os diferentes tipos de informações que podem ser extraídos a partir de tais fontes de dados não estruturados, como documentos de texto. Informações temporais descrevem as mudanças que acontecem através da ocorrência de eventos, e fornecem uma maneira de gravar, ordenar e medir a duração de tais ocorrências. A impossibilidade de identificar e extrair informação temporal a partir de documentos textuais faz com que seja difícil entender como os eventos são organizados em ordem cronológica. Além disso, em muitas situações, o significado das expressões temporais é impreciso, e não pode ser descrito com precisão, o que leva a erros de interpretação. As soluções existentes proporcionam formas alternativas de representar expressões temporais imprecisas. Elas são, entretanto, específicas e difíceis de generalizar. Além disso, a análise de dados temporais pode ser particularmente ineficiente na presença de erros ortográficos. As abordagens existentes usam métodos de similaridade para procurar palavras válidas dentro de um texto. No entanto, elas não são suficientes para processos erros de ortografia de uma forma eficiente. Nesta tese é apresentada uma metodologia para analisar e normalizar das expressões temporais imprecisas, em que, após a coleta e pré-processamento de dados sobre a forma como as pessoas interpretam descrições vagas de tempo no texto, diferentes técnicas são comparadas a fim de criar e selecionar o modelo de normalização mais apropriada para diferentes tipos de expressões imprecisas. Também são comparados um sistema baseado em regras e uma abordagem de aprendizagem de máquina na tentativa de identificar expressões temporais em texto, e é analisado o processo de produção de padrões de anotação, identificando possíveis fontes de problemas, dando algumas recomendações para serem consideradas no futuro esforços de anotação manual. Finalmente, é proposto um mapa fonético e é avaliado como a codificação de informação fonética poderia ser usado a fim de auxiliar os métodos de busca de similaridade e melhorar a qualidade da informação extraída.Abstract: Information Extraction systems and techniques are able to deal with the increasing amount of unstructured data available nowadays. Time is amongst the different kinds of information that may be extracted from such unstructured data sources, including text documents. Time describes changes which happen through the occurrence of events, and provides a way to record, order, and measure the duration of such occurrences. The inability to identify and extract temporal information from text makes it difficult to understand how the events are organized in a chronological order. Moreover, in many situations, the meaning of temporal expressions is imprecise, and cannot be accurately described, leading to interpretation errors. Existing solutions provide alternative ways of representing imprecise temporal expressions, though they are specific and hard to generalise. Furthermore, the analysis of temporal data may be particularly inefficient in the presence of spelling errors. Existing approaches use string similarity methods to search for valid words within a text. However, they are not rich enough to processes misspellings in an efficient way. In this thesis, we present a methodology to analyse and normalise of imprecise temporal expressions, in which, after collecting and pre-processing data on how people interpret vague descriptions of time in text, we compare different techniques in order to create and select the most appropriate normalisation model for different kinds of imprecise expressions. We also compare how a rule-based system and a machine learning approach perform on trying to identify temporal expression from text, and we analyse the process of producing gold standards, identifying possible sources of issues, giving some recommendations to be considered in future manual annotation efforts. Finally, we propose a phonetic map and evaluate how encoding phonetic information could be used in order to assist similarity search methods and improve information extraction quality