Two-level Word Class Categorization Model in Analytic Languages and Its Implications for POS Tagging in Modern Chinese Corpora

The study of word classes has a history of over 4000 years, and the word class problem in over 1000 analytic languages like Modern Chinese can be seen as the Goldbach Conjecture in linguistics. This paper first outlines the existing problems in the POS tagging of Modern Chinese corpora with a case study of 自信. Then it introduces the Two-level Word Class Categorization Model in analytic languages, which is based on the perspectives of language as a complex adaptive system and the nature of major parts of speech as propositional speech act functions. Finally, the implications of Two-level Word Class Categorization Model for POS tagging in Modern Chinese corpora are explored.

Proceedings of the Conference on Natural Language Processing 2010

This book contains state-of-the-art contributions to the 10th conference on Natural Language Processing, KONVENS 2010 (Konferenz zur Verarbeitung natürlicher Sprache), with a focus on semantic processing. The KONVENS in general aims at offering a broad perspective on current research and developments within the interdisciplinary field of natural language processing. The central theme draws specific attention towards addressing linguistic aspects ofmeaning, covering deep as well as shallow approaches to semantic processing. The contributions address both knowledgebased and data-driven methods for modelling and acquiring semantic information, and discuss the role of semantic information in applications of language technology. The articles demonstrate the importance of semantic processing, and present novel and creative approaches to natural language processing in general. Some contributions put their focus on developing and improving NLP systems for tasks like Named Entity Recognition or Word Sense Disambiguation, or focus on semantic knowledge acquisition and exploitation with respect to collaboratively built ressources, or harvesting semantic information in virtual games. Others are set within the context of real-world applications, such as Authoring Aids, Text Summarisation and Information Retrieval. The collection highlights the importance of semantic processing for different areas and applications in Natural Language Processing, and provides the reader with an overview of current research in this field

Knowledge Expansion of a Statistical Machine Translation System using Morphological Resources

Translation capability of a Phrase-Based Statistical Machine Translation (PBSMT) system mostly depends on parallel data and phrases that are not present in the training data are not correctly translated. This paper describes a method that efficiently expands the existing knowledge of a PBSMT system without adding more parallel data but using external morphological resources. A set of new phrase associations is added to translation and reordering models; each of them corresponds to a morphological variation of the source/target/both phrases of an existing association. New associations are generated using a string similarity score based on morphosyntactic information. We tested our approach on En-Fr and Fr-En translations and results showed improvements of the performance in terms of automatic scores (BLEU and Meteor) and reduction of out-of-vocabulary (OOV) words. We believe that our knowledge expansion framework is generic and could be used to add different types of information to the model.JRC.G.2-Global security and crisis managemen

A design proposal of an online corpus-driven dictionary of Portuguese for University Students

University students are expected to read and write academic texts as part of typical literacy practices in higher education settings. Hyland (2009, p. viii-ix) states that meeting these literacy demands involves “learning to use language in new ways”. In order to support the mastery of written academic Portuguese, the primary aim of this PhD research was to propose a design of an online corpus-driven dictionary of Portuguese for university students (DOPU) attending Portuguese-medium institutions, speakers of Brazilian Portuguese (BP) and European Portuguese (EP), either as a mother tongue or as an additional language. The semi-automated approach to dictionary-making (Gantar et al., 2016), which is the latest method for dictionary compilation and had never been employed for Portuguese, was tested as a means of provision of lexical content that would serve as a basis for compiling entries of DOPU. It consists of automatic extraction of data from the corpus and import into dictionary writing system, where lexicographers then analyse, validate and edit the information. Thus, evaluation of this method for designing DOPU was a secondary goal of this research. The procedure was performed on the Sketch Engine (Kilgarriff et al., 2004) corpus tool and the dictionary writing system used was iLex (Erlandsen, 2010). A number of new resources and tools were created especially for the extraction, given the unsuitability of the existing ones. These were: a 40 million-word corpus of academic texts (CoPEP), balanced between BP and EP and covering six areas of knowledge, a sketch grammar, and GDEX configurations for academic Portuguese. Evaluation of the adoption of the semi-automated approach in the context of the DOPU design indicated that although further development of these brand-new resources and tools, as well as the procedure itself, would greatly contribute to increasing the quality of DOPU’s lexical content, the extracted data can already be used as a basis for entry writing. The positive results of the experiment also suggest that this approach should be highly beneficial to other lexicographic projects of Portuguese as well.No ensino superior, espera-se que estudantes participem, em maior ou menor extensão, em atividades de leitura e escrita de textos que tipicamente circulam no contexto universitário, como artigos, livros, exames, ensaios, monografias, projetos, trabalhos de conclusão de curso, dissertações, teses, entre outros. Contudo, essas práticas costumam se apresentar como verdadeiros desafios aos alunos, que não estão familiarizados com esses novos gêneros discursivos. Conforme Hyland (2009, p. viii-ix), a condição para se ter sucesso nessas práticas é “aprender a usar a língua de novas maneiras”. A linguagem acadêmica é objeto de pesquisa há muitos anos, sendo especialmente desenvolvida no âmbito da língua inglesa. Se por um lado, durante um longo período todas as atenções estavam voltadas para o English for Academic Purposes (EAP) (inglês para fins acadêmicos), tendo em vista o incomparável apelo comercial dessa área, mais recentemente tem-se entendido que falantes de inglês como língua materna também precisam aprender inglês acadêmico, pois, como dito acima, trata-se de uma nova maneira de usar a língua, que os estudantes universitários desconhecem. Nesse sentido, é natural que a grande maioria de matérias pedagógicos como livros, manuais, gramáticas, listas de palavras e dicionários, por exemplo, sejam produzidos para o contexto de uso da língua inglesa. Assim como o inglês e tantas outras línguas, o português também é usado em universidades como língua na e pela qual se constrói conhecimento. Aliás, nos últimos 15 anos, temos vivenciado um fenômeno de expansão do acesso ao ensino universitário no Brasil, paralelamente a um grande aumento da presença de alunos estrangeiros fazendo ensino superior no Brasil e em Portugal, o que reforça a natureza do português como língua de construção e difusão científica. É de se saudar os esforços e as medidas de política linguística da Comunidade dos Países de Língua Portuguesa (CPLP) para apoiar e fomentar o português como língua da ciência. Apesar dessa clara importância do português acadêmico, sabemos que sua presença como objeto de estudo de uma área específica ainda é bastante restrita. Tem-se observado algum crescimento no que diz respeito à abordagem discursiva da linguagem acadêmica; contudo, descrições ao nível léxico-gramatical ainda são bastante escassas. Em especial, no que concerne recursos lexicográficos como auxiliares pedagógicos, a existência de um dicionário de português acadêmico especialmente criado para atender as necessidades de estudantes universitários é desconhecida. Nesse sentido, tendo em vista a demanda apresentada acima e a lacuna nos estudos atuais, a presente pesquisa de doutorado buscou colaborar tanto com o campo dos recursos ao ensino de português acadêmico quanto com o de elaboração de recursos lexicográficos através da proposta de desenho de um dicionário online corpus-driven de português para estudantes universitários (DOPU). Baseando-se em uma perspectiva de português como língua pluricêntrica, este dicionário contempla as variedades português brasileiro (PB) e europeu (PE). Além disso, o público-alvo se constitui por falantes de português como língua materna e como língua adicional. Para a construção do desenho, adotou-se a mais moderna abordagem de compilação de dicionários atualmente existente, qual seja, a semi-automated approach to dictionary-making (Gantar et al., 2016). Esse método consiste na extração automática de dados de um corpus e importação para um sistema de escrita de dicionários, no qual lexicógrafos analisam, editam e validam as informações que foram automaticamente pré-organizadas nos campos da entrada conforme definições previamente estabelecidas. Esta abordagem é revolucionária no sentido em que o ponto de partida da análise lexical do corpus não mais se dá na ferramenta de análise de corpus, mas sim diretamente no sistema de escrita de dicionários. Experimentar essa abordagem no desenvolvimento do desenho do DOPU constitui-se em um objetivo secundário desta pesquisa de doutorado, uma vez que tal método nunca foi aplicado para a construção de dicionários de português. Os programas utilizados para a aplicação do procedimento de extração foram o Sketch Engine (SkE) (Kilgarriff et al., 2004), provavelmente a mais sofisticada ferramenta de criação, análise e manutenção de corpus da atualidade, e o iLex (Erlandsen, 2010), um sistema de escrita de dicionários bastante flexível e com alta capacidade de processamento de dados. Para a implementação da abordagem, são necessários: um corpus anotado com classes de palavra; uma sketch grammar (trata-se de um arquivo com relações gramaticais e diretivas de processamento para o sistema do SkE computar diferentes tipos de relações através de cálculos estáticos); uma configuração de GDEX, isto é, Good Dictionary Examples – bons exemplos para dicionários (trata-se de uma configuração com classificadores para avaliar frases e atribuir pontuações conforme os critérios estabelecidos); e definições de parâmetros (frequência mínima dos colocados e das relações gramaticais). Tendo em vista a inadequação de corpora de português, bem como da sketch grammar e do GDEX existentes para o português, em função do propósito dessa extração de dados, qual seja, a compilação de entradas para o DOPU, foi necessário elaborar novos recursos. Foi compilado o Corpus de Português Escrito em Periódicos (CoPEP), com 40 milhões de palavras, equilibrado entre as variedades PB e PE, e que cobre seis áreas de conhecimento. Os metadados do corpus foram detalhadamente anotados, permitindo fazer pesquisas avançadas. É o primeiro corpus internacional de português acadêmico de que temos notícia. De forma a padronizar a análise lexical e diminuir desequilíbrios na contagem estatística, o CoPEP foi pós-processado com o conversor Lince de forma a atualizar as ortografias de cada variedade conforme a determinação do Acordo Ortográfico da Língua Portuguesa, de 1990. Uma sketch grammar foi especialmente elaborada para o CoPEP, e, nesse sentido, pode ser aplicada a outros corpora de português anotados pelo mesmo anotador. Optou-se por usar o anotador oferecido por padrão no SkE, qual seja, o Freeling v3. Criou-se uma sketch grammar com mais e mais precisas relações gramaticais do que aquela oferecida por padrão pelo SkE. Assim, usuários trabalhando com corpora de português anotados com Freeling no SkE poderão usar a minha versão, que já está disponível no Sketch Engine. Uma configuração de GDEX havia sido produzida para fornecer exemplos para a compilação do Oxford Portuguese Dicionary (2015). No entanto, por ser bastante geral, elaborada para um corpus Web e por buscar selecionar exemplos para um dicionário bilíngue português-inglês/inglês-português, julgou-se mais apropriado criar uma configuração completamente nova. Assim, desenvolvi tal recurso, tendo em vista as características de uso da língua como apresentadas no CoPEP e o perfil do usuário do DOPU. O procedimento de extração automática de dados do CoPEP e importação para o iLex tomou como base o procedimento usado para a criação de dicionários de esloveno (criadores desse método), fazendo-se adaptações. Acrescentaram-se dois elementos ao processo de extração: o longest-commonest match (LCM), que mostra a realização mais comum do par keyword e colocado, ajudando a entender o uso mais típico das colocações; e sugestões para atribuição de etiquetas com variedade típica, tanto para a keyword quanto para o colocado. A avaliação do processo de escrita de entradas-piloto indicou que o método de extração de dados do CoPEP e importação para o iLex foi extremamente positivo, dado que a análise lexical pôde ser bastante sofisticada sem demandar o tempo rotineiro necessário quando se parte das linhas de concordância para elaboração de entradas. Alguns dados que nesta pesquisa não foram extraídos automaticamente e que tiveram que ser analisados manualmente na ferramenta de corpus poderão ser incluídos numa próxima versão do procedimento. Análise do processo de criação dos recursos necessários indicou que aprimoramentos podem ser feitos, assim aumentando a acurácia da extração. Espera-se que o desenho de dicionário online corpus-driven de português para estudantes universitários proposto por esta pesquisa de doutorado sirva como base para o desenvolvimento de outras pesquisas relacionadas de forma que a sustentar a elaboração do DOPU

Phraseology in Corpus-based transaltion studies : stylistic study of two contempoarary Chinese translation of Cervantes's Don Quijote

The present work sets out to investigate the stylistic profiles of two modern Chinese versions of Cervantes???s Don Quijote (I): by Yang Jiang (1978), the first direct translation from Castilian to Chinese, and by Liu Jingsheng (1995), which is one of the most commercially successful versions of the Castilian literary classic. This thesis focuses on a detailed linguistic analysis carried out with the help of the latest textual analytical tools, natural language processing applications and statistical packages. The type of linguistic phenomenon singled out for study is four-character expressions (FCEXs), which are a very typical category of Chinese phraseology. The work opens with the creation of a descriptive framework for the annotation of linguistic data extracted from the parallel corpus of Don Quijote. Subsequently, the classified and extracted data are put through several statistical tests. The results of these tests prove to be very revealing regarding the different use of FCEXs in the two Chinese translations. The computational modelling of the linguistic data would seem to indicate that among other findings, while Liu???s use of archaic idioms has followed the general patterns of the original and also of Yang???s work in the first half of Don Quijote I, noticeable variations begin to emerge in the second half of Liu???s more recent version. Such an idiosyncratic use of archaisms by Liu, which may be defined as style shifting or style variation, is then analyzed in quantitative terms through the application of the proposed context-motivated theory (CMT). The results of applying the CMT-derived statistical models show that the detected stylistic variation may well point to the internal consistency of the translator in rendering the second half of Part I of the novel, which reflects his freer, more creative and experimental style of translation. Through the introduction and testing of quantitative research methods adapted from corpus linguistics and textual statistics, this thesis has made a major contribution to methodological innovation in the study of style within the context of corpus-based translation studies.Imperial Users onl

Phraseology in Corpus-Based Translation Studies: A Stylistic Study of Two Contemporary Chinese Translations of Cervantes's Don Quijote

The present work sets out to investigate the stylistic profiles of two modern Chinese versions of Cervantes’s Don Quijote (I): by Yang Jiang (1978), the first direct translation from Castilian to Chinese, and by Liu Jingsheng (1995), which is one of the most commercially successful versions of the Castilian literary classic. This thesis focuses on a detailed linguistic analysis carried out with the help of the latest textual analytical tools, natural language processing applications and statistical packages. The type of linguistic phenomenon singled out for study is four-character expressions (FCEXs), which are a very typical category of Chinese phraseology. The work opens with the creation of a descriptive framework for the annotation of linguistic data extracted from the parallel corpus of Don Quijote. Subsequently, the classified and extracted data are put through several statistical tests. The results of these tests prove to be very revealing regarding the different use of FCEXs in the two Chinese translations. The computational modelling of the linguistic data would seem to indicate that among other findings, while Liu’s use of archaic idioms has followed the general patterns of the original and also of Yang’s work in the first half of Don Quijote I, noticeable variations begin to emerge in the second half of Liu’s more recent version. Such an idiosyncratic use of archaisms by Liu, which may be defined as style shifting or style variation, is then analyzed in quantitative terms through the application of the proposed context-motivated theory (CMT). The results of applying the CMT-derived statistical models show that the detected stylistic variation may well point to the internal consistency of the translator in rendering the second half of Part I of the novel, which reflects his freer, more creative and experimental style of translation. Through the introduction and testing of quantitative research methods adapted from corpus linguistics and textual statistics, this thesis has made a major contribution to methodological innovation in the study of style within the context of corpus-based translation studies

A corpus-driven study of features of Chinese students' undergraduate writing in UK universities

Chinese people now comprise the ‘largest single overseas student group in the UK’ with more than 85,000 Chinese students registered at UK institutions in 2009 (British Council, 2010a). While there have been many studies carried out on short argumentative essays from this group (e.g. Chen, 2009), and on postgraduate theses (e.g. Hyland, 2008b), there has been comparatively little research conducted on the high-stakes genre of undergraduate assignments. This study examines assessed writing from Chinese and British undergraduates studying in UK universities between 2000 and 2008; these are investigated using corpus linguistic procedures, supported by qualitative reading. A particular focus is the use of lexical chunks, or recurring strings of words. Findings from the literature on Chinese students’ written English indicate high use of informal chunks, connecting chunks, and those containing first person pronouns (e.g. Milton, 1999). This study found that while the Chinese students make greater use of particular connectors and the first person plural, both student groups make (limited) use of informal language. These areas of difference are more apparent in year 1/2 assignments than those from year 3, suggesting that students gradually conform to the academy’s expectations. Unexpected findings which have not been previously identified in the literature include Chinese students’ significantly higher use of tables, figures (or ‘visuals’) and lists, compared to the British students’ writing. Detailed exploration of writing within Biology, Economics and Engineering suggests that using visuals and lists are different, yet equally acceptable, ways of writing assignments. Since the writing of both student groups has been judged by discipline specialists to be of a high standard, it is argued that the difference in use of visuals and lists illustrates the range of acceptability at undergraduate level. The thesis proposes that scholars therefore need to consider expanding the notion of what constitutes ‘good’ student writing

Creating large semantic lexical resources for the Finnish language

Finnish belongs into the Finno-Ugric language family, and it is spoken by the vast majority of the people living in Finland. The motivation for this thesis is to contribute to the development of a semantic tagger for Finnish. This tool is a parallel of the English Semantic Tagger which has been developed at the University Centre for Computer Corpus Research on Language (UCREL) at Lancaster University since the beginning of the 1990s and which has over the years proven to be a very powerful tool in automatic semantic analysis of English spoken and written data. The English Semantic Tagger has various successful applications in the fields of natural language processing and corpus linguistics, and new application areas emerge all the time. The semantic lexical resources that I have created in this thesis provide the knowledge base for the Finnish Semantic Tagger. My main contributions are the lexical resources themselves, along with a set of methods and guidelines for their creation and expansion as a general language resource and as tailored for domain-specific applications. Furthermore, I propose and carry out several methods for evaluating semantic lexical resources. In addition to the English Semantic Tagger, which was developed first, and the Finnish Semantic Tagger second, equivalent semantic taggers have now been developed for Czech, Chinese, Dutch, French, Italian, Malay, Portuguese, Russian, Spanish, Urdu, and Welsh. All these semantic taggers taken together form a program framework called the UCREL Semantic Analysis System (USAS) which enables the development of not only monolingual but also various types of multilingual applications. Large-scale semantic lexical resources designed for Finnish using semantic fields as the organizing principle have not been attempted previously. Thus, the Finnish semantic lexicons created in this thesis are a unique and novel resource. The lexical coverage on the test corpora containing general modern standard Finnish, which has been the focus of the lexicon development, ranges from 94.58% to 97.91%. However, the results are also very promising in the analysis of domain-specific text (95.36%), older Finnish text (92.11–93.05%), and Internet discussions (91.97–94.14%). The results of the evaluation of lexical coverage are comparable to the results obtained with the English equivalents and thus indicate that the Finnish semantic lexical resources indeed cover the majority of core Finnish vocabulary

A CORPUS-BASED CONTRASTIVE ANALYSIS OF SADNESS EXPRESSIONS IN ENGLISH AND CHINESE

Ph.DDOCTOR OF PHILOSOPH