Integrating source-language context into log-linear models of statistical machine translation

The translation features typically used in state-of-the-art statistical machine translation (SMT) model dependencies between the source and target phrases, but not among the phrases in the source language themselves. A swathe of research has demonstrated that integrating source context modelling directly into log-linear phrase-based SMT (PB-SMT) and hierarchical PB-SMT (HPB-SMT), and can positively influence the weighting and selection of target phrases, and thus improve translation quality. In this thesis we present novel approaches to incorporate source-language contextual modelling into the state-of-the-art SMT models in order to enhance the quality of lexical selection. We investigate the effectiveness of use of a range of contextual features, including lexical features of neighbouring words, part-of-speech tags, supertags, sentence-similarity features, dependency information, and semantic roles. We explored a series of language pairs featuring typologically different languages, and examined the scalability of our research to larger amounts of training data. While our results are mixed across feature selections, language pairs, and learning curves, we observe that including contextual features of the source sentence in general produces improvements. The most significant improvements involve the integration of long-distance contextual features, such as dependency relations in combination with part-of-speech tags in Dutch-to-English subtitle translation, the combination of dependency parse and semantic role information in English-to-Dutch parliamentary debate translation, supertag features in English-to-Chinese translation, or combination of supertag and lexical features in English-to-Dutch subtitle translation. Furthermore, we investigate the applicability of our lexical contextual model in another closely related NLP problem, namely machine transliteration

Improving the role of language model in statistical machine translation (Indonesian-Javanese)

The statistical machine translation (SMT) is widely used by researchers and practitioners in recent years. SMT works with quality that is determined by several important factors, two of which are language and translation model. Research on improving the translation model has been done quite a lot, but the problem of optimizing the language model for use on machine translators has not received much attention. On translator machines, language models usually use trigram models as standard. In this paper, we conducted experiments with four strategies to analyze the role of the language model used in the Indonesian-Javanese translation machine and show improvement compared to the baseline system with the standard language model. The results of this research indicate that the use of 3-gram language models is highly recommended in SMT

Unification-based constraints for statistical machine translation

Morphology and syntax have both received attention in statistical machine translation research, but they are usually treated independently and the historical emphasis on translation into English has meant that many morphosyntactic issues remain under-researched. Languages with richer morphologies pose additional problems and conventional approaches tend to perform poorly when either source or target language has rich morphology. In both computational and theoretical linguistics, feature structures together with the associated operation of unification have proven a powerful tool for modelling many morphosyntactic aspects of natural language. In this thesis, we propose a framework that extends a state-of-the-art syntax-based model with a feature structure lexicon and unification-based constraints on the target-side of the synchronous grammar. Whilst our framework is language-independent, we focus on problems in the translation of English to German, a language pair that has a high degree of syntactic reordering and rich target-side morphology. We first apply our approach to modelling agreement and case government phenomena. We use the lexicon to link surface form words with grammatical feature values, such as case, gender, and number, and we use constraints to enforce feature value identity for the words in agreement and government relations. We demonstrate improvements in translation quality of up to 0.5 BLEU over a strong baseline model. We then examine verbal complex production, another aspect of translation that requires the coordination of linguistic features over multiple words, often with long-range discontinuities. We develop a feature structure representation of verbal complex types, using constraint failure as an indicator of translation error and use this to automatically identify and quantify errors that occur in our baseline system. A manual analysis and classification of errors informs an extended version of the model that incorporates information derived from a parse of the source. We identify clause spans and use model features to encourage the generation of complete verbal complex types. We are able to improve accuracy as measured using precision and recall against values extracted from the reference test sets. Our framework allows for the incorporation of rich linguistic information and we present sketches of further applications that could be explored in future work

The Roles of Language Models and Hierarchical Models in Neural Sequence-to-Sequence Prediction

With the advent of deep learning, research in many areas of machine learning is converging towards the same set of methods and models. For example, long short-term memory networks are not only popular for various tasks in natural language processing (NLP) such as speech recognition, machine translation, handwriting recognition, syntactic parsing, etc., but they are also applicable to seemingly unrelated fields such as robot control, time series prediction, and bioinformatics. Recent advances in contextual word embeddings like BERT boast with achieving state-of-the-art results on 11 NLP tasks with the same model. Before deep learning, a speech recognizer and a syntactic parser used to have little in common as systems were much more tailored towards the task at hand. At the core of this development is the tendency to view each task as yet another data mapping problem, neglecting the particular characteristics and (soft) requirements tasks often have in practice. This often goes along with a sharp break of deep learning methods with previous research in the specific area. This work can be understood as an antithesis to this paradigm. We show how traditional symbolic statistical machine translation models can still improve neural machine translation (NMT) while reducing the risk for common pathologies of NMT such as hallucinations and neologisms. Other external symbolic models such as spell checkers and morphology databases help neural grammatical error correction. We also focus on language models that often do not play a role in vanilla end-to-end approaches and apply them in different ways to word reordering, grammatical error correction, low-resource NMT, and document-level NMT. Finally, we demonstrate the benefit of hierarchical models in sequence-to-sequence prediction. Hand-engineered covering grammars are effective in preventing catastrophic errors in neural text normalization systems. Our operation sequence model for interpretable NMT represents translation as a series of actions that modify the translation state, and can also be seen as derivation in a formal grammar.EPSRC grant EP/L027623/1 EPSRC Tier-2 capital grant EP/P020259/

A Formal Model of Ambiguity and its Applications in Machine Translation

Systems that process natural language must cope with and resolve ambiguity. In this dissertation, a model of language processing is advocated in which multiple inputs and multiple analyses of inputs are considered concurrently and a single analysis is only a last resort. Compared to conventional models, this approach can be understood as replacing single-element inputs and outputs with weighted sets of inputs and outputs. Although processing components must deal with sets (rather than individual elements), constraints are imposed on the elements of these sets, and the representations from existing models may be reused. However, to deal efficiently with large (or infinite) sets, compact representations of sets that share structure between elements, such as weighted finite-state transducers and synchronous context-free grammars, are necessary. These representations and algorithms for manipulating them are discussed in depth in depth. To establish the effectiveness and tractability of the proposed processing model, it is applied to several problems in machine translation. Starting with spoken language translation, it is shown that translating a set of transcription hypotheses yields better translations compared to a baseline in which a single (1-best) transcription hypothesis is selected and then translated, independent of the translation model formalism used. More subtle forms of ambiguity that arise even in text-only translation (such as decisions conventionally made during system development about how to preprocess text) are then discussed, and it is shown that the ambiguity-preserving paradigm can be employed in these cases as well, again leading to improved translation quality. A model for supervised learning that learns from training data where sets (rather than single elements) of correct labels are provided for each training instance and use it to learn a model of compound word segmentation is also introduced, which is used as a preprocessing step in machine translation

Time and the quest for knowledge in the poetry of William Blake: a discussion of Tiriel, the Book of Urizen, the Song of Los and the Four Zoas

The physical appearances and specific behaviour of the characters in Tiriel , even the subtly ironical choice of names, suggest Blake's persistent opposition to the prevalent materialist-determinist philosophy of his day and to any form of dogmatism. This opposition accounts for the imaginative assimilation of originally unrelated literary material within a new symbolic context. Human misery does not originate from innate limitations or from a primordial fall from Divine Grace. It is caused by the immanent phenomenon of legalism in thought,, ethics and aesthetics. Physical, intellectual and emotional oppression deformation and corruption begin in childhood and are primarily perpetrated and perpetuated by repressive methods of education. Har and Tiriel are self-centred promulgators and, together with the other members of their family, warped products of Natural Law and Natural Religion. Tiriel's quest demonstrates that an increase in empirical knowledge is not necessarily accompanied by spiritual progress, nor does it improve the human condition. The complex vagueness of aspects of the poem contributes toward a more definite shaping of Blake's thought and symbolism in his later 'prophecies.' Portions of The Book of Urizen may be read as satire directed against the philosophic premises of seventeenth and eighteenth-century rationalism in general, and of Locke's theory of knowledge,. in particular. Theme,, structure and symbolism of the poem reflect this opposition and implicitly affirm Blake's own idealist metaphysics of reality. Abstracted from Eternity, Urizen's monolithic world has no extrinsic cause. It is a projection of his limited self-awareness. However, his solipsism fails to resolve the persistent contradiction between ideality and reality, thought and thing, subject and object. Los imposes temporal order and physical form on Urizen's disorganised thoughts. The limited anthropomorphic universe, produced by this intervention is a prison for mind and body, thought and desire. Henceforth, sensation and reflection determine the will to act. Man has rendered himself dependent on the fictitious 'substance' of matter, and on an equally mysterious remote deity. Both are only known by their 'accidents.' Natural science and Natural Religion are their respective rationalised form of worship. Both the pursuits of knowledge and of happiness require the suspension of desire. In The Song of Los Blake adopts a supra-historical perspective. Representative personages from biblical history, the history of religions generally,, philosophy and science are associated by their common failure to sustain their visionary powers. Blake incorporates into his poetic typology of decline,, structural elements derived from biblical, classical and modern conceptions of history without adopting their respective philosophical backgrounds. The notion of scientific progress and the advance of civilisation, concurrent with linear historical process, are dismissed. The achievements of empirical science, organised religion and autocratic government--synonymous with intellectual and physical oppression--kindle Orcls "thought creating fires." Despite its apocalyptic connotations, his violent outburst is of a highly ambivalent nature. The Four Zoas adumbrates the spiritual history of mankind. The poem is also a complex epic phenomenology of the human mind. Eden is an aspect of ideal reality where natural and human organisms are identified, and where life is sustained by loving self-sacrifice. After the Han's Fall elemental uproar reflects the mind's regression to the level of a perturbed oceanic consciousness which can no longer integrate the dissociated phenomena of the generative world into a living human form, thriving on love and understanding. Nature is transformed into a self-engendering monster. The human mind is englobed by the illusion of reality conceived as external and material, and by a fatalistic view of temporal process. Nevertheless, both misconceptions impose a degree of stability and order on the anarchic forces released by the cosmic catastrophe. Man's Fall is due to the dissociation of reason and affection. "Mental forms" are externalised and idolised. Eventually, under Urizen's control, imaginative energy in forced into rigid geometric form and regular motion. The beautiful illusion of the pseudo-Platonic "Mundane Shell" reflects the essential structure of Urizen's intelligence. however, it does not provide a lasting solution to the human dilemma. after the Fall. After the collapse of his creation, Urizen explores his alien environment by empirical means. he is a prisoner of his own restricted conception of reality. Unexpectedly, in Night VII(a), the Spectre of Urthona and Los are transformed into labourers of the Apocalypse. Regenoration starts with the annihilation of 'self.' Aware of his responsibilities, Los builds Golgonoozat the city of art. Emulating Christ's self-sacrifice, visionary activity is a form of self-denial. Time becomes a function of imaginative creativity. The imaginative world created by Los incorporates visionary time and space. Natural existence is realised as being endowed with regenerative qualities. Los no longer rejects Orc but sublimates his energies. Orc's destructive powers become an integral aspect Of the Last Judgment. Throughout Night VIII the providential and redemptive character of mortal life is stressed. Plunging into "the river of space" is a baptismal, if painful, experience. Although guided by Divine Providence, individual man has to work for his own salvation. In Night IX prophetic and apocalyptic views are fused as Los acts in a temporal context when tearing down the material, social and metaphysical barriers to vision erected by Urizen. The symbolism of Revelation is employed to adumbrate the artist's ultimate task in history. History is not beyond human control. Submission to the "Divine Vision" is an active ethical achievement capable of generating a powerful social dynamic, rather than tentatively removing it. Tyranny is overthrown because once the visionary poet has revealed its deceptions, mankind follows his example and removes it physically. This optimistic vision of the Last Judgment is an affirmation of the poet's absolute faith in the power of inspired vision to regenerate and humanize all aspects of life in this world

Writing as Material Practice: Substance, Surface and Medium

Writing as Material Practice grapples with the issue of writing as a form of material culture in its ancient and more recent manifestations, and in the contexts of production and consumption. Fifteen case studies explore the artefactual nature of writing — the ways in which materials, techniques, colour, scale, orientation and visibility inform the creation of inscribed objects and spaces, as well as structure subsequent engagement, perception and meaning making. Covering a temporal span of some 5000 years, from c.3200 BCE to the present day, and ranging in spatial context from the Americas to the Near East, the chapters in this volume bring a variety of perspectives which contribute to both specific and broader questions of writing materialities. The authors also aim to place past graphical systems in their social contexts so they can be understood in relation to the people who created and attributed meaning to writing and associated symbolic modes through a diverse array of individual and wider social practices

Kenyon College Catalog 2017-2018

https://digital.kenyon.edu/coursecatalogs/1208/thumbnail.jp