2,859 research outputs found

    Distributionally Robust Recurrent Decoders with Random Network Distillation

    Full text link
    Neural machine learning models can successfully model language that is similar to their training distribution, but they are highly susceptible to degradation under distribution shift, which occurs in many practical applications when processing out-of-domain (OOD) text. This has been attributed to “shortcut learning”":" relying on weak correlations over arbitrary large contexts. We propose a method based on OOD detection with Random Network Distillation to allow an autoregressive language model to automatically disregard OOD context during inference, smoothly transitioning towards a less expressive but more robust model as the data becomes more OOD, while retaining its full context capability when operating in-distribution. We apply our method to a GRU architecture, demonstrating improvements on multiple language modeling (LM) datasets

    Distributionally Robust Recurrent Decoders with Random Network Distillation

    Get PDF
    Neural machine learning models can successfully model language that is similar to their training distribution, but they are highly susceptible to degradation under distribution shift, which occurs in many practical applications when processing out-of-domain (OOD) text. This has been attributed to "shortcut learning": relying on weak correlations over arbitrary large contexts. We propose a method based on OOD detection with Random Network Distillation to allow an autoregressive language model to automatically disregard OOD context during inference, smoothly transitioning towards a less expressive but more robust model as the data becomes more OOD while retaining its full context capability when operating in-distribution. We apply our method to a GRU architecture, demonstrating improvements on multiple language modeling (LM) datasets.Comment: 8 pages, 1 figur

    Deep Architectures for Neural Machine Translation

    Get PDF
    It has been shown that increasing model depth improves the quality of neural machine translation. However, different architectural variants to increase model depth have been proposed, and so far, there has been no thorough comparative study. In this work, we describe and evaluate several existing approaches to introduce depth in neural machine translation. Additionally, we explore novel architectural variants, including deep transition RNNs, and we vary how attention is used in the deep decoder. We introduce a novel "BiDeep" RNN architecture that combines deep transition RNNs and stacked RNNs. Our evaluation is carried out on the English to German WMT news translation dataset, using a single-GPU machine for both training and inference. We find that several of our proposed architectures improve upon existing approaches in terms of speed and translation quality. We obtain best improvements with a BiDeep RNN of combined depth 8, obtaining an average improvement of 1.5 BLEU over a strong shallow baseline. We release our code for ease of adoption.Comment: WMT 2017 research trac

    Trends in social acceptance of renewable energy across Europe. A literature review

    Get PDF
    Social acceptance has proven to be a significant barrier in the implementation of renewable energy systems (hereinafter "RES"). While a general acceptance of RES is high, low local acceptance has hindered the development of renewable energy projects (hereinafter "REP"). This study assesses the determinants of local and general social acceptance of REP across Europe through a qualitative analysis from 25 case studies of the most significant social drivers and barriers that include all European countries. These case studies contain qualitative and quantitative analyses of the main factors for social acceptance of many representative groups including residents, stakeholders, and experts. Understanding the influences of social acceptance enables us to create strategies that will promote the development of REP by mitigating any public opposition

    Portafolio de negocios del sector empresarial e industrial del cantón Latacunga, provincia de Cotopaxi en el periodo 2011-2012

    Get PDF
    El presente proyecto de investigación se detalla la información más destacada y relevante de las empresas industriales del cantón Latacunga para el cual se realizo primeramente los siguientes procesos: En el primer capítulo se desarrolla la parte teórica del proyecto, en el cual se fundamenta los procesos que se realizara en los capítulos siguientes, el cual consta de 4 categorías fundamentales como Administración, Marketing, Marketing Estratégico y por ultimo Portafolio de Negocios, al cual se le dará mayor relevancia en su desarrollo teórico. En el segundo capítulo se desarrolla el estudio de mercado utilizando el método deductivo, y para lo cual se utilizara las diferentes técnicas de investigación como la observación, la entrevista para la recolección de la información, como muestra del proyecto se escogió a las empresas registradas en la Superintendencia de Compañías eligió a las más representativas del cantón. En el tercer capítulo se desarrolla en si el desarrollo de la propuesta del tema, el cual consta de 11 puntos, el cual se maneja para todas las empresas, numero al cual se realizo la investigación, pero en si se desarrolla con mayor énfasis a la parte de su cartera de negocios, así como sus clientes y proveedores

    The University of Edinburgh’s Neural MT Systems for WMT17

    Get PDF
    This paper describes the University of Edinburgh's submissions to the WMT17 shared news translation and biomedical translation tasks. We participated in 12 translation directions for news, translating between English and Czech, German, Latvian, Russian, Turkish and Chinese. For the biomedical task we submitted systems for English to Czech, German, Polish and Romanian. Our systems are neural machine translation systems trained with Nematus, an attentional encoder-decoder. We follow our setup from last year and build BPE-based models with parallel and back-translated monolingual training data. Novelties this year include the use of deep architectures, layer normalization, and more compact models due to weight tying and improvements in BPE segmentations. We perform extensive ablative experiments, reporting on the effectivenes of layer normalization, deep architectures, and different ensembling techniques.Comment: WMT 2017 shared task track; for Bibtex, see http://homepages.inf.ed.ac.uk/rsennric/bib.html#uedin-nmt:201

    The University of Edinburgh's English-German and English-Hausa Submissions to the WMT21 News Translation Task

    Get PDF
    This paper presents the University of Edinburgh's constrained submissions of English-German and English-Hausa systems to the WMT 2021 shared task on news translation. We build En-De systems in three stages: corpus filtering, back-translation, and fine-tuning. For En-Ha we use an iterative back-translation approach on top of pre-trained En-De models and investigate vocabulary embedding mapping
    corecore