46,319 research outputs found

    SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

    Full text link
    Transfer learning has fundamentally changed the landscape of natural language processing (NLP) research. Many existing state-of-the-art models are first pre-trained on a large text corpus and then fine-tuned on downstream tasks. However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and forget the knowledge of the pre-trained model. To address the above issue in a more principled manner, we propose a new computational framework for robust and efficient fine-tuning for pre-trained language models. Specifically, our proposed framework contains two important ingredients: 1. Smoothness-inducing regularization, which effectively manages the capacity of the model; 2. Bregman proximal point optimization, which is a class of trust-region methods and can prevent knowledge forgetting. Our experiments demonstrate that our proposed method achieves the state-of-the-art performance on multiple NLP benchmarks.Comment: The 58th annual meeting of the Association for Computational Linguistics (ACL 2020

    Introduction (to Special Issue on Tibetan Natural Language Processing)

    Get PDF
    This introduction surveys research on Tibetan NLP, both in China and in the West, as well as contextualizing the articles contained in the special issue

    Neuro-linguistic-programming: a critical review of NLP research and the application of NLP in coaching

    Get PDF
    The huge popularity of Neuro-Linguistic Programming (NLP) over the past three decades has in some ways mirrored the growth in coaching psychology. This paper is part of a series of four papers in a special issue within ICPR that aims to explore NLP coaching from diverse perspectives, offering personal insights or reviews of evidence. As part of this process a pair of authors were invited to advance the case for and the case against NLP. This paper aims to adopt a critical stance; reviewing the concept of NLP, exploring the claims made by advocates and critically reviewing the evidence from a psychological perspective. In undertaking this review we completed a series of literature searches using a range of discovery tools to identify research papers, based on pre-determined search criteria. This review led us to the conclusion that unique NLP practices are poorly supported by research evidence
    • …
    corecore