317 research outputs found
Reward Gaming in Conditional Text Generation
To align conditional text generation model outputs with desired behaviors,
there has been an increasing focus on training the model using reinforcement
learning (RL) with reward functions learned from human annotations. Under this
framework, we identify three common cases where high rewards are incorrectly
assigned to undesirable patterns: noise-induced spurious correlation, naturally
occurring spurious correlation, and covariate shift. We show that even though
learned metrics achieve high performance on the distribution of the data used
to train the reward function, the undesirable patterns may be amplified during
RL training of the text generation model. While there has been discussion about
reward gaming in the RL or safety community, in this discussion piece, we would
like to highlight reward gaming in the natural language generation (NLG)
community using concrete conditional text generation examples and discuss
potential fixes and areas for future work
How Good Are Large Language Models at Out-of-Distribution Detection?
Out-of-distribution (OOD) detection plays a vital role in enhancing the
reliability of machine learning (ML) models. The emergence of large language
models (LLMs) has catalyzed a paradigm shift within the ML community,
showcasing their exceptional capabilities across diverse natural language
processing tasks. While existing research has probed OOD detection with smaller
encoder-based Transformers like BERT and RoBERTa, the stark differences in
scales, pre-training objectives, and inference paradigms call into question the
applicability of these findings to LLMs. This paper embarks on a pioneering
empirical investigation of OOD detection in the domain of LLMs, focusing on
LLaMA series ranging from 7B to 65B in size. We thoroughly evaluate
commonly-used OOD detectors, scrutinizing their performance in both zero-grad
and fine-tuning scenarios. Notably, we alter previous discriminative
in-distribution fine-tuning into generative fine-tuning, aligning the
pre-training objective of LLMs with downstream tasks. Our findings unveil that
a simple cosine distance OOD detector demonstrates superior efficacy,
outperforming other OOD detectors. We provide an intriguing explanation for
this phenomenon by highlighting the isotropic nature of the embedding spaces of
LLMs, which distinctly contrasts with the anisotropic property observed in
smaller BERT family models. The new insight enhances our understanding of how
LLMs detect OOD data, thereby enhancing their adaptability and reliability in
dynamic environments.Comment: Work in progres
One Wide Feedforward is All You Need
The Transformer architecture has two main non-embedding components: Attention
and the Feed Forward Network (FFN). Attention captures interdependencies
between words regardless of their position, while the FFN non-linearly
transforms each input token independently. In this work we explore the role of
the FFN, and find that despite taking up a significant fraction of the model's
parameters, it is highly redundant. Concretely, we are able to substantially
reduce the number of parameters with only a modest drop in accuracy by removing
the FFN on the decoder layers and sharing a single FFN across the encoder.
Finally we scale this architecture back to its original size by increasing the
hidden dimension of the shared FFN, achieving substantial gains in both
accuracy and latency with respect to the original Transformer Big
Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation
Zero-shot translation (ZST), which is generally based on a multilingual
neural machine translation model, aims to translate between unseen language
pairs in training data. The common practice to guide the zero-shot language
mapping during inference is to deliberately insert the source and target
language IDs, e.g., for English and for German. Recent studies have
shown that language IDs sometimes fail to navigate the ZST task, making them
suffer from the off-target problem (non-target language words exist in the
generated translation) and, therefore, difficult to apply the current
multilingual translation model to a broad range of zero-shot language
scenarios. To understand when and why the navigation capabilities of language
IDs are weakened, we compare two extreme decoder input cases in the ZST
directions: Off-Target (OFF) and On-Target (ON) cases. By contrastively
visualizing the contextual word representations (CWRs) of these cases with
teacher forcing, we show that 1) the CWRs of different languages are
effectively distributed in separate regions when the sentence and ID are
matched (ON setting), and 2) if the sentence and ID are unmatched (OFF
setting), the CWRs of different languages are chaotically distributed. Our
analyses suggest that although they work well in ideal ON settings, language
IDs become fragile and lose their navigation ability when faced with off-target
tokens, which commonly exist during inference but are rare in training
scenarios. In response, we employ unlikelihood tuning on the negative (OFF)
samples to minimize their probability such that the language IDs can
discriminate between the on- and off-target tokens during training. Experiments
spanning 40 ZST directions show that our method reduces the off-target ratio by
-48.0% on average, leading to a +9.1 BLEU improvement with only an extra +0.3%
tuning cost
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation
Reliable automatic evaluation of summarization systems is challenging due to
the multifaceted and subjective nature of the task. This is especially the case
for languages other than English, where human evaluations are scarce. In this
work, we introduce SEAHORSE, a dataset for multilingual, multifaceted
summarization evaluation. SEAHORSE consists of 96K summaries with human ratings
along 6 dimensions of text quality: comprehensibility, repetition, grammar,
attribution, main ideas, and conciseness, covering 6 languages, 9 systems and 4
datasets. As a result of its size and scope, SEAHORSE can serve both as a
benchmark to evaluate learnt metrics, as well as a large-scale resource for
training such metrics. We show that metrics trained with SEAHORSE achieve
strong performance on the out-of-domain meta-evaluation benchmarks TRUE
(Honovich et al., 2022) and mFACE (Aharoni et al., 2022). We make the SEAHORSE
dataset and metrics publicly available for future research on multilingual and
multifaceted summarization evaluation
Exploring Enhanced Code-Switched Noising for Pretraining in Neural Machine Translation
Multilingual pretraining approaches in Neural Machine Translation (NMT) have shown that training models to denoise synthetic code-switched data can yield impressive performance gains --- owing to better multilingual semantic representations and transfer learning. However, they generated the synthetic code-switched data using non-contextual, one-to-one word translations obtained from lexicons - which can lead to significant noise in a variety of cases, including the poor handling of polysemes and multi-word expressions, violation of linguistic agreement and inability to scale to agglutinative languages. To overcome these limitations, we propose an approach called Contextual Code-Switching (CCS), where contextual, many-to-many word translations are generated using a `base' NMT model. We conduct experiments on 3 different language families - Romance, Uralic, and Indo-Aryan - and show significant improvements (by up to 5.5 spBLEU points) over the previous lexicon-based SOTA approaches. We also observe that small CCS models can perform comparably or better than massive models like mBART50 and mRASP2, depending on the size of data provided. We empirically analyse several key factors responsible for these - including context, many-to-many substitutions, code-switching language count etc. - and prove that they all contribute to enhanced pretraining of multilingual NMT models
Machine Learning and Natural Language Processing in Stock Prediction
In this thesis, we first study the two ill-posed natural language processing tasks related to stock prediction, i.e. stock movement prediction and financial document-level event extraction. While implementing stock prediction and event extraction, we encountered difficulties that could be resolved by utilizing out-of-distribution detection. Consequently, we presented a new approach for out-of-distribution detection, which is the third focus of this thesis. First, we systematically build a platform to study the NLP-aided stock auto-trading algorithms. Our platform is characterized by three features: (1) We provide financial news for each specific stock. (2) We provide various stock factors for each stock. (3) We evaluate performance from more financial-relevant metrics. Such a design allows us to develop and evaluate NLP-aided stock auto-trading algorithms in a more realistic setting. We also propose a system to automatically learn a good feature representation from various input information. The key to our algorithm is a method called semantic role labelling Pooling (SRLP), which leverages Semantic Role Labeling (SRL) to create a compact representation of each news paragraph. Based on SRLP, we further incorporate other stock factors to make the stock movement prediction. In addition, we propose a self-supervised learning strategy based on SRLP to enhance the out-of-distribution generalization performance of our system. Through our experimental study, we show that the proposed method achieves better performance and outperforms all strong baselines’ annualized rate of return as well as the maximum drawdown in back-testing. Second, we propose a generative solution for document-level event extraction that takes into account recent developments in generative event extraction, which have been successful at the sentence level but have not yet been explored for document-level extraction. Our proposed solution includes an encoding scheme to capture entity-to-document level information and a decoding scheme that takes into account all relevant contexts. Extensive experimental results demonstrate that our generative-based solution can perform as well as state-of-theart methods that use specialized structures for document event extraction. This allows our method to serve as an easy-to-use and strong baseline for future research in this area. Finally, we propose a new unsupervised OOD detection model that separates, extracts, and learns the semantic role labelling guided fine-grained local feature representation from different sentence arguments and the full sentence using a margin-based contrastive loss. Then we demonstrate the benefit of applying a self-supervised approach to enhance such global-local feature learning by predicting the SRL extracted role. We conduct our experiments and achieve state-of-the-art performance on out-of-distribution benchmarks.Thesis (Ph.D.) -- University of Adelaide, School of Computer and Mathematical Sciences, 202
Binary and Ternary Natural Language Generation
Ternary and binary neural networks enable multiplication-free computation and
promise multiple orders of magnitude efficiency gains over full-precision
networks if implemented on specialized hardware. However, since both the
parameter and the output space are highly discretized, such networks have
proven very difficult to optimize. The difficulties are compounded for the
class of transformer text generation models due to the sensitivity of the
attention operation to quantization and the noise-compounding effects of
autoregressive decoding in the high-cardinality output space. We approach the
problem with a mix of statistics-based quantization for the weights and elastic
quantization of the activations and demonstrate the first ternary and binary
transformer models on the downstream tasks of summarization and machine
translation. Our ternary BART base achieves an R1 score of 41 on the
CNN/DailyMail benchmark, which is merely 3.9 points behind the full model while
being 16x more efficient. Our binary model, while less accurate, achieves a
highly non-trivial score of 35.6. For machine translation, we achieved BLEU
scores of 21.7 and 17.6 on the WMT16 En-Ro benchmark, compared with a full
precision mBART model score of 26.8. We also compare our approach in the 8-bit
activation setting, where our ternary and even binary weight models can match
or outperform the best existing 8-bit weight models in the literature. Our code
and models are available at:
https://github.com/facebookresearch/Ternary_Binary_TransformerComment: ACL 2023 Ora
DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text
Large language models (LLMs) have notably enhanced the fluency and diversity
of machine-generated text. However, this progress also presents a significant
challenge in detecting the origin of a given text, and current research on
detection methods lags behind the rapid evolution of LLMs. Conventional
training-based methods have limitations in flexibility, particularly when
adapting to new domains, and they often lack explanatory power. To address this
gap, we propose a novel training-free detection strategy called Divergent
N-Gram Analysis (DNA-GPT). Given a text, we first truncate it in the middle and
then use only the preceding portion as input to the LLMs to regenerate the new
remaining parts. By analyzing the differences between the original and new
remaining parts through N-gram analysis in black-box or probability divergence
in white-box, we can clearly illustrate significant discrepancies between
machine-generated and human-written text. We conducted extensive experiments on
the most advanced LLMs from OpenAI, including text-davinci-003, GPT-3.5-turbo,
and GPT-4, as well as open-source models such as GPT-NeoX-20B and LLaMa-13B.
Results show that our zero-shot approach exhibits state-of-the-art performance
in distinguishing between human and GPT-generated text on four English and one
German dataset, outperforming OpenAI's own classifier, which is trained on
millions of text. Additionally, our methods provide reasonable explanations and
evidence to support our claim, which is a unique feature of explainable
detection. Our method is also robust under the revised text attack and can
additionally solve model sourcing. Codes are available at
https://github.com/Xianjun-Yang/DNA-GPT
- …