1,022 research outputs found
Zero-shot language transfer for cross-lingual sentence retrieval using bidirectional attention model
We present a neural architecture for cross-lingual mate sentence retrieval which encodes sentences in a joint multilingual space and learns to distinguish true translation pairs from semantically related sentences across languages. The proposed model combines a recurrent sequence encoder with a bidirectional attention layer and an intra-sentence attention mechanism. This way the final fixed-size sentence representations in each training sentence pair depend on the selection of contextualized token representations from the other sentence. The representations of both sentences are then combined using the bilinear product function to predict the relevance score. We show that, coupled with a shared
multilingual word embedding space, the proposed model strongly outperforms unsupervised cross-lingual ranking functions, and that further boosts can be achieved by combining the two approaches. Most importantly, we demonstrate the model's effectiveness in zero-shot language transfer settings: our multilingual framework boosts cross-lingual sentence retrieval performance for unseen language pairs without any training examples. This enables robust cross-lingual sentence retrieval
also for pairs of resource-lean languages, without any parallel data
ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation
Recent cross-lingual cross-modal works attempt to extend Vision-Language
Pre-training (VLP) models to non-English inputs and achieve impressive
performance. However, these models focus only on understanding tasks utilizing
encoder-only architecture. In this paper, we propose ERNIE-UniX2, a unified
cross-lingual cross-modal pre-training framework for both generation and
understanding tasks. ERNIE-UniX2 integrates multiple pre-training paradigms
(e.g., contrastive learning and language modeling) based on encoder-decoder
architecture and attempts to learn a better joint representation across
languages and modalities. Furthermore, ERNIE-UniX2 can be seamlessly fine-tuned
for varieties of generation and understanding downstream tasks. Pre-trained on
both multilingual text-only and image-text datasets, ERNIE-UniX2 achieves SOTA
results on various cross-lingual cross-modal generation and understanding tasks
such as multimodal machine translation and multilingual visual question
answering.Comment: 13 pages, 2 figure
Crosslingual Retrieval Augmented In-context Learning for Bangla
The promise of Large Language Models (LLMs) in Natural Language Processing
has often been overshadowed by their limited performance in low-resource
languages such as Bangla. To address this, our paper presents a pioneering
approach that utilizes cross-lingual retrieval augmented in-context learning.
By strategically sourcing semantically similar prompts from high-resource
language, we enable multilingual pretrained language models (MPLMs), especially
the generative model BLOOMZ, to successfully boost performance on Bangla tasks.
Our extensive evaluation highlights that the cross-lingual retrieval augmented
prompts bring steady improvements to MPLMs over the zero-shot performance.Comment: In The 1st Bangla Language Processing (BLP) Workshop, held in
conjunction with The Conference on Empirical Methods in Natural Language
Processing (EMNLP), December 202
- …