611 research outputs found
Multilingual Acoustic Word Embedding Models for Processing Zero-Resource Languages
Acoustic word embeddings are fixed-dimensional representations of
variable-length speech segments. In settings where unlabelled speech is the
only available resource, such embeddings can be used in "zero-resource" speech
search, indexing and discovery systems. Here we propose to train a single
supervised embedding model on labelled data from multiple well-resourced
languages and then apply it to unseen zero-resource languages. For this
transfer learning approach, we consider two multilingual recurrent neural
network models: a discriminative classifier trained on the joint vocabularies
of all training languages, and a correspondence autoencoder trained to
reconstruct word pairs. We test these using a word discrimination task on six
target zero-resource languages. When trained on seven well-resourced languages,
both models perform similarly and outperform unsupervised models trained on the
zero-resource languages. With just a single training language, the second model
works better, but performance depends more on the particular training--testing
language pair.Comment: 5 pages, 4 figures, 1 table; accepted to ICASSP 2020. arXiv admin
note: text overlap with arXiv:1811.0040
Improved acoustic word embeddings for zero-resource languages using multilingual transfer
Acoustic word embeddings are fixed-dimensional representations of
variable-length speech segments. Such embeddings can form the basis for speech
search, indexing and discovery systems when conventional speech recognition is
not possible. In zero-resource settings where unlabelled speech is the only
available resource, we need a method that gives robust embeddings on an
arbitrary language. Here we explore multilingual transfer: we train a single
supervised embedding model on labelled data from multiple well-resourced
languages and then apply it to unseen zero-resource languages. We consider
three multilingual recurrent neural network (RNN) models: a classifier trained
on the joint vocabularies of all training languages; a Siamese RNN trained to
discriminate between same and different words from multiple languages; and a
correspondence autoencoder (CAE) RNN trained to reconstruct word pairs. In a
word discrimination task on six target languages, all of these models
outperform state-of-the-art unsupervised models trained on the zero-resource
languages themselves, giving relative improvements of more than 30% in average
precision. When using only a few training languages, the multilingual CAE
performs better, but with more training languages the other multilingual models
perform similarly. Using more training languages is generally beneficial, but
improvements are marginal on some languages. We present probing experiments
which show that the CAE encodes more phonetic, word duration, language identity
and speaker information than the other multilingual models.Comment: 11 pages, 7 figures, 8 tables. arXiv admin note: text overlap with
arXiv:2002.02109. Submitted to the IEEE Transactions on Audio, Speech and
Language Processin
Empirical evaluation of sequence-to-sequence models for word discovery in low-resource settings
Since Bahdanau et al. [1] first introduced attention for neural machine translation, most sequence-to-sequence models made use of attention mechanisms [2, 3, 4]. While they produce soft-alignment matrices that could be interpreted as alignment between target and source languages, we lack metrics to quantify their quality, being unclear which approach produces the best alignments. This paper presents an empirical evaluation of 3 of the main sequence-to-sequence models for word discovery from unsegmented phoneme sequences: CNN, RNN and Transformer-based. This task consists in aligning word sequences in a source language with phoneme sequences in a target language, inferring from it word segmentation on the target side [5]. Evaluating word segmentation quality can be seen as an extrinsic evaluation of the soft-alignment matrices produced during training. Our experiments in a low-resource scenario on Mboshi and English languages (both aligned to French) show that RNNs surprisingly outperform CNNs and Transformer for this task. Our results are confirmed by an intrinsic evaluation of alignment quality through the use Average Normalized Entropy (ANE). Lastly, we improve our best word discovery model by using an alignment entropy confidence measure that accumulates ANE over all the occurrences of a given alignment pair in the collection
Cognate-aware morphological segmentation for multilingual neural translation
This article describes the Aalto University entry to the WMT18 News
Translation Shared Task. We participate in the multilingual subtrack with a
system trained under the constrained condition to translate from English to
both Finnish and Estonian. The system is based on the Transformer model. We
focus on improving the consistency of morphological segmentation for words that
are similar orthographically, semantically, and distributionally; such words
include etymological cognates, loan words, and proper names. For this, we
introduce Cognate Morfessor, a multilingual variant of the Morfessor method. We
show that our approach improves the translation quality particularly for
Estonian, which has less resources for training the translation model.Comment: To appear in WMT1
Multilingual and Unsupervised Subword Modelingfor Zero-Resource Languages
Subword modeling for zero-resource languages aims to learn low-level
representations of speech audio without using transcriptions or other resources
from the target language (such as text corpora or pronunciation dictionaries).
A good representation should capture phonetic content and abstract away from
other types of variability, such as speaker differences and channel noise.
Previous work in this area has primarily focused unsupervised learning from
target language data only, and has been evaluated only intrinsically. Here we
directly compare multiple methods, including some that use only target language
speech data and some that use transcribed speech from other (non-target)
languages, and we evaluate using two intrinsic measures as well as on a
downstream unsupervised word segmentation and clustering task. We find that
combining two existing target-language-only methods yields better features than
either method alone. Nevertheless, even better results are obtained by
extracting target language bottleneck features using a model trained on other
languages. Cross-lingual training using just one other language is enough to
provide this benefit, but multilingual training helps even more. In addition to
these results, which hold across both intrinsic measures and the extrinsic
task, we discuss the qualitative differences between the different types of
learned features.Comment: 17 pages, 6 figures, 7 tables. Accepted for publication in Computer
Speech and Language. arXiv admin note: text overlap with arXiv:1803.0886
- âŠ