7,755 research outputs found
Comparison of Speech Representations for Automatic Quality Estimation in Multi-Speaker Text-to-Speech Synthesis
We aim to characterize how different speakers contribute to the perceived
output quality of multi-speaker Text-to-Speech (TTS) synthesis. We
automatically rate the quality of TTS using a neural network (NN) trained on
human mean opinion score (MOS) ratings. First, we train and evaluate our NN
model on 13 different TTS and voice conversion (VC) systems from the ASVSpoof
2019 Logical Access (LA) Dataset. Since it is not known how best to represent
speech for this task, we compare 8 different representations alongside MOSNet
frame-based features. Our representations include image-based spectrogram
features and x-vector embeddings that explicitly model different types of noise
such as T60 reverberation time. Our NN predicts MOS with a high correlation to
human judgments. We report prediction correlation and error. A key finding is
the quality achieved for certain speakers seems consistent, regardless of the
TTS or VC system. It is widely accepted that some speakers give higher quality
than others for building a TTS system: our method provides an automatic way to
identify such speakers. Finally, to see if our quality prediction models
generalize, we predict quality scores for synthetic speech using a separate
multi-speaker TTS system that was trained on LibriTTS data, and conduct our own
MOS listening test to compare human ratings with our NN predictions.Comment: accepted at Speaker Odyssey 202
Deep Learning for Audio Signal Processing
Given the recent surge in developments of deep learning, this article
provides a review of the state-of-the-art deep learning techniques for audio
signal processing. Speech, music, and environmental sound processing are
considered side-by-side, in order to point out similarities and differences
between the domains, highlighting general methods, problems, key references,
and potential for cross-fertilization between areas. The dominant feature
representations (in particular, log-mel spectra and raw waveform) and deep
learning models are reviewed, including convolutional neural networks, variants
of the long short-term memory architecture, as well as more audio-specific
neural network models. Subsequently, prominent deep learning application areas
are covered, i.e. audio recognition (automatic speech recognition, music
information retrieval, environmental sound detection, localization and
tracking) and synthesis and transformation (source separation, audio
enhancement, generative models for speech, sound, and music synthesis).
Finally, key issues and future questions regarding deep learning applied to
audio signal processing are identified.Comment: 15 pages, 2 pdf figure
Transfer Learning for Speech and Language Processing
Transfer learning is a vital technique that generalizes models trained for
one setting or task to other settings or tasks. For example in speech
recognition, an acoustic model trained for one language can be used to
recognize speech in another language, with little or no re-training data.
Transfer learning is closely related to multi-task learning (cross-lingual vs.
multilingual), and is traditionally studied in the name of `model adaptation'.
Recent advance in deep learning shows that transfer learning becomes much
easier and more effective with high-level abstract features learned by deep
models, and the `transfer' can be conducted not only between data distributions
and data types, but also between model structures (e.g., shallow nets and deep
nets) or even model types (e.g., Bayesian models and neural models). This
review paper summarizes some recent prominent research towards this direction,
particularly for speech and language processing. We also report some results
from our group and highlight the potential of this very interesting research
field.Comment: 13 pages, APSIPA 201
- …