43 research outputs found
Improving Automatic Speech Recognition on Endangered Languages
As the world moves towards a more globalized scenario, it has brought along with it the extinction of several languages. It has been estimated that over the next century, over half of the world\u27s languages will be extinct, and an alarming 43% of the world\u27s languages are at different levels of endangerment or extinction already. The survival of many of these languages depends on the pressure imposed on the dwindling speakers of these languages. Often there is a strong correlation between endangered languages and the number and quality of recordings and documentations of each. But why do we care about preserving these less prevalent languages? The behavior of cultures is often expressed in the form of speech via one\u27s native language. The memories, ideas, major events, practices, cultures and lessons learnt, both individual as well as the community\u27s, are all communicated to the outside world via language. So, language preservation is crucial to understanding the behavior of these communities.
Deep learning models have been shown to dramatically improve speech recognition accuracy but require large amounts of labelled data. Unfortunately, resource constrained languages typically fall short of the necessary data for successful training. To help alleviate the problem, data augmentation techniques fabricate many new samples from each sample. The aim of this master\u27s thesis is to examine the effect of different augmentation techniques on speech recognition of resource constrained languages. The augmentation methods being experimented with are noise augmentation, pitch augmentation, speed augmentation as well as voice transformation augmentation using Generative Adversarial Networks (GANs). This thesis also examines the effectiveness of GANs in voice transformation and its limitations. The information gained from this study will further augment the collection of data, specifically, in understanding the conditions required for the data to be collected in, so that GANs can effectively perform voice transformation. Training of the original data on the Deep Speech model resulted in 95.03% WER. Training the Seneca data on a Deep Speech model that was pretrained on an English dataset, reduced the WER to 70.43%. On adding 15 augmented samples per sample, the WER reduced to 68.33%. Finally, adding 25 augmented samples per sample, the WER reduced to 48.23%. Experiments to find the best augmentation method among noise addition, pitch variation, speed variation augmentation and GAN augmentation revealed that GAN augmentation performed the best, with a WER reduction to 60.03%
UNSUPERVISED DOMAIN ADAPTATION FOR SPEAKER VERIFICATION IN THE WILD
Performance of automatic speaker verification (ASV) systems is very sensitive
to mismatch between training (source) and testing (target) domains. The
best way to address domain mismatch is to perform matched condition training
– gather sufficient labeled samples from the target domain and use them in
training. However, in many cases this is too expensive or impractical. Usually,
gaining access to unlabeled target domain data, e.g., from open source online
media, and labeled data from other domains is more feasible. This work focuses
on making ASV systems robust to uncontrolled (‘wild’) conditions, with
the help of some unlabeled data acquired from such conditions.
Given acoustic features from both domains, we propose learning a mapping
function – a deep convolutional neural network (CNN) with an encoder-decoder
architecture – between features of both the domains. We explore training the
network in two different scenarios: training on paired speech samples from
both domains and training on unpaired data. In the former case, where the
paired data is usually obtained via simulation, the CNN is treated as a nonii
ABSTRACT
linear regression function and is trained to minimize L2 loss between original
and predicted features from target domain. We provide empirical evidence that
this approach introduces distortions that affect verification performance. To
address this, we explore training the CNN using adversarial loss (along with
L2), which makes the predicted features indistinguishable from the original
ones, and thus, improve verification performance.
The above framework using simulated paired data, though effective, cannot
be used to train the network on unpaired data obtained by independently
sampling speech from both domains. In this case, we first train a CNN using
adversarial loss to map features from target to source. We, then, map the
predicted features back to the target domain using an auxiliary network, and
minimize a cycle-consistency loss between the original and reconstructed target
features.
Our unsupervised adaptation approach complements its supervised counterpart,
where adaptation is done using labeled data from both domains. We
focus on three domain mismatch scenarios: (1) sampling frequency mismatch
between the domains, (2) channel mismatch, and (3) robustness to far-field and
noisy speech acquired from wild conditions
CycleGANAS: Differentiable Neural Architecture Search for CycleGAN
We develop a Neural Architecture Search (NAS) framework for CycleGAN that
carries out unpaired image-to-image translation task. Extending previous NAS
techniques for Generative Adversarial Networks (GANs) to CycleGAN is not
straightforward due to the task difference and greater search space. We design
architectures that consist of a stack of simple ResNet-based cells and develop
a search method that effectively explore the large search space. We show that
our framework, called CycleGANAS, not only effectively discovers
high-performance architectures that either match or surpass the performance of
the original CycleGAN, but also successfully address the data imbalance by
individual architecture search for each translation direction. To our best
knowledge, it is the first NAS result for CycleGAN and shed light on NAS for
more complex structures
Super Denoise Net: Speech Super Resolution with Noise Cancellation in Low Sampling Rate Noisy Environments
Speech super-resolution (SSR) aims to predict a high resolution (HR) speech
signal from its low resolution (LR) corresponding part. Most neural SSR models
focus on producing the final result in a noise-free environment by recovering
the spectrogram of high-frequency part of the signal and concatenating it with
the original low-frequency part. Although these methods achieve high accuracy,
they become less effective when facing the real-world scenario, where
unavoidable noise is present. To address this problem, we propose a Super
Denoise Net (SDNet), a neural network for a joint task of super-resolution and
noise reduction from a low sampling rate signal. To that end, we design gated
convolution and lattice convolution blocks to enhance the repair capability and
capture information in the time-frequency axis, respectively. The experiments
show our method outperforms baseline speech denoising and SSR models on DNS
2020 no-reverb test set with higher objective and subjective scores
A Review of Deep Learning Techniques for Speech Processing
The field of speech processing has undergone a transformative shift with the
advent of deep learning. The use of multiple processing layers has enabled the
creation of models capable of extracting intricate features from speech data.
This development has paved the way for unparalleled advancements in speech
recognition, text-to-speech synthesis, automatic speech recognition, and
emotion recognition, propelling the performance of these tasks to unprecedented
heights. The power of deep learning techniques has opened up new avenues for
research and innovation in the field of speech processing, with far-reaching
implications for a range of industries and applications. This review paper
provides a comprehensive overview of the key deep learning models and their
applications in speech-processing tasks. We begin by tracing the evolution of
speech processing research, from early approaches, such as MFCC and HMM, to
more recent advances in deep learning architectures, such as CNNs, RNNs,
transformers, conformers, and diffusion models. We categorize the approaches
and compare their strengths and weaknesses for solving speech-processing tasks.
Furthermore, we extensively cover various speech-processing tasks, datasets,
and benchmarks used in the literature and describe how different deep-learning
networks have been utilized to tackle these tasks. Additionally, we discuss the
challenges and future directions of deep learning in speech processing,
including the need for more parameter-efficient, interpretable models and the
potential of deep learning for multimodal speech processing. By examining the
field's evolution, comparing and contrasting different approaches, and
highlighting future directions and challenges, we hope to inspire further
research in this exciting and rapidly advancing field
Disentanglement Learning for Text-Free Voice Conversion
Voice conversion (VC) aims to change the perceived speaker identity of a speech signal from one to another, while preserving the linguistic content. Recent state-of-the-art VC systems typically are dependent on automatic speech recognition (ASR) models and they have gained great successes. Results of recent challenges show these VC systems have reached a level of performance close to real human voices. However, they are highly relying on the performance of the ASR models, which might experience degradations in practical applications because of the mismatch between training and test data.
VC systems independent of ASR models are typically regarded as text-free systems. They commonly apply disentanglement learning methods to remove the speaker information of a speech signal, for example, vector quantisation (VQ) or instance normalisation (IN). However, text-free VC systems have not reached the same level of performance as text-dependent systems. This thesis mainly studies disentanglement learning methods for improving the performance of text-free VC systems. Three major contributions are summarised as follows.
Firstly, in order to improve the performance of an auto-encoder based VC model, the information loss issue caused by the VQ of the model is studied. Two disentanglement learning methods are exploited to replace the VQ of the model. Experiments show that these two methods improve the naturalness and intelligibility performance of the model, but hurt the speaker similarity performance of the model. The reason for the degradation of the speaker similarity performance is studied in the further analysis experiments.
Next, the performance and the robustness of Generative Adversarial Networks (GAN) based VC models are studied. In order to improve the performance and the robustness of an GAN based VC model, a new model is proposed. This new model introduces a new speaker adaptation layer for alleviating the information loss issue caused by a speaker adaptation method based on IN. Experiments show that the proposed model outperformed the baseline models on VC performance and robustness.
The third contribution studies whether Self-Supervised Learning (SSL) based VC models can reach the same level of performance of the state-of-the-art text-dependent models. An encoder-decoder framework is established for experiments. In this framework, the performance of a VC systems implemented with a SSL model can be compared to a VC system implemented with an ASR model. Experiment results show that SSL based VC models can reach the same level of naturalness performance of the state-of-the-art text- dependent VC models. Also, SSL based VC models gained advantages on intelligibility performance when tested on out of domain target speakers. But they performed worse on speaker similarity
Audio representations for deep learning in sound synthesis: A review
The rise of deep learning algorithms has led many researchers to withdraw from using classic signal processing methods for sound generation. Deep learning models have achieved expressive voice synthesis, realistic sound textures, and musical notes from virtual instruments. However, the most suitable deep learning architecture is still under investigation. The choice of architecture is tightly coupled to the audio representations. A sound’s original waveform can be too dense and rich for deep learning models to deal with efficiently - and complexity increases training time and computational cost. Also, it does not represent sound in the manner in which it is perceived. Therefore, in many cases, the raw audio has been transformed into a compressed and more meaningful form using upsampling, feature-extraction, or even by adopting a higher level illustration of the waveform. Furthermore, conditional on the form chosen, additional conditioning representations, different model architectures, and numerous metrics for evaluating the reconstructed sound have been investigated. This paper provides an overview of audio representations applied to sound synthesis using deep learning. Additionally, it presents the most significant methods for developing and evaluating a sound synthesis architecture using deep learning models, always depending on the audio representation