259 research outputs found
Deep Learning for Environmentally Robust Speech Recognition: An Overview of Recent Developments
Eliminating the negative effect of non-stationary environmental noise is a
long-standing research topic for automatic speech recognition that stills
remains an important challenge. Data-driven supervised approaches, including
ones based on deep neural networks, have recently emerged as potential
alternatives to traditional unsupervised approaches and with sufficient
training, can alleviate the shortcomings of the unsupervised methods in various
real-life acoustic environments. In this light, we review recently developed,
representative deep learning approaches for tackling non-stationary additive
and convolutional degradation of speech with the aim of providing guidelines
for those involved in the development of environmentally robust speech
recognition systems. We separately discuss single- and multi-channel techniques
developed for the front-end and back-end of speech recognition systems, as well
as joint front-end and back-end training frameworks
Switching Variational Auto-Encoders for Noise-Agnostic Audio-visual Speech Enhancement
Recently, audio-visual speech enhancement has been tackled in the
unsupervised settings based on variational auto-encoders (VAEs), where during
training only clean data is used to train a generative model for speech, which
at test time is combined with a noise model, e.g. nonnegative matrix
factorization (NMF), whose parameters are learned without supervision.
Consequently, the proposed model is agnostic to the noise type. When visual
data are clean, audio-visual VAE-based architectures usually outperform the
audio-only counterpart. The opposite happens when the visual data are corrupted
by clutter, e.g. the speaker not facing the camera. In this paper, we propose
to find the optimal combination of these two architectures through time. More
precisely, we introduce the use of a latent sequential variable with Markovian
dependencies to switch between different VAE architectures through time in an
unsupervised manner: leading to switching variational auto-encoder (SwVAE). We
propose a variational factorization to approximate the computationally
intractable posterior distribution. We also derive the corresponding
variational expectation-maximization algorithm to estimate the parameters of
the model and enhance the speech signal. Our experiments demonstrate the
promising performance of SwVAE.Comment: 2021 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP
Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation
We propose a novel deep learning model, which supports permutation invariant
training (PIT), for speaker independent multi-talker speech separation,
commonly known as the cocktail-party problem. Different from most of the prior
arts that treat speech separation as a multi-class regression problem and the
deep clustering technique that considers it a segmentation (or clustering)
problem, our model optimizes for the separation regression error, ignoring the
order of mixing sources. This strategy cleverly solves the long-lasting label
permutation problem that has prevented progress on deep learning based
techniques for speech separation. Experiments on the equal-energy mixing setup
of a Danish corpus confirms the effectiveness of PIT. We believe improvements
built upon PIT can eventually solve the cocktail-party problem and enable
real-world adoption of, e.g., automatic meeting transcription and multi-party
human-computer interaction, where overlapping speech is common.Comment: 5 page
- …