3,573 research outputs found
Fully Learnable Front-End for Multi-Channel Acoustic Modeling using Semi-Supervised Learning
In this work, we investigated the teacher-student training paradigm to train
a fully learnable multi-channel acoustic model for far-field automatic speech
recognition (ASR). Using a large offline teacher model trained on beamformed
audio, we trained a simpler multi-channel student acoustic model used in the
speech recognition system. For the student, both multi-channel feature
extraction layers and the higher classification layers were jointly trained
using the logits from the teacher model. In our experiments, compared to a
baseline model trained on about 600 hours of transcribed data, a relative
word-error rate (WER) reduction of about 27.3% was achieved when using an
additional 1800 hours of untranscribed data. We also investigated the benefit
of pre-training the multi-channel front end to output the beamformed log-mel
filter bank energies (LFBE) using L2 loss. We find that pre-training improves
the word error rate by 10.7% when compared to a multi-channel model directly
initialized with a beamformer and mel-filter bank coefficients for the front
end. Finally, combining pre-training and teacher-student training produces a
WER reduction of 31% compared to our baseline.Comment: To appear in ICASSP 202
Conditional Teacher-Student Learning
The teacher-student (T/S) learning has been shown to be effective for a
variety of problems such as domain adaptation and model compression. One
shortcoming of the T/S learning is that a teacher model, not always perfect,
sporadically produces wrong guidance in form of posterior probabilities that
misleads the student model towards a suboptimal performance. To overcome this
problem, we propose a conditional T/S learning scheme, in which a "smart"
student model selectively chooses to learn from either the teacher model or the
ground truth labels conditioned on whether the teacher can correctly predict
the ground truth. Unlike a naive linear combination of the two knowledge
sources, the conditional learning is exclusively engaged with the teacher model
when the teacher model's prediction is correct, and otherwise backs off to the
ground truth. Thus, the student model is able to learn effectively from the
teacher and even potentially surpass the teacher. We examine the proposed
learning scheme on two tasks: domain adaptation on CHiME-3 dataset and speaker
adaptation on Microsoft short message dictation dataset. The proposed method
achieves 9.8% and 12.8% relative word error rate reductions, respectively, over
T/S learning for environment adaptation and speaker-independent model for
speaker adaptation.Comment: 5 pages, 1 figure, ICASSP 201
Learning to Behave Like Clean Speech: Dual-Branch Knowledge Distillation for Noise-Robust Fake Audio Detection
Most research in fake audio detection (FAD) focuses on improving performance
on standard noise-free datasets. However, in actual situations, there is
usually noise interference, which will cause significant performance
degradation in FAD systems. To improve the noise robustness, we propose a
dual-branch knowledge distillation fake audio detection (DKDFAD) method.
Specifically, a parallel data flow of the clean teacher branch and the noisy
student branch is designed, and interactive fusion and response-based
teacher-student paradigms are proposed to guide the training of noisy data from
the data distribution and decision-making perspectives. In the noise branch,
speech enhancement is first introduced for denoising, which reduces the
interference of strong noise. The proposed interactive fusion combines
denoising features and noise features to reduce the impact of speech distortion
and seek consistency with the data distribution of clean branch. The
teacher-student paradigm maps the student's decision space to the teacher's
decision space, making noisy speech behave as clean. In addition, a joint
training method is used to optimize the two branches to achieve global
optimality. Experimental results based on multiple datasets show that the
proposed method performs well in noisy environments and maintains performance
in cross-dataset experiments
- …