399 research outputs found
DNN-Based Source Enhancement to Increase Objective Sound Quality Assessment Score
We propose a training method for deep neural network (DNN)-based source enhancement to increase objective sound quality assessment (OSQA) scores such as the perceptual evaluation of speech quality (PESQ). In many conventional studies, DNNs have been used as a mapping function to estimate time-frequency masks and trained to minimize an analytically tractable objective function such as the mean squared error (MSE). Since OSQA scores have been used widely for soundquality evaluation, constructing DNNs to increase OSQA scores would be better than using the minimum-MSE to create highquality output signals. However, since most OSQA scores are not analytically tractable, i.e., they are black boxes, the gradient of the objective function cannot be calculated by simply applying back-propagation. To calculate the gradient of the OSQA-based objective function, we formulated a DNN optimization scheme on the basis of black-box optimization, which is used for training a computer that plays a game. For a black-box-optimization scheme, we adopt the policy gradient method for calculating the gradient on the basis of a sampling algorithm. To simulate output signals using the sampling algorithm, DNNs are used to estimate the probability-density function of the output signals that maximize OSQA scores. The OSQA scores are calculated from the simulated output signals, and the DNNs are trained to increase the probability of generating the simulated output signals that achieve high OSQA scores. Through several experiments, we found that OSQA scores significantly increased by applying the proposed method, even though the MSE was not minimized
Reinforcement Learning of Speech Recognition System Based on Policy Gradient and Hypothesis Selection
Speech recognition systems have achieved high recognition performance for
several tasks. However, the performance of such systems is dependent on the
tremendously costly development work of preparing vast amounts of task-matched
transcribed speech data for supervised training. The key problem here is the
cost of transcribing speech data. The cost is repeatedly required to support
new languages and new tasks. Assuming broad network services for transcribing
speech data for many users, a system would become more self-sufficient and more
useful if it possessed the ability to learn from very light feedback from the
users without annoying them. In this paper, we propose a general reinforcement
learning framework for speech recognition systems based on the policy gradient
method. As a particular instance of the framework, we also propose a hypothesis
selection-based reinforcement learning method. The proposed framework provides
a new view for several existing training and adaptation methods. The
experimental results show that the proposed method improves the recognition
performance compared to unsupervised adaptation.Comment: 5 pages, 6 figure
On the Relationship Between Short-Time Objective Intelligibility and Short-Time Spectral-Amplitude Mean-Square Error for Speech Enhancement
The majority of deep neural network (DNN) based speech enhancement algorithms
rely on the mean-square error (MSE) criterion of short-time spectral amplitudes
(STSA), which has no apparent link to human perception, e.g. speech
intelligibility. Short-Time Objective Intelligibility (STOI), a popular
state-of-the-art speech intelligibility estimator, on the other hand, relies on
linear correlation of speech temporal envelopes. This raises the question if a
DNN training criterion based on envelope linear correlation (ELC) can lead to
improved speech intelligibility performance of DNN based speech enhancement
algorithms compared to algorithms based on the STSA-MSE criterion. In this
paper we derive that, under certain general conditions, the STSA-MSE and ELC
criteria are practically equivalent, and we provide empirical data to support
our theoretical results. Furthermore, our experimental findings suggest that
the standard STSA minimum-MSE estimator is near optimal, if the objective is to
enhance noisy speech in a manner which is optimal with respect to the STOI
speech intelligibility estimator
Attention-based Speech Enhancement Using Human Quality Perception Modelling
Perceptually-inspired objective functions such as the perceptual evaluation
of speech quality (PESQ), signal-to-distortion ratio (SDR), and short-time
objective intelligibility (STOI), have recently been used to optimize
performance of deep-learning-based speech enhancement algorithms. These
objective functions, however, do not always strongly correlate with a
listener's assessment of perceptual quality, so optimizing with these measures
often results in poorer performance in real-world scenarios. In this work, we
propose an attention-based enhancement approach that uses learned speech
embedding vectors from a mean-opinion score (MOS) prediction model and a speech
enhancement module to jointly enhance noisy speech. The MOS prediction model
estimates the perceptual MOS of speech quality, as assessed by human listeners,
directly from the audio signal. The enhancement module also employs a quantized
language model that enforces spectral constraints for better speech realism and
performance. We train the model using real-world noisy speech data that has
been captured in everyday environments and test it using unseen corpora. The
results show that our proposed approach significantly outperforms other
approaches that are optimized with objective measures, where the predicted
quality scores strongly correlate with human judgments.Comment: 11 pages, 4 figures, 3 tables, submitted in journal TASLP 202
Single-Microphone Speech Enhancement and Separation Using Deep Learning
The cocktail party problem comprises the challenging task of understanding a
speech signal in a complex acoustic environment, where multiple speakers and
background noise signals simultaneously interfere with the speech signal of
interest. A signal processing algorithm that can effectively increase the
speech intelligibility and quality of speech signals in such complicated
acoustic situations is highly desirable. Especially for applications involving
mobile communication devices and hearing assistive devices. Due to the
re-emergence of machine learning techniques, today, known as deep learning, the
challenges involved with such algorithms might be overcome. In this PhD thesis,
we study and develop deep learning-based techniques for two sub-disciplines of
the cocktail party problem: single-microphone speech enhancement and
single-microphone multi-talker speech separation. Specifically, we conduct
in-depth empirical analysis of the generalizability capability of modern deep
learning-based single-microphone speech enhancement algorithms. We show that
performance of such algorithms is closely linked to the training data, and good
generalizability can be achieved with carefully designed training data.
Furthermore, we propose uPIT, a deep learning-based algorithm for
single-microphone speech separation and we report state-of-the-art results on a
speaker-independent multi-talker speech separation task. Additionally, we show
that uPIT works well for joint speech separation and enhancement without
explicit prior knowledge about the noise type or number of speakers. Finally,
we show that deep learning-based speech enhancement algorithms designed to
minimize the classical short-time spectral amplitude mean squared error leads
to enhanced speech signals which are essentially optimal in terms of STOI, a
state-of-the-art speech intelligibility estimator.Comment: PhD Thesis. 233 page
- …