129,136 research outputs found
A Two-Stage Training Framework for Joint Speech Compression and Enhancement
This paper considers the joint compression and enhancement problem for speech
signal in the presence of noise. Recently, the SoundStream codec, which relies
on end-to-end joint training of an encoder-decoder pair and a residual vector
quantizer by a combination of adversarial and reconstruction losses,has shown
very promising performance, especially in subjective perception quality. In
this work, we provide a theoretical result to show that, to simultaneously
achieve low distortion and high perception in the presence of noise, there
exist an optimal two-stage optimization procedure for the joint compression and
enhancement problem. This procedure firstly optimizes an encoder-decoder pair
using only distortion loss and then fixes the encoder to optimize a perceptual
decoder using perception loss. Based on this result, we construct a two-stage
training framework for joint compression and enhancement of noisy speech
signal. Unlike existing training methods which are heuristic, the proposed
two-stage training method has a theoretical foundation. Finally, experimental
results for various noise and bit-rate conditions are provided. The results
demonstrate that a codec trained by the proposed framework can outperform
SoundStream and other representative codecs in terms of both objective and
subjective evaluation metrics. Code is available at
\textit{https://github.com/jscscloris/SEStream}
Recommended from our members
Parallel data compression
Data compression schemes remove data redundancy in communicated and stored data and increase the effective capacities of communication and storage devices. Parallel algorithms and implementations for textual data compression are surveyed. Related concepts from parallel computation and information theory are briefly discussed. Static and dynamic methods for codeword construction and transmission on various models of parallel computation are described. Included are parallel methods which boost system speed by coding data concurrently, and approaches which employ multiple compression techniques to improve compression ratios. Theoretical and empirical comparisons are reported and areas for future research are suggested
The Audio Degradation Toolbox and its Application to Robustness Evaluation
We introduce the Audio Degradation Toolbox (ADT) for the controlled degradation of audio signals, and propose its usage as a means of evaluating and comparing the robustness of audio processing algorithms. Music recordings encountered in practical applications are subject to varied, sometimes unpredictable degradation. For example, audio is degraded by low-quality microphones, noisy recording environments, MP3 compression, dynamic compression in broadcasting or vinyl decay. In spite of this, no standard software for the degradation of audio exists, and music processing methods are usually evaluated against clean data. The ADT fills this gap by providing Matlab scripts that emulate a wide range of degradation types. We describe 14 degradation units, and how they can be chained to create more complex, `real-world' degradations. The ADT also provides functionality to adjust existing ground-truth, correcting for temporal distortions introduced by degradation. Using four different music informatics tasks, we show that performance strongly depends on the combination of method and degradation applied. We demonstrate that specific degradations can reduce or even reverse the performance difference between two competing methods. ADT source code, sounds, impulse responses and definitions are freely available for download
- …