Structured Sparsity Models for Reverberant Speech Separation

We tackle the multi-party speech recovery problem through modeling the acoustic of the reverberant chambers. Our approach exploits structured sparsity models to perform room modeling and speech recovery. We propose a scheme for characterizing the room acoustic from the unknown competing speech sources relying on localization of the early images of the speakers by sparse approximation of the spatial spectra of the virtual sources in a free-space model. The images are then clustered exploiting the low-rank structure of the spectro-temporal components belonging to each source. This enables us to identify the early support of the room impulse response function and its unique map to the room geometry. To further tackle the ambiguity of the reflection ratios, we propose a novel formulation of the reverberation model and estimate the absorption coefficients through a convex optimization exploiting joint sparsity model formulated upon spatio-spectral sparsity of concurrent speech representation. The acoustic parameters are then incorporated for separating individual speech signals through either structured sparse recovery or inverse filtering the acoustic channels. The experiments conducted on real data recordings demonstrate the effectiveness of the proposed approach for multi-party speech recovery and recognition

Fonctions de coût pour l'estimation des filtres acoustiques dans les mélanges réverbérants

On se place dans le cadre du traitement des signaux audio multicanaux et multi-sources. À partir du mélange de plusieurs sources sonores enregistrées en milieu réverbérant, on cherche à estimer les réponses acoustiques (ou filtres de mélange) entre les sources et les microphones. Ce problème inverse ne peut être résolu qu'en prenant en compte des hypothèses sur la nature des filtres. Notre approche consiste d'une part à identifier mathématiquement les hypothèses nécessaires sur les filtres pour pouvoir les estimer et d'autre part à construire des fonctions de coût et des algorithmes permettant de les estimer effectivement. Premièrement, nous avons considéré le cas où les signaux sources sont connus. Nous avons développé une méthode d'estimation des filtres basée sur une régularisation convexe prenant en compte à la fois la nature parcimonieuse des filtres et leur enveloppe de forme exponentielle décroissante. Nous avons effectué des enregistrements en environnement réel qui ont confirmé l'efficacité de cet algorithme. Deuxièmement, nous avons considéré le cas où les signaux sources sont inconnus, mais statistiquement indépendants. Les filtres de mélange peuvent alors être estimés à une indétermination de permutation et de gain près à chaque fréquence par des techniques d'analyse en composantes indépendantes. Nous avons apporté une étude exhaustive des garanties théoriques par lesquelles l'indétermination de permutation peut être levée dans le cas où les filtres sont parcimonieux dans le domaine temporel. Troisièmement, nous avons commencé à analyser les hypothèses sous lesquelles notre algorithme d'estimation des filtres pourrait être étendu à l'estimation conjointe des signaux sources et des filtres et montré un premier résultat négatif inattendu : dans le cadre de la déconvolution parcimonieuse aveugle, pour une famille assez large de fonctions de coût régularisées, le minimum global est trivial. Des contraintes supplémentaires sur les signaux sources ou les filtres sont donc nécessaires.This work is focused on the processing of multichannel and multisource audio signals. From an audio mixture of several audio sources recorded in a reverberant room, we wish to estimate the acoustic responses (a.k.a. mixing filters) between the sources and the microphones. To solve this inverse problem one need to take into account additional hypotheses on the nature of the acoustic responses. Our approach consists in first identifying mathematically the necessary hypotheses on the acoustic responses for their estimation and then building cost functions and algorithms to effectively estimate them. First, we considered the case where the source signals are known. We developed a method to estimate the acoustic responses based on a convex regularization which exploits both the temporal sparsity of the filters and the exponentially decaying envelope. Real-world experiments confirmed the effectiveness of this method on real data. Then, we considered the case where the sources signal are unknown, but statistically independent. The mixing filters can be estimated up to a permutation and scaling ambiguity. We brought up an exhaustive study of the theoretical conditions under which we can solve the indeterminacy, when the multichannel filters are sparse in the temporal domain. Finally, we started to analyse the hypotheses under which this algorithm could be extended to the joint estimation of the sources and the filters, and showed a first unexpected results : in the context of blind deconvolution with sparse priors, for a quite large family of regularised cost functions, the global minimum is trivial. Additional constraints on the source signals and the filters are needed.RENNES1-Bibl. électronique (352382106) / SudocSudocFranceF

An investigation of the utility of monaural sound source separation via nonnegative matrix factorization applied to acoustic echo and reverberation mitigation for hands-free telephony

In this thesis we investigate the applicability and utility of Monaural Sound Source Separation (MSSS) via Nonnegative Matrix Factorization (NMF) for various problems related to audio for hands-free telephony. We first investigate MSSS via NMF as an alternative acoustic echo reduction approach to existing approaches such as Acoustic Echo Cancellation (AEC). To this end, we present the single-channel acoustic echo problem as an MSSS problem, in which the objective is to extract the users signal from a mixture also containing acoustic echo and noise. To perform separation, NMF is used to decompose the near-end microphone signal onto the union of two nonnegative bases in the magnitude Short Time Fourier Transform domain. One of these bases is for the spectral energy of the acoustic echo signal, and is formed from the in- coming far-end user’s speech, while the other basis is for the spectral energy of the near-end speaker, and is trained with speech data a priori. In comparison to AEC, the speaker extraction approach obviates Double-Talk Detection (DTD), and is demonstrated to attain its maximal echo mitigation performance immediately upon initiation and to maintain that performance during and after room changes for similar computational requirements. Speaker extraction is also shown to introduce distortion of the near-end speech signal during double-talk, which is quantified by means of a speech distortion measure and compared to that of AEC. Subsequently, we address Double-Talk Detection (DTD) for block-based AEC algorithms. We propose a novel block-based DTD algorithm that uses the available signals and the estimate of the echo signal that is produced by NMF-based speaker extraction to compute a suitably normalized correlation-based decision variable, which is compared to a fixed threshold to decide on doubletalk. Using a standard evaluation technique, the proposed algorithm is shown to have comparable detection performance to an existing conventional block-based DTD algorithm. It is also demonstrated to inherit the room change insensitivity of speaker extraction, with the proposed DTD algorithm generating minimal false doubletalk indications upon initiation and in response to room changes in comparison to the existing conventional DTD. We also show that this property allows its paired AEC to converge at a rate close to the optimum. Another focus of this thesis is the problem of inverting a single measurement of a non- minimum phase Room Impulse Response (RIR). We describe the process by which percep- tually detrimental all-pass phase distortion arises in reverberant speech filtered by the inverse of the minimum phase component of the RIR; in short, such distortion arises from inverting the magnitude response of the high-Q maximum phase zeros of the RIR. We then propose two novel partial inversion schemes that precisely mitigate this distortion. One of these schemes employs NMF-based MSSS to separate the all-pass phase distortion from the target speech in the magnitude STFT domain, while the other approach modifies the inverse minimum phase filter such that the magnitude response of the maximum phase zeros of the RIR is not fully compensated. Subjective listening tests reveal that the proposed schemes generally produce better quality output speech than a comparable inversion technique