Audio source separation into the wild

International audienceThis review chapter is dedicated to multichannel audio source separation in real-life environment. We explore some of the major achievements in the field and discuss some of the remaining challenges. We will explore several important practical scenarios, e.g. moving sources and/or microphones, varying number of sources and sensors, high reverberation levels, spatially diffuse sources, and synchronization problems. Several applications such as smart assistants, cellular phones, hearing aids and robots, will be discussed. Our perspectives on the future of the field will be given as concluding remarks of this chapter

Real Time Blind Source Separation in Reverberant Environments

An online convolutive blind source separation solution has been developed for use in reverberant environments with stationary sources. Results are presented for simulation and real world data. The system achieves a separation SINR of 16.8 dB when operating on a two source mixture, with a total acoustic delay was 270 ms. This is on par with, and in many respects outperforms various published algorithms [1],[2]. A number of instantaneous blind source separation algorithms have been developed, including a block wise and recursive ICA algorithm, and a clustering based algorithm, able to obtain up to 110 dB SIR performance. The system has been realised in both Matlab and C, and is modular, allowing for easy update of the ICA algorithm that is the core of the unmixing process

Uma investigação sobre métodos de separação cega de fontes sonoras envolvendo representações não-negativas e diversidade espacial

The problem of blind source separation finds many applications across different areas, thus justifying the ever increasing number of works in this topic. This work focuses on studying this problem for sound sources, employing non-negative signals’ representations, while also taking advantage of the spatial diversity induced by the use of multiple channels; this particular feature has recently opened up new research directions regarding the proper modeling of multichannel source separation This work studies two different algorithms: NMF-SCM (sound source separation using non-negative matrix factorization and direction-of-arrival-based spatial covariance model), whose model represents the state of the art, taking in consideration not only the characteristics of the sources but also the enviroment into which they were captured on; and NTF (non-negative tensor factorization), whose simplified model is the multichannel equivalent of NMF (non-negative matrix factorization). During the development of this work both algorithms were implemented. A vectorized and parallelized NMF-SCM implementation is presented; and some improvements are proposed to the NTF algorithm, as well as a method for blind determination of the number of sources in multichannel mixtures.A separação cega de fontes é um problema com diversas aplicações em várias áreas e que, por isso, vem sendo alvo de um grande número de pesquisas. Este trabalho foca no estudo do problema de separação cega de fontes sonoras utilizando representações não-negativas com o aproveitamento da diversidade espacial permitido pelo desenvolvimento de métodos multicanais, que abriram novas oportunidades de pesquisa e resultaram no surgimento de novas modelagens para o problema de separação de fontes. Neste trabalho são estudados dois algoritmos distintos, a NMF-SCM (separação de áudio multicanal utilizando fatoração não-negativa de matrizes e com modelo de covariância espacial baseado em direção-de-chegada), que representa o estado da arte da modelagem deste problema, modelando não apenas as características das fontes, mas também o ambiente em que a mistura foi capturada; e a NTF (fatoração de tensores não-negativos), que apresenta uma modelagem simplificada do problema multicanal, de forma análoga à NMF (fatoração não-negativa de matrizes), e que não utiliza explicitamente a diversidade espacial. Durante o desenvolvimento deste trabalho ambos os algoritmos foram implementados. Uma versão vetorizada e paralelizada da NMF-SCM é apresentada, assim como alterações ao algoritmo da NTF visando à melhoria em seu desempenho e também à utilização explícita da diversidade espacial. Por último, é proposto um método para a determinação cega do número de fontes presentes em misturas multicanais

Object-based Modeling of Audio for Coding and Source Separation

This thesis studies several data decomposition algorithms for obtaining an object-based representation of an audio signal. The estimation of the representation parameters are coupled with audio-specific criteria, such as the spectral redundancy, sparsity, perceptual relevance and spatial position of sounds. The objective is to obtain an audio signal representation that is composed of meaningful entities called audio objects that reflect the properties of real-world sound objects and events. The estimation of the object-based model is based on magnitude spectrogram redundancy using non-negative matrix factorization with extensions to multichannel and complex-valued data. The benefits of working with object-based audio representations over the conventional time-frequency bin-wise processing are studied. The two main applications of the object-based audio representations proposed in this thesis are spatial audio coding and sound source separation from multichannel microphone array recordings. In the proposed spatial audio coding algorithm, the audio objects are estimated from the multichannel magnitude spectrogram. The audio objects are used for recovering the content of each original channel from a single downmixed signal, using time-frequency filtering. The perceptual relevance of modeling the audio signal is considered in the estimation of the parameters of the object-based model, and the sparsity of the model is utilized in encoding its parameters. Additionally, a quantization of the model parameters is proposed that reflects the perceptual relevance of each quantized element. The proposed object-based spatial audio coding algorithm is evaluated via listening tests and comparing the overall perceptual quality to conventional time-frequency block-wise methods at the same bitrates. The proposed approach is found to produce comparable coding efficiency while providing additional functionality via the object-based coding domain representation, such as the blind separation of the mixture of sound sources in the encoded channels. For the sound source separation from multichannel audio recorded by a microphone array, a method combining an object-based magnitude model and spatial covariance matrix estimation is considered. A direction of arrival-based model for the spatial covariance matrices of the sound sources is proposed. Unlike the conventional approaches, the estimation of the parameters of the proposed spatial covariance matrix model ensures a spatially coherent solution for the spatial parameterization of the sound sources. The separation quality is measured with objective criteria and the proposed method is shown to improve over the state-of-the-art sound source separation methods, with recordings done using a small microphone array

Fonctions de coût pour l'estimation des filtres acoustiques dans les mélanges réverbérants

On se place dans le cadre du traitement des signaux audio multicanaux et multi-sources. À partir du mélange de plusieurs sources sonores enregistrées en milieu réverbérant, on cherche à estimer les réponses acoustiques (ou filtres de mélange) entre les sources et les microphones. Ce problème inverse ne peut être résolu qu'en prenant en compte des hypothèses sur la nature des filtres. Notre approche consiste d'une part à identifier mathématiquement les hypothèses nécessaires sur les filtres pour pouvoir les estimer et d'autre part à construire des fonctions de coût et des algorithmes permettant de les estimer effectivement. Premièrement, nous avons considéré le cas où les signaux sources sont connus. Nous avons développé une méthode d'estimation des filtres basée sur une régularisation convexe prenant en compte à la fois la nature parcimonieuse des filtres et leur enveloppe de forme exponentielle décroissante. Nous avons effectué des enregistrements en environnement réel qui ont confirmé l'efficacité de cet algorithme. Deuxièmement, nous avons considéré le cas où les signaux sources sont inconnus, mais statistiquement indépendants. Les filtres de mélange peuvent alors être estimés à une indétermination de permutation et de gain près à chaque fréquence par des techniques d'analyse en composantes indépendantes. Nous avons apporté une étude exhaustive des garanties théoriques par lesquelles l'indétermination de permutation peut être levée dans le cas où les filtres sont parcimonieux dans le domaine temporel. Troisièmement, nous avons commencé à analyser les hypothèses sous lesquelles notre algorithme d'estimation des filtres pourrait être étendu à l'estimation conjointe des signaux sources et des filtres et montré un premier résultat négatif inattendu : dans le cadre de la déconvolution parcimonieuse aveugle, pour une famille assez large de fonctions de coût régularisées, le minimum global est trivial. Des contraintes supplémentaires sur les signaux sources ou les filtres sont donc nécessaires.This work is focused on the processing of multichannel and multisource audio signals. From an audio mixture of several audio sources recorded in a reverberant room, we wish to estimate the acoustic responses (a.k.a. mixing filters) between the sources and the microphones. To solve this inverse problem one need to take into account additional hypotheses on the nature of the acoustic responses. Our approach consists in first identifying mathematically the necessary hypotheses on the acoustic responses for their estimation and then building cost functions and algorithms to effectively estimate them. First, we considered the case where the source signals are known. We developed a method to estimate the acoustic responses based on a convex regularization which exploits both the temporal sparsity of the filters and the exponentially decaying envelope. Real-world experiments confirmed the effectiveness of this method on real data. Then, we considered the case where the sources signal are unknown, but statistically independent. The mixing filters can be estimated up to a permutation and scaling ambiguity. We brought up an exhaustive study of the theoretical conditions under which we can solve the indeterminacy, when the multichannel filters are sparse in the temporal domain. Finally, we started to analyse the hypotheses under which this algorithm could be extended to the joint estimation of the sources and the filters, and showed a first unexpected results : in the context of blind deconvolution with sparse priors, for a quite large family of regularised cost functions, the global minimum is trivial. Additional constraints on the source signals and the filters are needed.RENNES1-Bibl. électronique (352382106) / SudocSudocFranceF