Contributions to theory and algorithms of independent component analysis and signal separation

This thesis addresses the problem of blind signal separation (BSS) using independent component analysis (ICA). In blind signal separation, signals from multiple sources arrive simultaneously at a sensor array, so that each sensor array output contains a mixture of source signals. Sets of sensor outputs are processed to recover the source signals or to identify the mixing system. The term blind refers to the fact that no explicit knowledge of source signals or mixing system is available. Independent component analysis approach uses statistical independence of the source signals to solve the blind signal separation problems. Application domains for the material presented in this thesis include communications, biomedical, audio, image, and sensor array signal processing. In this thesis reliable algorithms for ICA-based blind source separation are developed. In blind source separation problem the goal is to recover all original source signals using the observed mixtures only. The objective is to develop algorithms that are either adaptive to unknown source distributions or do not need to utilize the source distribution information at all. Two parametric methods that can adapt to a wide class of source distributions including skewed distributions are proposed. Another nonparametric technique with desirable large sample properties is also proposed. It is based on characteristic functions and thereby avoids the need to model the source distributions. Experimental results showing reliable performance are given on all of the presented methods. In this thesis theoretical conditions under which instantaneous ICA-based blind signal processing problems can be solved are established. These results extend the celebrated results by Comon of the traditional linear real-valued model. The results are further extended to complex-valued signals and to nonlinear mixing systems. Conditions for identification, uniqueness, and separation are established both for real and complex-valued linear models, and for a proposed class of non-linear mixing systems.reviewe

Underdetermined convolutive source separation using two dimensional non-negative factorization techniques

PhD ThesisIn this thesis the underdetermined audio source separation has been considered, that is, estimating the original audio sources from the observed mixture when the number of audio sources is greater than the number of channels. The separation has been carried out using two approaches; the blind audio source separation and the informed audio source separation. The blind audio source separation approach depends on the mixture signal only and it assumes that the separation has been accomplished without any prior information (or as little as possible) about the sources. The informed audio source separation uses the exemplar in addition to the mixture signal to emulate the targeted speech signal to be separated. Both approaches are based on the two dimensional factorization techniques that decompose the signal into two tensors that are convolved in both the temporal and spectral directions. Both approaches are applied on the convolutive mixture and the high-reverberant convolutive mixture which are more realistic than the instantaneous mixture. In this work a novel algorithm based on the nonnegative matrix factor two dimensional deconvolution (NMF2D) with adaptive sparsity has been proposed to separate the audio sources that have been mixed in an underdetermined convolutive mixture. Additionally, a novel Gamma Exponential Process has been proposed for estimating the convolutive parameters and number of components of the NMF2D/ NTF2D, and to initialize the NMF2D parameters. In addition, the effects of different window length have been investigated to determine the best fit model that suit the characteristics of the audio signal. Furthermore, a novel algorithm, namely the fusion K models of full-rank weighted nonnegative tensor factor two dimensional deconvolution (K-wNTF2D) has been proposed. The K-wNTF2D is developed for its ability in modelling both the spectral and temporal changes, and the spatial covariance matrix that addresses the high reverberation problem. Variable sparsity that derived from the Gibbs distribution is optimized under the Itakura-Saito divergence and adapted into the K-wNTF2D model. The tensors of this algorithm have been initialized by a novel initialization method, namely the SVD two-dimensional deconvolution (SVD2D). Finally, two novel informed source separation algorithms, namely, the semi-exemplar based algorithm and the exemplar-based algorithm, have been proposed. These algorithms based on the NMF2D model and the proposed two dimensional nonnegative matrix partial co-factorization (2DNMPCF) model. The idea of incorporating the exemplar is to inform the proposed separation algorithms about the targeted signal to be separated by initializing its parameters and guide the proposed separation algorithms. The adaptive sparsity is derived for both ii of the proposed algorithms. Also, a multistage of the proposed exemplar based algorithm has been proposed in order to further enhance the separation performance. Results have shown that the proposed separation algorithms are very promising, more flexible, and offer an alternative model to the conventional methods

Parametric Approach to Blind Deconvolution of Nonlinear Channels

A parametric procedure for the blind inversion of nonlinear channels is proposed, based on a recent method of blind source separation in nonlinear mixtures. Experiments show that the proposed algorithms perform efficiently, even in the presence of hard distortion. The method, based on the minimization of the output mutual information, needs the knowledge of log-derivative of input distribution (the so-called score function). Each algorithm consists of three adaptive blocks: one devoted to adaptive estimation of the score function, and two other blocks estimating the inverses of the linear and nonlinear parts of the channel, (quasi-)optimally adapted using the estimated score functions. This paper is mainly concerned by the nonlinear part, for which we propose two parametric models, the first based on a polynomial model and the second on a neural network, while [14, 15] proposed non-parametric approaches

Notes on nonnegative tensor factorization of the spectrogram for audio source separation : statistical insights and towards self-clustering of the spatial cues

International audienceNonnegative tensor factorization (NTF) of multichannel spectrograms under PARAFAC structure has recently been proposed by Fitzgerald et al as a mean of performing blind source separation (BSS) of multichannel audio data. In this paper we investigate the statistical source models implied by this approach. We show that it implicitly assumes a nonpoint-source model contrasting with usual BSS assumptions and we clarify the links between the measure of fit chosen for the NTF and the implied statistical distribution of the sources. While the original approach of Fitzgeral et al requires a posterior clustering of the spatial cues to group the NTF components into sources, we discuss means of performing the clustering within the factorization. In the results section we test the impact of the simplifying nonpoint-source assumption on underdetermined linear instantaneous mixtures of musical sources and discuss the limits of the approach for such mixtures

BLIND SOURCE SEPARATION USING MAXIMUM ENTROPY PDF ESTIMATION BASED ON FRACTIONAL MOMENTS

Abstract. Recovering a set of independent sources which are linearly mixed is the main task of the blind source separation. Utilizing different methods such as infomax principle, mutual information and maximum likelihood leads to simple iterative procedures such as natural gradient algorithms. These algorithms depend on a nonlinear function (known as score or activation function) of source distributions. Since there is no prior knowledge of source distributions, the optimality of the algorithms is based on the choice of a suitable parametric density model. In this paper, we propose an adaptive optimal score function based on the fractional moments of the sources. In order to obtain a parametric model for the source distributions, we use a few sampled fractional moments to construct the maximum entropy probability density function (PDF) estimation . By applying an optimization method we can obtain the optimal fractional moments that best fit the source distributions. Using the fractional moments (FM) instead of the integer moments causes the maximum entropy estimated PDF to converge to the true PDF much faster . The simulation results show that unlike the most previous proposed models for the nonlinear score function, which are limited to some sorts of source families such as sub-gaussian and super-gaussian or some forms of source distribution models such as generalized gaussian distribution, our new model achieves better results for every source signal without any prior assumption for its randomness behavior

Towards Single-Channel Unsupervised Source Separation of Speech Mixtures: The Layered Harmonics/Formants Separation-Tracking Model

Speaker models for blind source separation are typically based on HMMs consisting of vast numbers of states to capture source spectral variation, and trained on large amounts of isolated speech. Since observations can be similar between sources, inference relies on sequential constraints from the state transition matrix which are, however, quite weak. To avoid these problems, we propose a strategy of capturing local deformations of the time-frequency energy distribution. Since consecutive spectral frames are highly correlated, each frame can be accurately described as a nonuniform deformation of its predecessor. A smooth pattern of deformations is indicative of a single speaker, and the cliffs in the deformation fields may indicate a speaker switch. Further, the log-spectrum of speech can be decomposed into two additive layers, separately describing the harmonics and formant structure. We model smooth deformations as hidden transformation variables in both layers, using MRFs with overlapping subwindows as observations, assumed to be a noisy sum of the two layers. Loopy belief propagation provides for efficient inference. Without any pre-trained speech or speaker models, this approach can be used to fill in missing time-frequency observations, and the local entropy of the deformation fields indicate source boundaries for separation

Adaptive methods for score function modeling in blind source separation

In signal processing and related fields, multichannel measurements are often encountered. Depending on the application, for instance, multiple antennas, multiple microphones or multiple biomedical sensors are used for the data acquisition. Such systems can be described using Multiple-Input Multiple-Output (MIMO) system models. In many cases, several source signals are present at the same time and there is only limited knowledge of their properties and how they contribute to each sensor output. If the source signals and the physical system are unknown and only the sensor outputs are observed, the processing methods developed for recovering the original signals are called blind. In Blind Source Separation (BSS) the goal is to recover the source signals from the observed mixed signals (mixtures). Blindness means that neither the sources nor the mixing system is known. Separation can be based on the theoretically limiting but practically feasible assumption that the sources are statistically independent. This assumption connects BSS and Independent Component Analysis (ICA). The usage of mutual information as a measure of independence leads to iterative estimation of the score functions of the mixtures. The purpose of this thesis is to develop BSS methods that can adapt to different source distributions. Adaptation makes it possible to separate sources without knowing the source distributions or even the characteristics of source distributions. Special attention is paid to methods that allow also asymmetric source distributions. Asymmetric distributions occur in important applications such as communications and biomedical signal processing. Adaptive techniques are proposed for the modeling of score functions or estimating functions. Three approaches based on the Pearson system, the Extended Generalized Lambda Distribution (EGLD) and adaptively combined fixed estimating functions are proposed. The Pearson system and the EGLD are parametric families of distributions and they are used to model the distributions of the mixtures. The strength of these parametric families is that they contain a wide class of distributions, including asymmetric distributions with positive and negative kurtosis, while the estimation of the parameters is still a relatively simple procedure. The methods may be implemented using existing ICA algorithms. The reliable performance of the proposed methods is demonstrated in extensive simulations. In addition to symmetric source distributions, asymmetric distributions, such as Rayleigh and lognormal distribution, are utilized in simulations. The score adaptive methods outperform commonly used methods due to their ability to adapt to asymmetric distributions.reviewe