Mixing and non-mixing local minima of the entropy contrast for blind source separation

In this paper, both non-mixing and mixing local minima of the entropy are analyzed from the viewpoint of blind source separation (BSS); they correspond respectively to acceptable and spurious solutions of the BSS problem. The contribution of this work is twofold. First, a Taylor development is used to show that the \textit{exact} output entropy cost function has a non-mixing minimum when this output is proportional to \textit{any} of the non-Gaussian sources, and not only when the output is proportional to the lowest entropic source. Second, in order to prove that mixing entropy minima exist when the source densities are strongly multimodal, an entropy approximator is proposed. The latter has the major advantage that an error bound can be provided. Even if this approximator (and the associated bound) is used here in the BSS context, it can be applied for estimating the entropy of any random variable with multimodal density.Comment: 11 pages, 6 figures, To appear in IEEE Transactions on Information Theor

Least Dependent Component Analysis Based on Mutual Information

We propose to use precise estimators of mutual information (MI) to find least dependent components in a linearly mixed signal. On the one hand this seems to lead to better blind source separation than with any other presently available algorithm. On the other hand it has the advantage, compared to other implementations of `independent' component analysis (ICA) some of which are based on crude approximations for MI, that the numerical values of the MI can be used for: (i) estimating residual dependencies between the output components; (ii) estimating the reliability of the output, by comparing the pairwise MIs with those of re-mixed components; (iii) clustering the output according to the residual interdependencies. For the MI estimator we use a recently proposed k-nearest neighbor based algorithm. For time sequences we combine this with delay embedding, in order to take into account non-trivial time correlations. After several tests with artificial data, we apply the resulting MILCA (Mutual Information based Least dependent Component Analysis) algorithm to a real-world dataset, the ECG of a pregnant woman. The software implementation of the MILCA algorithm is freely available at http://www.fz-juelich.de/nic/cs/softwareComment: 18 pages, 20 figures, Phys. Rev. E (in press

BMICA-independent component analysis based on B-spline mutual information estimator

The information theoretic concept of mutual information provides a general framework to evaluate dependencies between variables. Its estimation however using B-Spline has not been used before in creating an approach for Independent Component Analysis. In this paper we present a B-Spline estimator for mutual information to find the independent components in mixed signals. Tested using electroencephalography (EEG) signals the resulting BMICA (B-Spline Mutual Information Independent Component Analysis) exhibits better performance than the standard Independent Component Analysis algorithms of FastICA, JADE, SOBI and EFICA in similar simulations. BMICA was found to be also more reliable than the 'renown' FastICA

On the conditions for valid objective functions in blind separation of independent and dependent sources

It is well known that independent sources can be blindly detected and separated, one by one, from linear mixtures by identifying local extrema of certain objective functions (contrasts), like negentropy, Non-Gaussianity measures, kurtosis, etc. It was also suggested in [1], and verified in practice in [2,4], that some of these measures remain useful for particular cases with dependent sources, but not much work has been done in this respect and a rigorous theoretical ground still lacks. In this paper, it is shown that, if a specific type of pairwise dependence among sources exists, called Linear Conditional Expectation (LCE) law, then a family of objective functions are valid for their separation. Interestingly, this particular type of dependence arises in modeling material abundances in the spectral unmixing problem of remote sensed images. In this work, a theoretical novel approach is used to analyze Shannon entropy (SE), Non-Gaussianity (NG) measure and absolute moments of arbitrarily order, i.e. Generic Absolute (GA) moments for the separation of sources allowing them to be dependent. We provide theoretical results that show the conditions under which sources are isolated by searching for a maximum or a minimum. Also, simple and efficient algorithms based on Parzen windows estimations of probability density functions (pdfs) and Newton-Raphson iterations are proposed for the separation of dependent or independent sources. A set of simulation results on synthetic data and an application to the blind spectral unmixing problem are provided in order to validate our theoretical results and compare these algorithms against FastICA and a very recently proposed algorithm for dependent sources, the Bounded Component Analysis algorithm (BCA). It is shown that, for dependent sources verifying the LCE law, the NG measure provides the best separation results.Fil: Caiafa, Cesar Federico. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico la Plata. Instituto Argentino de Radioastronomia (i); Argentina. Universidad de Buenos Aires. Facultad de Ingeniería; Argentin