Faster Training in Nonlinear ICA using MISEP

MISEP has been proposed as a generalization of the INFOMAX method in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs, making the method suitable to the separation of components with a wide range of statistical distributions. In all implementations up to now, MISEP had used multilayer perceptrons (MLPs) to perform the nonlinear ICA operation. Use of MLPs sometimes leads to a relatively slow training. This has been attributed, at least in part, to the non-local character of the MLP's units. This paper investigates the possibility of using a network of radial basis function (RBF) units for performing the nonlinear ICA operation. It shows that the local character of the RBF network's units allows a significant speedup in the training of the system. The paper gives a brief introduction to the basics of the MISEP method, and presents experimental results showing the speed advantage of using an RBF-based network to perform the ICA operation

MISEP - Linear and Nonlinear ICA Based on Mutual Information

MISEP is a method for linear and nonlinear ICA, that is able to handle a large variety of situations. It is an extension of the well known INFOMAX method, in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs. The method can therefore separate linear and nonlinear mixtures of components with a wide range of statistical distributions. This paper presents the basis of the MISEP method, as well as experimental results obtained with it. The results illustrate the applicability of the method to various situations, and show that, although the nonlinear blind separation problem is ill-posed, use of regularization allows the problem to be solved when the nonlinear mixture is relatively smooth

Separating a Real-Life Nonlinear Image Mixture

When acquiring an image of a paper document, the image printed on the back page sometimes shows through. The mixture of the front- and back-page images thus obtained is markedly nonlinear, and thus constitutes a good real-life test case for nonlinear blind source separation. This paper addresses a difficult version of this problem, corresponding to the use of "onion skin" paper, which results in a relatively strong nonlinearity of the mixture, which becomes close to singular in the lighter regions of the images. The separation is achieved through the MISEP technique, which is an extension of the well known INFOMAX method. The separation results are assessed with objective quality measures. They show an improvement over the results obtained with linear separation, but have room for further improvement

WAVELET BASED NONLINEAR SEPARATION OF IMAGES

This work addresses a real-life problem corresponding to the separation of the nonlinear mixture of images which arises when we scan a paper document and the image from the back page shows through. The proposed solution consists of a non-iterative procedure that is based on two simple observations: (1) the high frequency content of images is sparse, and (2) the image printed on each side of the paper appears more strongly in the mixture acquired from that side than in the mixture acquired from the opposite side. These ideas had already been used in the context of nonlinear denoising source separation (DSS). However, in that method the degree of separation achieved by applying these ideas was relatively weak, and the separation had to be improved by iterating within the DSS scheme. In this paper the application of these ideas is improved by changing the competition function and the wavelet transform that is used. These improvements allow us to achieve a good separation in one shot, without the need to integrate the process into an iterative DSS scheme. The resulting separation process is both nonlinear and non-local. We present experimental results that show that the method achieves a good separation quality

Non-linear ICA based on Cramer-Wold metric

Non-linear source separation is a challenging open problem with many applications. We extend a recently proposed Adversarial Non-linear ICA (ANICA) model, and introduce Cramer-Wold ICA (CW-ICA). In contrast to ANICA we use a simple, closed--form optimization target instead of a discriminator--based independence measure. Our results show that CW-ICA achieves comparable results to ANICA, while foregoing the need for adversarial training