Faster Training in Nonlinear ICA using MISEP

MISEP has been proposed as a generalization of the INFOMAX method in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs, making the method suitable to the separation of components with a wide range of statistical distributions. In all implementations up to now, MISEP had used multilayer perceptrons (MLPs) to perform the nonlinear ICA operation. Use of MLPs sometimes leads to a relatively slow training. This has been attributed, at least in part, to the non-local character of the MLP's units. This paper investigates the possibility of using a network of radial basis function (RBF) units for performing the nonlinear ICA operation. It shows that the local character of the RBF network's units allows a significant speedup in the training of the system. The paper gives a brief introduction to the basics of the MISEP method, and presents experimental results showing the speed advantage of using an RBF-based network to perform the ICA operation

MISEP - Linear and Nonlinear ICA Based on Mutual Information

MISEP is a method for linear and nonlinear ICA, that is able to handle a large variety of situations. It is an extension of the well known INFOMAX method, in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs. The method can therefore separate linear and nonlinear mixtures of components with a wide range of statistical distributions. This paper presents the basis of the MISEP method, as well as experimental results obtained with it. The results illustrate the applicability of the method to various situations, and show that, although the nonlinear blind separation problem is ill-posed, use of regularization allows the problem to be solved when the nonlinear mixture is relatively smooth

Separating a Real-Life Nonlinear Image Mixture

When acquiring an image of a paper document, the image printed on the back page sometimes shows through. The mixture of the front- and back-page images thus obtained is markedly nonlinear, and thus constitutes a good real-life test case for nonlinear blind source separation. This paper addresses a difficult version of this problem, corresponding to the use of "onion skin" paper, which results in a relatively strong nonlinearity of the mixture, which becomes close to singular in the lighter regions of the images. The separation is achieved through the MISEP technique, which is an extension of the well known INFOMAX method. The separation results are assessed with objective quality measures. They show an improvement over the results obtained with linear separation, but have room for further improvement

Nonlinear unmixing of hyperspectral images: Models and algorithms

When considering the problem of unmixing hyperspectral images, most of the literature in the geoscience and image processing areas relies on the widely used linear mixing model (LMM). However, the LMM may be not valid, and other nonlinear models need to be considered, for instance, when there are multiscattering effects or intimate interactions. Consequently, over the last few years, several significant contributions have been proposed to overcome the limitations inherent in the LMM. In this article, we present an overview of recent advances in nonlinear unmixing modeling

A Comprehensive Review on Various Estimation Techniques for Multi Input Multi Output Channel

لقد تطورت مشكلة تقدير القناة اللاسلكية بسبب بعض التأثيرات غير المرغوب فيها للخواص الفيزيائية للقناة على الإشارات المرسلة. في نهاية المستقبل، التشوه، والتأخير، والتوهين، والتداخلات، ونوبات الطور هي أكثر المشكلات التي تواجهها مع الإشارات المستقبلة. من أجل التغلب على تأثيرات القناة وتوفير جودة كاملة تقريبًا لنقل البيانات، يلزم تقدير معلومات القناة. في أنظمة المخرجات متعددة المدخلات والمخرجات (MIMO)، يعتبر تقدير القناة خطوة أكثر تعقيدًا مقارنة بأنظمة المخرجات ذات المدخلات المفردة، SISO، نظرًا لأن عدد القنوات الفرعية التي تحتاج إلى تقدير أكبر بكثير من انظمة SISO. الهدف الأساسي من هذه الورقة البحثية هو مراجعة شاملة لاغلب الخوارزميات الشهيرة والفعالة التي تم ابتكارها لحل مشكلة تقدير قناة MIMO في أنظمة الاتصالات اللاسلكية. في هذه الورقة، تم تصنيف هذه التقنيات إلى ثلاث مجموعات: غير المكفوفين، شبه الأعمى وتقدير أعمى. لكل مجموعة، يتم تقديم توضيح مختصر لخوارزميات التقدير المألوفة. وأخيرًا، نقارن بين هذه التقنيات استنادًا إلى التعقيد الحسابي والكمون ودقة التقدير.The problem of wireless channel estimation has been evolving due to some undesirable effects of channel physical properties on transmitted signals. At the receiver end, distortions, delays, attenuations, interferences, and phase shifts are the most issues encounter together with the received signals. In order to overcome channel effects and provide almost a perfect quality of data transmission, channel parameter estimation is needed. In Multiple Input-Multiple Output systems (MIMO), channel estimation is a more complicated step as compared with the Single Input-Single Output systems, SISO, because of the fact that the number of sub-channels that needs estimate is much greater than SISO systems. The fundamental objective of this research paper is to go over the famous and efficient algorithms that have been innovated to solve the problem of MIMO channel estimation in wireless communication systems. In this paper, these techniques have been classified into three groups: non-blind, semi-blind and blind estimation. For each group, a brief illustration is presented for familiar estimation algorithms. Finally, we compare between these techniques based on computational complexity, latency and estimation accuracy

MLP-BASED SOURCE SEPARATION FOR MLP-LIKE NONLINEAR MIXTURES

In this paper, the nonlinear blind source separation problem is addressed by using a multilayer perceptron (MLP) as separating system, which is justified in the universal approximation property of MLP networks. An adaptive learning algorithm for a perceptron with two hidden-layers is presented. The algorithm minimizes the mutual information between the outputs of the MLP. The performance of the proposed method is illustrated by some experiments. 1. INTRODUCTION. Blind Source Separation (BSS) is a fundamental problem in signal processing. It consists of retrieving unobserved sources s1(t),..., sN (t), assumed to be statistically independent (which is phisically plausible when the source

Variational approach for learning Markov processes from time series data

Inference, prediction and control of complex dynamical systems from time series is important in many areas, including financial markets, power grid management, climate and weather modeling, or molecular dynamics. The analysis of such highly nonlinear dynamical systems is facilitated by the fact that we can often find a (generally nonlinear) transformation of the system coordinates to features in which the dynamics can be excellently approximated by a linear Markovian model. Moreover, the large number of system variables often change collectively on large time- and length-scales, facilitating a low-dimensional analysis in feature space. In this paper, we introduce a variational approach for Markov processes (VAMP) that allows us to find optimal feature mappings and optimal Markovian models of the dynamics from given time series data. The key insight is that the best linear model can be obtained from the top singular components of the Koopman operator. This leads to the definition of a family of score functions called VAMP-r which can be calculated from data, and can be employed to optimize a Markovian model. In addition, based on the relationship between the variational scores and approximation errors of Koopman operators, we propose a new VAMP-E score, which can be applied to cross-validation for hyper-parameter optimization and model selection in VAMP. VAMP is valid for both reversible and nonreversible processes and for stationary and non-stationary processes or realizations