Flexible methods for blind separation of complex signals

One of the main matter in Blind Source Separation (BSS) performed with a neural network approach is the choice of the nonlinear activation function (AF). In fact if the shape of the activation function is chosen as the cumulative density function (c.d.f.) of the original source the problem is solved. For this scope in this thesis a flexible approach is introduced and the shape of the activation functions is changed during the learning process using the so-called “spline functions”. The problem is complicated in the case of separation of complex sources where there is the problem of the dichotomy between analyticity and boundedness of the complex activation functions. The problem is solved introducing the “splitting function” model as activation function. The “splitting function” is a couple of “spline function” which wind off the real and the imaginary part of the complex activation function, each of one depending from the real and imaginary variable. A more realistic model is the “generalized splitting function”, which is formed by a couple of two bi-dimensional functions (surfaces), one for the real and one for the imaginary part of the complex function, each depending by both the real and imaginary part of the complex variable. Unfortunately the linear environment is unrealistic in many practical applications. In this way there is the need of extending BSS problem in the nonlinear environment: in this case both the activation function than the nonlinear distorting function are realized by the “splitting function” made of “spline function”. The complex and instantaneous separation in linear and nonlinear environment allow us to perform a complex-valued extension of the well-known INFOMAX algorithm in several practical situations, such as convolutive mixtures, fMRI signal analysis and bandpass signal transmission. In addition advanced characteristics on the proposed approach are introduced and deeply described. First of all it is shows as splines are universal nonlinear functions for BSS problem: they are able to perform separation in anyway. Then it is analyzed as the “splitting solution” allows the algorithm to obtain a phase recovery: usually there is a phase ambiguity. Finally a Cramér-Rao lower bound for ICA is discussed. Several experimental results, tested by different objective indexes, show the effectiveness of the proposed approaches

Contributions to theory and algorithms of independent component analysis and signal separation

This thesis addresses the problem of blind signal separation (BSS) using independent component analysis (ICA). In blind signal separation, signals from multiple sources arrive simultaneously at a sensor array, so that each sensor array output contains a mixture of source signals. Sets of sensor outputs are processed to recover the source signals or to identify the mixing system. The term blind refers to the fact that no explicit knowledge of source signals or mixing system is available. Independent component analysis approach uses statistical independence of the source signals to solve the blind signal separation problems. Application domains for the material presented in this thesis include communications, biomedical, audio, image, and sensor array signal processing. In this thesis reliable algorithms for ICA-based blind source separation are developed. In blind source separation problem the goal is to recover all original source signals using the observed mixtures only. The objective is to develop algorithms that are either adaptive to unknown source distributions or do not need to utilize the source distribution information at all. Two parametric methods that can adapt to a wide class of source distributions including skewed distributions are proposed. Another nonparametric technique with desirable large sample properties is also proposed. It is based on characteristic functions and thereby avoids the need to model the source distributions. Experimental results showing reliable performance are given on all of the presented methods. In this thesis theoretical conditions under which instantaneous ICA-based blind signal processing problems can be solved are established. These results extend the celebrated results by Comon of the traditional linear real-valued model. The results are further extended to complex-valued signals and to nonlinear mixing systems. Conditions for identification, uniqueness, and separation are established both for real and complex-valued linear models, and for a proposed class of non-linear mixing systems.reviewe

Flexible methods for blind separation of complex signals

One of the main matter in Blind Source Separation (BSS) performed with a neural network approach is the choice of the nonlinear activation function (AF). In fact if the shape of the activation function is chosen as the cumulative density function (c.d.f.) of the original source the problem is solved. For this scope in this thesis a flexible approach is introduced and the shape of the activation functions is changed during the learning process using the so-called “spline functions”. The problem is complicated in the case of separation of complex sources where there is the problem of the dichotomy between analyticity and boundedness of the complex activation functions. The problem is solved introducing the “splitting function” model as activation function. The “splitting function” is a couple of “spline function” which wind off the real and the imaginary part of the complex activation function, each of one depending from the real and imaginary variable. A more realistic model is the “generalized splitting function”, which is formed by a couple of two bi-dimensional functions (surfaces), one for the real and one for the imaginary part of the complex function, each depending by both the real and imaginary part of the complex variable. Unfortunately the linear environment is unrealistic in many practical applications. In this way there is the need of extending BSS problem in the nonlinear environment: in this case both the activation function than the nonlinear distorting function are realized by the “splitting function” made of “spline function”. The complex and instantaneous separation in linear and nonlinear environment allow us to perform a complex-valued extension of the well-known INFOMAX algorithm in several practical situations, such as convolutive mixtures, fMRI signal analysis and bandpass signal transmission. In addition advanced characteristics on the proposed approach are introduced and deeply described. First of all it is shows as splines are universal nonlinear functions for BSS problem: they are able to perform separation in anyway. Then it is analyzed as the “splitting solution” allows the algorithm to obtain a phase recovery: usually there is a phase ambiguity. Finally a Cramér-Rao lower bound for ICA is discussed. Several experimental results, tested by different objective indexes, show the effectiveness of the proposed approaches

Adaptive signal processing algorithms for noncircular complex data

The complex domain provides a natural processing framework for a large class of signals encountered in communications, radar, biomedical engineering and renewable energy. Statistical signal processing in C has traditionally been viewed as a straightforward extension of the corresponding algorithms in the real domain R, however, recent developments in augmented complex statistics show that, in general, this leads to under-modelling. This direct treatment of complex-valued signals has led to advances in so called widely linear modelling and the introduction of a generalised framework for the differentiability of both analytic and non-analytic complex and quaternion functions. In this thesis, supervised and blind complex adaptive algorithms capable of processing the generality of complex and quaternion signals (both circular and noncircular) in both noise-free and noisy environments are developed; their usefulness in real-world applications is demonstrated through case studies. The focus of this thesis is on the use of augmented statistics and widely linear modelling. The standard complex least mean square (CLMS) algorithm is extended to perform optimally for the generality of complex-valued signals, and is shown to outperform the CLMS algorithm. Next, extraction of latent complex-valued signals from large mixtures is addressed. This is achieved by developing several classes of complex blind source extraction algorithms based on fundamental signal properties such as smoothness, predictability and degree of Gaussianity, with the analysis of the existence and uniqueness of the solutions also provided. These algorithms are shown to facilitate real-time applications, such as those in brain computer interfacing (BCI). Due to their modified cost functions and the widely linear mixing model, this class of algorithms perform well in both noise-free and noisy environments. Next, based on a widely linear quaternion model, the FastICA algorithm is extended to the quaternion domain to provide separation of the generality of quaternion signals. The enhanced performances of the widely linear algorithms are illustrated in renewable energy and biomedical applications, in particular, for the prediction of wind profiles and extraction of artifacts from EEG recordings

Statistical signal processing of nonstationary tensor-valued data

Real-world signals, such as the evolution of three-dimensional vector fields over time, can exhibit highly structured probabilistic interactions across their multiple constitutive dimensions. This calls for analysis tools capable of directly capturing the inherent multi-way couplings present in such data. Yet, current analyses typically employ multivariate matrix models and their associated linear algebras which are agnostic to the global data structure and can only describe local linear pairwise relationships between data entries. To address this issue, this thesis uses the property of linear separability -- a notion intrinsic to multi-dimensional data structures called tensors -- as a linchpin to consider the probabilistic, statistical and spectral separability under one umbrella. This helps to both enhance physical meaning in the analysis and reduce the dimensionality of tensor-valued problems. We first introduce a new identifiable probability distribution which appropriately models the interactions between random tensors, whereby linear relationships are considered between tensor fibres as opposed to between individual entries as in standard matrix analysis. Unlike existing models, the proposed tensor probability distribution formulation is shown to yield a unique maximum likelihood estimator which is demonstrated to be statistically efficient. Both matrices and vectors are lower-order tensors, and this gives us a unique opportunity to consider some matrix signal processing models under the more powerful framework of multilinear tensor algebra. By introducing a model for the joint distribution of multiple random tensors, it is also possible to treat random tensor regression analyses and subspace methods within a unified separability framework. Practical utility of the proposed analysis is demonstrated through case studies over synthetic and real-world tensor-valued data, including the evolution over time of global atmospheric temperatures and international interest rates. Another overarching theme in this thesis is the nonstationarity inherent to real-world signals, which typically consist of both deterministic and stochastic components. This thesis aims to help bridge the gap between formal probabilistic theory of stochastic processes and empirical signal processing methods for deterministic signals by providing a spectral model for a class of nonstationary signals, whereby the deterministic and stochastic time-domain signal properties are designated respectively by the first- and second-order moments of the signal in the frequency domain. By virtue of the assumed probabilistic model, novel tests for nonstationarity detection are devised and demonstrated to be effective in low-SNR environments. The proposed spectral analysis framework, which is intrinsically complex-valued, is facilitated by augmented complex algebra in order to fully capture the joint distribution of the real and imaginary parts of complex random variables, using a compact formulation. Finally, motivated by the need for signal processing algorithms which naturally cater for the nonstationarity inherent to real-world tensors, the above contributions are employed simultaneously to derive a general statistical signal processing framework for nonstationary tensors. This is achieved by introducing a new augmented complex multilinear algebra which allows for a concise description of the multilinear interactions between the real and imaginary parts of complex tensors. These contributions are further supported by new physically meaningful empirical results on the statistical analysis of nonstationary global atmospheric temperatures.Open Acces

Gradient Flow Based Matrix Joint Diagonalization for Independent Componenet Analysis

In this thesis, employing the theory of matrix Lie groups, we develop gradient based flows for the problem of Simultaneous or Joint Diagonalization (JD) of a set of symmetric matrices. This problem has applications in many fields especially in the field of Independent Component Analysis (ICA). We consider both orthogonal and non-orthogonal JD. We view the JD problem as minimization of a common quadric cost function on a matrix group. We derive gradient based flows together with suitable discretizations for minimization of this cost function on the Riemannian manifolds of O(n) and GL(n).\\ We use the developed JD methods to introduce a new class of ICA algorithms that sphere the data, however do not restrict the subsequent search for the un-mixing matrix to orthogonal matrices. These methods provide robust ICA algorithms in Gaussian noise by making effective use of both second and higher order statistics

Linear Transmit-Receive Strategies for Multi-user MIMO Wireless Communications

Die Notwendigkeit zur Unterdrueckung von Interferenzen auf der einen Seite und zur Ausnutzung der durch Mehrfachzugriffsverfahren erzielbaren Gewinne auf der anderen Seite rueckte die raeumlichen Mehrfachzugriffsverfahren (Space Division Multiple Access, SDMA) in den Fokus der Forschung. Ein Vertreter der raeumlichen Mehrfachzugriffsverfahren, die lineare Vorkodierung, fand aufgrund steigender Anzahl an Nutzern und Antennen in heutigen und zukuenftigen Mobilkommunikationssystemen besondere Beachtung, da diese Verfahren das Design von Algorithmen zur Vorcodierung vereinfachen. Aus diesem Grund leistet diese Dissertation einen Beitrag zur Entwicklung linearer Sende- und Empfangstechniken fuer MIMO-Technologie mit mehreren Nutzern. Zunaechst stellen wir ein Framework zur Approximation des Datendurchsatzes in Broadcast-MIMO-Kanaelen mit mehreren Nutzern vor. In diesem Framework nehmen wir das lineare Vorkodierverfahren regularisierte Blockdiagonalisierung (RBD) an. Durch den Vergleich von Dirty Paper Coding (DPC) und linearen Vorkodieralgorithmen (z.B. Zero Forcing (ZF) und Blockdiagonalisierung (BD)) ist es uns moeglich, untere und obere Schranken fuer den Unterschied bezueglich Datenraten und bezueglich Leistung zwischen beiden anzugeben. Im Weiteren entwickeln wir einen Algorithmus fuer koordiniertes Beamforming (Coordinated Beamforming, CBF), dessen Loesung sich in geschlossener Form angeben laesst. Dieser CBF-Algorithmus basiert auf der SeDJoCo-Transformation und loest bisher vorhandene Probleme im Bereich CBF. Im Anschluss schlagen wir einen iterativen CBF-Algorithmus namens FlexCoBF (flexible coordinated beamforming) fuer MIMO-Broadcast-Kanaele mit mehreren Nutzern vor. Im Vergleich mit bis dato existierenden iterativen CBF-Algorithmen kann als vielversprechendster Vorteil die freie Wahl der linearen Sende- und Empfangsstrategie herausgestellt werden. Das heisst, jede existierende Methode der linearen Vorkodierung kann als Sendestrategie genutzt werden, waehrend die Strategie zum Empfangsbeamforming frei aus MRC oder MMSE gewaehlt werden darf. Im Hinblick auf Szenarien, in denen Mobilfunkzellen in Clustern zusammengefasst sind, erweitern wir FlexCoBF noch weiter. Hier wurde das Konzept der koordinierten Mehrpunktverbindung (Coordinated Multipoint (CoMP) transmission) integriert. Zuletzt stellen wir drei Moeglichkeiten vor, Kanalzustandsinformationen (Channel State Information, CSI) unter verschiedenen Kanalumstaenden zu erlangen. Die Qualitaet der Kanalzustandsinformationen hat einen starken Einfluss auf die Guete des Uebertragungssystems. Die durch unsere neuen Algorithmen erzielten Verbesserungen haben wir mittels numerischer Simulationen von Summenraten und Bitfehlerraten belegt.In order to combat interference and exploit large multiplexing gains of the multi-antenna systems, a particular interest in spatial division multiple access (SDMA) techniques has emerged. Linear precoding techniques, as one of the SDMA strategies, have obtained more attention due to the fact that an increasing number of users and antennas involved into the existing and future mobile communication systems requires a simplification of the precoding design. Therefore, this thesis contributes to the design of linear transmit and receive strategies for multi-user MIMO broadcast channels in a single cell and clustered multiple cells. First, we present a throughput approximation framework for multi-user MIMO broadcast channels employing regularized block diagonalization (RBD) linear precoding. Comparing dirty paper coding (DPC) and linear precoding algorithms (e.g., zero forcing (ZF) and block diagonalization (BD)), we further quantify lower and upper bounds of the rate and power offset between them as a function of the system parameters such as the number of users and antennas. Next, we develop a novel closed-form coordinated beamforming (CBF) algorithm (i.e., SeDJoCo based closed-form CBF) to solve the existing open problem of CBF. Our new algorithm can support a MIMO system with an arbitrary number of users and transmit antennas. Moreover, the application of our new algorithm is not only for CBF, but also for blind source separation (BSS), since the same mathematical model has been used in BSS application.Then, we further propose a new iterative CBF algorithm (i.e., flexible coordinated beamforming (FlexCoBF)) for multi-user MIMO broadcast channels. Compared to the existing iterative CBF algorithms, the most promising advantage of our new algorithm is that it provides freedom in the choice of the linear transmit and receive beamforming strategies, i.e., any existing linear precoding method can be chosen as the transmit strategy and the receive beamforming strategy can be flexibly chosen from MRC or MMSE receivers. Considering clustered multiple cell scenarios, we extend the FlexCoBF algorithm further and introduce the concept of the coordinated multipoint (CoMP) transmission. Finally, we present three strategies for channel state information (CSI) acquisition regarding various channel conditions and channel estimation strategies. The CSI knowledge is required at the base station in order to implement SDMA techniques. The quality of the obtained CSI heavily affects the system performance. The performance enhancement achieved by our new strategies has been demonstrated by numerical simulation results in terms of the system sum rate and the bit error rate