Adaptive blind signal separation.

by Chi-Chiu Cheung.Thesis (M.Phil.)--Chinese University of Hong Kong, 1997.Includes bibliographical references (leaves 124-131).Abstract --- p.iAcknowledgments --- p.iiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- The Blind Signal Separation Problem --- p.1Chapter 1.2 --- Contributions of this Thesis --- p.3Chapter 1.3 --- Applications of the Problem --- p.4Chapter 1.4 --- Organization of the Thesis --- p.5Chapter 2 --- The Blind Signal Separation Problem --- p.7Chapter 2.1 --- The General Blind Signal Separation Problem --- p.7Chapter 2.2 --- Convolutive Linear Mixing Process --- p.8Chapter 2.3 --- Instantaneous Linear Mixing Process --- p.9Chapter 2.4 --- Problem Definition and Assumptions in this Thesis --- p.9Chapter 3 --- Literature Review --- p.13Chapter 3.1 --- Previous Works on Blind Signal Separation with Instantaneous Mixture --- p.13Chapter 3.1.1 --- Algebraic Approaches --- p.14Chapter 3.1.2 --- Neural approaches --- p.15Chapter 3.2 --- Previous Works on Blind Signal Separation with Convolutive Mixture --- p.20Chapter 4 --- The Information-theoretic ICA Scheme --- p.22Chapter 4.1 --- The Bayesian YING-YANG Learning Scheme --- p.22Chapter 4.2 --- The Information-theoretic ICA Scheme --- p.25Chapter 4.2.1 --- Derivation of the cost function from YING-YANG Machine --- p.25Chapter 4.2.2 --- Connections to previous information-theoretic approaches --- p.26Chapter 4.2.3 --- Derivation of the Algorithms --- p.27Chapter 4.2.4 --- Roles and Constraints on the Nonlinearities --- p.30Chapter 4.3 --- Direction and Motivation for the Analysis of the Nonlinearity --- p.30Chapter 5 --- Properties of the Cost Function and the Algorithms --- p.32Chapter 5.1 --- Lemmas and Corollaries --- p.32Chapter 5.1.1 --- Singularity of J(V) --- p.33Chapter 5.1.2 --- Continuity of J(V) --- p.34Chapter 5.1.3 --- Behavior of J(V) along a radially outward line --- p.35Chapter 5.1.4 --- Impossibility of divergence of the information-theoretic ICA al- gorithms with a large class of nonlinearities --- p.36Chapter 5.1.5 --- Number and stability of correct solutions in the 2-channel case --- p.37Chapter 5.1.6 --- Scale for the equilibrium points --- p.39Chapter 5.1.7 --- Absence of local maximum of J(V) --- p.43Chapter 6 --- The Algorithms with Cubic Nonlinearity --- p.44Chapter 6.1 --- The Cubic Nonlinearity --- p.44Chapter 6.2 --- Theoretical Results on the 2-Channel Case --- p.46Chapter 6.2.1 --- Equilibrium points --- p.46Chapter 6.2.2 --- Stability of the equilibrium points --- p.49Chapter 6.2.3 --- An alternative proof for the stability of the equilibrium points --- p.50Chapter 6.2.4 --- Convergence Analysis --- p.52Chapter 6.3 --- Experiments on the 2-Channel Case --- p.53Chapter 6.3.1 --- Experiments on two sub-Gaussian sources --- p.54Chapter 6.3.2 --- Experiments on two super-Gaussian sources --- p.55Chapter 6.3.3 --- Experiments on one super-Gaussian source and one sub-Gaussian source which are globally sub-Gaussian --- p.57Chapter 6.3.4 --- Experiments on one super-Gaussian source and one sub-Gaussian source which are globally super-Gaussian --- p.59Chapter 6.3.5 --- Experiments on asymmetric exponentially distributed signals .。 --- p.60Chapter 6.3.6 --- Demonstration on exactly and nearly singular initial points --- p.61Chapter 6.4 --- Theoretical Results on the 3-Channel Case --- p.63Chapter 6.4.1 --- Equilibrium points --- p.63Chapter 6.4.2 --- Stability --- p.66Chapter 6.5 --- Experiments on the 3-Channel Case --- p.66Chapter 6.5.1 --- Experiments on three pairwise globally sub-Gaussian sources --- p.67Chapter 6.5.2 --- Experiments on three sources consisting of globally sub-Gaussian and globally super-Gaussian pairs --- p.67Chapter 6.5.3 --- Experiments on three pairwise globally super-Gaussian sources --- p.69Chapter 7 --- Nonlinearity and Separation Capability --- p.71Chapter 7.1 --- Theoretical Argument --- p.71Chapter 7.1.1 --- Nonlinearities that strictly match the source distribution --- p.72Chapter 7.1.2 --- Nonlinearities that loosely match the source distribution --- p.72Chapter 7.2 --- Experiment Verification --- p.76Chapter 7.2.1 --- Experiments on reversed sigmoid --- p.76Chapter 7.2.2 --- Experiments on the cubic root nonlinearity --- p.77Chapter 7.2.3 --- Experimental verification of Theorem 2 --- p.77Chapter 7.2.4 --- Experiments on the MMI algorithm --- p.78Chapter 8 --- Implementation with Mixture of Densities --- p.80Chapter 8.1 --- Implementation of the Information-theoretic ICA scheme with Mixture of Densities --- p.80Chapter 8.1.1 --- The mixture of densities --- p.81Chapter 8.1.2 --- Derivation of the algorithms --- p.82Chapter 8.2 --- Experimental Verification on the Nonlinearity Adaptation --- p.84Chapter 8.2.1 --- Experiment 1: Two channels of sub-Gaussian sources --- p.84Chapter 8.2.2 --- Experiment 2: Two channels of super-Gaussian sources --- p.85Chapter 8.2.3 --- Experiment 3: Three channels of different signals --- p.89Chapter 8.3 --- Seeking the Simplest Workable Mixtures of Densities ......... .。 --- p.91Chapter 8.3.1 --- Number of components --- p.91Chapter 8.3.2 --- Mixture of two densities with only biases changeable --- p.93Chapter 9 --- ICA with Non-Kullback Cost Function --- p.97Chapter 9.1 --- Derivation of ICA Algorithms from Non-Kullback Separation Functionals --- p.97Chapter 9.1.1 --- Positive Convex Divergence --- p.97Chapter 9.1.2 --- Lp Divergence --- p.100Chapter 9.1.3 --- De-correlation Index --- p.102Chapter 9.2 --- Experiments on the ICA Algorithm Based on Positive Convex Divergence --- p.103Chapter 9.2.1 --- Experiments on the algorithm with fixed nonlinearities --- p.103Chapter 9.2.2 --- Experiments on the algorithm with mixture of densities --- p.106Chapter 10 --- Conclusions --- p.107Chapter A --- Proof for Stability of the Equilibrium Points of the Algorithm with Cubic Nonlinearity on Two Channels of Signals --- p.110Chapter A.1 --- Stability of Solution Group A --- p.110Chapter A.2 --- Stability of Solution Group B --- p.111Chapter B --- Proof for Stability of the Equilibrium Points of the Algorithm with Cubic Nonlinearity on Three Channels of Signals --- p.119Chapter C --- Proof for Theorem2 --- p.122Bibliography --- p.12

Separation of Synchronous Sources

This thesis studies the Separation of Synchronous Sources (SSS) problem, which deals with the separation of signals resulting from a linear mixing of sources whose phases are synchronous. While this study is made in a form independent of the application, a motivation from a neuroscience perspective is presented. Traditional methods for Blind Source Separation, such as Independent Component Analysis (ICA), cannot address this problem because synchronous sources are highly dependent. We provide sufficient conditions for SSS to be an identifiable problem, and quantify the effect of prewhitening on the difficulty of SSS. We also present two algorithms to solve SSS. Extensive studies on simulated data illustrate that these algorithms yield substantially better results when compared with ICA methods. We conclude that these algorithms can successfully perform SSS in varying configurations (number of sources, number of sensors, level of additive noise, phase lag between sources, among others). Theoretical properties of one of these algorithms are also presented. Future work is discussed extensively, showing that this area of study is far from resolved and still presents interesting challenges

Object-based audio capture : separating acoustically-mixed sounds

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 1999.Includes bibliographical references (p. 111-114).This thesis investigates how a digital system can recognize and isolate individual sound sources, or audio objects, from an environment containing several sounds. The main contribution of this work is the application of object-based audio capture to unconstrained real-world environments. Several potential applications for object-based audio capture are outlined, and current blind source separation and deconvolution (BSSD) algorithms that have been applied to acoustically-mixed sounds are reviewed. An explanation of the acoustics issues in object-based audio capture is provided, including an argument for using overdetermined mixtures to yield better source separation. A thorough discussion of the difficulties imposed by a real-world environment is offered, followed by several experiments which compare how different filter configurations and filter lengths, as well as reverberant environments, all have an impact on the performance of object-based audio capture. A real-world implementation of object-based audio capture in a conference room with two people speaking is also discussed. This thesis concludes with future directions for research in object-based audio capture.Alexander George Westner.S.M

Flexible methods for blind separation of complex signals

One of the main matter in Blind Source Separation (BSS) performed with a neural network approach is the choice of the nonlinear activation function (AF). In fact if the shape of the activation function is chosen as the cumulative density function (c.d.f.) of the original source the problem is solved. For this scope in this thesis a flexible approach is introduced and the shape of the activation functions is changed during the learning process using the so-called “spline functions”. The problem is complicated in the case of separation of complex sources where there is the problem of the dichotomy between analyticity and boundedness of the complex activation functions. The problem is solved introducing the “splitting function” model as activation function. The “splitting function” is a couple of “spline function” which wind off the real and the imaginary part of the complex activation function, each of one depending from the real and imaginary variable. A more realistic model is the “generalized splitting function”, which is formed by a couple of two bi-dimensional functions (surfaces), one for the real and one for the imaginary part of the complex function, each depending by both the real and imaginary part of the complex variable. Unfortunately the linear environment is unrealistic in many practical applications. In this way there is the need of extending BSS problem in the nonlinear environment: in this case both the activation function than the nonlinear distorting function are realized by the “splitting function” made of “spline function”. The complex and instantaneous separation in linear and nonlinear environment allow us to perform a complex-valued extension of the well-known INFOMAX algorithm in several practical situations, such as convolutive mixtures, fMRI signal analysis and bandpass signal transmission. In addition advanced characteristics on the proposed approach are introduced and deeply described. First of all it is shows as splines are universal nonlinear functions for BSS problem: they are able to perform separation in anyway. Then it is analyzed as the “splitting solution” allows the algorithm to obtain a phase recovery: usually there is a phase ambiguity. Finally a Cramér-Rao lower bound for ICA is discussed. Several experimental results, tested by different objective indexes, show the effectiveness of the proposed approaches

Flexible methods for blind separation of complex signals

One of the main matter in Blind Source Separation (BSS) performed with a neural network approach is the choice of the nonlinear activation function (AF). In fact if the shape of the activation function is chosen as the cumulative density function (c.d.f.) of the original source the problem is solved. For this scope in this thesis a flexible approach is introduced and the shape of the activation functions is changed during the learning process using the so-called “spline functions”. The problem is complicated in the case of separation of complex sources where there is the problem of the dichotomy between analyticity and boundedness of the complex activation functions. The problem is solved introducing the “splitting function” model as activation function. The “splitting function” is a couple of “spline function” which wind off the real and the imaginary part of the complex activation function, each of one depending from the real and imaginary variable. A more realistic model is the “generalized splitting function”, which is formed by a couple of two bi-dimensional functions (surfaces), one for the real and one for the imaginary part of the complex function, each depending by both the real and imaginary part of the complex variable. Unfortunately the linear environment is unrealistic in many practical applications. In this way there is the need of extending BSS problem in the nonlinear environment: in this case both the activation function than the nonlinear distorting function are realized by the “splitting function” made of “spline function”. The complex and instantaneous separation in linear and nonlinear environment allow us to perform a complex-valued extension of the well-known INFOMAX algorithm in several practical situations, such as convolutive mixtures, fMRI signal analysis and bandpass signal transmission. In addition advanced characteristics on the proposed approach are introduced and deeply described. First of all it is shows as splines are universal nonlinear functions for BSS problem: they are able to perform separation in anyway. Then it is analyzed as the “splitting solution” allows the algorithm to obtain a phase recovery: usually there is a phase ambiguity. Finally a Cramér-Rao lower bound for ICA is discussed. Several experimental results, tested by different objective indexes, show the effectiveness of the proposed approaches

Efficient Multiband Algorithms for Blind Source Separation

The problem of blind separation refers to recovering original signals, called source signals, from the mixed signals, called observation signals, in a reverberant environment. The mixture is a function of a sequence of original speech signals mixed in a reverberant room. The objective is to separate mixed signals to obtain the original signals without degradation and without prior information of the features of the sources. The strategy used to achieve this objective is to use multiple bands that work at a lower rate, have less computational cost and a quicker convergence than the conventional scheme. Our motivation is the competitive results of unequal-passbands scheme applications, in terms of the convergence speed. The objective of this research is to improve unequal-passbands schemes by improving the speed of convergence and reducing the computational cost. The first proposed work is a novel maximally decimated unequal-passbands scheme.This scheme uses multiple bands that make it work at a reduced sampling rate, and low computational cost. An adaptation approach is derived with an adaptation step that improved the convergence speed. The performance of the proposed scheme was measured in different ways. First, the mean square errors of various bands are measured and the results are compared to a maximally decimated equal-passbands scheme, which is currently the best performing method. The results show that the proposed scheme has a faster convergence rate than the maximally decimated equal-passbands scheme. Second, when the scheme is tested for white and coloured inputs using a low number of bands, it does not yield good results; but when the number of bands is increased, the speed of convergence is enhanced. Third, the scheme is tested for quick changes. It is shown that the performance of the proposed scheme is similar to that of the equal-passbands scheme. Fourth, the scheme is also tested in a stationary state. The experimental results confirm the theoretical work. For more challenging scenarios, an unequal-passbands scheme with over-sampled decimation is proposed; the greater number of bands, the more efficient the separation. The results are compared to the currently best performing method. Second, an experimental comparison is made between the proposed multiband scheme and the conventional scheme. The results show that the convergence speed and the signal-to-interference ratio of the proposed scheme are higher than that of the conventional scheme, and the computation cost is lower than that of the conventional scheme

Surprise and error: Common neuronal architecture for the processing of errors and novelty

According to recent accounts, the processing of errors and generally infrequent, surprising (novel) events share a common neuroanat-omical substrate. Direct empirical evidence for this common processing network in humans is, however, scarce. To test this hypothesis, we administered a hybrid error-monitoring/novelty-oddball task in which the frequency of novel, surprising trials was dynamically matched to the frequency of errors. Using scalp electroencephalographic recordings and event-related functional magnetic resonance imaging (fMRI), we compared neural responses to errors with neural responses to novel events. In Experiment 1, independent component analysis of scalp ERP data revealed a common neural generator implicated in the generation of both the error-related negativity (ERN) and the novelty-related frontocentral N2. In Experiment 2, this pattern was confirmed by a conjunction analysis of event-related fMRI, which showed significantly elevated BOLD activity following both types of trials in the posterior medial frontal cortex, including the anterior midcingulate cortex (aMCC), the neuronal generator of the ERN. Together, these findings provide direct evidence of a common neural system underlying the processing of errors and novel events. This appears to be at odds with prominent theories of the ERN and aMCC. Inparticular, there inforcement learning theory of the ERN may need to be modified because it may not suffice as a fully integrative model of aMCC function. Whenever course and outcome of anaction violates expectancies (not necessarily related to reward), the aMCC seems to be engaged in evaluating the necessity of behavioral adaptation. © 2012 the authors