Efficient independent component analysis

Independent component analysis (ICA) has been widely used for blind source separation in many fields such as brain imaging analysis, signal processing and telecommunication. Many statistical techniques based on M-estimates have been proposed for estimating the mixing matrix. Recently, several nonparametric methods have been developed, but in-depth analysis of asymptotic efficiency has not been available. We analyze ICA using semiparametric theories and propose a straightforward estimate based on the efficient score function by using B-spline approximations. The estimate is asymptotically efficient under moderate conditions and exhibits better performance than standard ICA methods in a variety of simulations.Comment: Published at http://dx.doi.org/10.1214/009053606000000939 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Quantifying identifiability in independent component analysis

We are interested in consistent estimation of the mixing matrix in the ICA model, when the error distribution is close to (but different from) Gaussian. In particular, we consider $n$ independent samples from the ICA model $X = A\epsilon$, where we assume that the coordinates of $\epsilon$ are independent and identically distributed according to a contaminated Gaussian distribution, and the amount of contamination is allowed to depend on $n$. We then investigate how the ability to consistently estimate the mixing matrix depends on the amount of contamination. Our results suggest that in an asymptotic sense, if the amount of contamination decreases at rate $1/\sqrt{n}$ or faster, then the mixing matrix is only identifiable up to transpose products. These results also have implications for causal inference from linear structural equation models with near-Gaussian additive noise.Comment: 22 pages, 2 figure

Statistical physics of independent component analysis

Statistical physics is used to investigate independent component analysis with polynomial contrast functions. While the replica method fails, an adapted cavity approach yields valid results. The learning curves, obtained in a suitable thermodynamic limit, display a first order phase transition from poor to perfect generalization.Comment: 7 pages, 1 figure, to appear in Europhys. Lett

Independent Component Analysis of Spatiotemporal Chaos

Two types of spatiotemporal chaos exhibited by ensembles of coupled nonlinear oscillators are analyzed using independent component analysis (ICA). For diffusively coupled complex Ginzburg-Landau oscillators that exhibit smooth amplitude patterns, ICA extracts localized one-humped basis vectors that reflect the characteristic hole structures of the system, and for nonlocally coupled complex Ginzburg-Landau oscillators with fractal amplitude patterns, ICA extracts localized basis vectors with characteristic gap structures. Statistics of the decomposed signals also provide insight into the complex dynamics of the spatiotemporal chaos.Comment: 5 pages, 6 figures, JPSJ Vol 74, No.

Macroeconomic Forecasting with Independent Component Analysis

This paper considers a factor model in which independent component analysis (ICA) is employed to construct common factors out of a large number of macroeconomic time series. The ICA has been regarded as a better method to separate unobserved sources that are statistically independent to each other. Two algorithms are employed to compute the independent factors. The first algorithm takes into account the kurtosis feature contained in the sample. The second algorithm accommodates the time dependence structure in the time series data. A straightforward forecasting model using the independent factors is then compared with the forecasting models using the principal components in Stock and Watson (2002). The results of this research can help us to gain more knowledge about the underlying economic sources and their impacts on the aggregate variables. The empirical findings suggest that the independent component method is a powerful method of macroeconomic data compression. Whether the ICA method is superior over the principal component method in forecasting the U.S. real output and inflation variables is however inconclusiveforecast, independent components, principal components

Heavy-tailed Independent Component Analysis

Independent component analysis (ICA) is the problem of efficiently recovering a matrix $A \in \mathbb{R}^{n\times n}$ from i.i.d. observations of $X=AS$ where $S \in \mathbb{R}^n$ is a random vector with mutually independent coordinates. This problem has been intensively studied, but all existing efficient algorithms with provable guarantees require that the coordinates $S_i$ have finite fourth moments. We consider the heavy-tailed ICA problem where we do not make this assumption, about the second moment. This problem also has received considerable attention in the applied literature. In the present work, we first give a provably efficient algorithm that works under the assumption that for constant $\gamma > 0$, each $S_i$ has finite $(1+\gamma)$-moment, thus substantially weakening the moment requirement condition for the ICA problem to be solvable. We then give an algorithm that works under the assumption that matrix $A$ has orthogonal columns but requires no moment assumptions. Our techniques draw ideas from convex geometry and exploit standard properties of the multivariate spherical Gaussian distribution in a novel way.Comment: 30 page