Blind identification of possibly under-determined convolutive MIMO systems

Blind identi¯cation of a Linear Time Invariant (LTI) Multiple-Input Multiple-Output (MIMO) system is of great importance in many applications, such as speech processing, multi-access communication, multi-sensor sonar/radar systems, and biomedical applications. The objective of blind identi¯cation for a MIMO system is to identify an unknown system, driven by Ni unobservable inputs, based on the No system outputs. We ¯rst present a novel blind approach for the identi¯cation of a over-determined (No ¸ Ni) MIMO system driven by white, mutually independent unobservable inputs. Samples of the system frequency response are obtained based on Parallel Factorization (PARAFAC) of three- or four-way tensors constructed respectively based on third- or fourth-order cross-spectra of the system outputs. We show that the information available in the higher-order spectra allows for the system response to be identi¯ed up to a constant scaling and permutation ambiguities and a linear phase ambiguity. Important features of the proposed approaches are that they do not require channel length information, need no phase unwrapping, and unlike the majority of existing methods, need no pre-whitening of the system outputs.While several methods have been proposed to blindly identify over-determined convolutive MIMO systems, very scarce results exist for under-determined (No < Ni) case, all of which refer to systems that either have some special structure, or special No, Ni values. We propose a novel approach for blind identi¯cation of under-determined convolutive MIMO systems of general dimensions. As long as min(No;Ni) ¸ 2, we can always ¯nd the appropriate order of statistics that guarantees identi¯ability of the system response within trivial ambiguities. We provide the description of the class of identi¯able MIMO systems for a certain order of statistics K, and an algorithm to reach the solution.Finally we propose a novel approach for blind identi¯cation and symbol recovery of a distributed antenna system with multiple carrier-frequency o®sets (CFO), arising due to mismatch between the oscillators of transmitters and receivers. The received base-band signal is over-sampled, and its polyphase components are used to formulate a virtual MIMO problem. By applying blind MIMO system estimation techniques, the system response is estimated and used to subsequently decouple the users and transform the multiple CFOs estimation problem into a set of independent single CFO estimation problems.Ph.D., Electrical Engineering -- Drexel University, 200

Hybrid solutions to instantaneous MIMO blind separation and decoding: narrowband, QAM and square cases

Future wireless communication systems are desired to support high data rates and high quality transmission when considering the growing multimedia applications. Increasing the channel throughput leads to the multiple input and multiple output and blind equalization techniques in recent years. Thereby blind MIMO equalization has attracted a great interest.Both system performance and computational complexities play important roles in real time communications. Reducing the computational load and providing accurate performances are the main challenges in present systems. In this thesis, a hybrid method which can provide an affordable complexity with good performance for Blind Equalization in large constellation MIMO systems is proposed first. Saving computational cost happens both in the signal sep- aration part and in signal detection part. First, based on Quadrature amplitude modulation signal characteristics, an efficient and simple nonlinear function for the Independent Compo- nent Analysis is introduced. Second, using the idea of the sphere decoding, we choose the soft information of channels in a sphere, and overcome the so- called curse of dimensionality of the Expectation Maximization (EM) algorithm and enhance the final results simultaneously. Mathematically, we demonstrate in the digital communication cases, the EM algorithm shows Newton -like convergence.Despite the widespread use of forward -error coding (FEC), most multiple input multiple output (MIMO) blind channel estimation techniques ignore its presence, and instead make the sim- plifying assumption that the transmitted symbols are uncoded. However, FEC induces code structure in the transmitted sequence that can be exploited to improve blind MIMO channel estimates. In final part of this work, we exploit the iterative channel estimation and decoding performance for blind MIMO equalization. Experiments show the improvements achievable by exploiting the existence of coding structures and that it can access the performance of a BCJR equalizer with perfect channel information in a reasonable SNR range. All results are confirmed experimentally for the example of blind equalization in block fading MIMO systems

Improving Monitoring and Diagnosis for Process Control using Independent Component Analysis

Statistical Process Control (SPC) is the general field concerned with monitoring the operation and performance of systems. SPC consists of a collection of techniques for characterizing the operation of a system using a probability distribution consistent with the system\u27s inputs and outputs. Classical SPC monitors a single variable to characterize the operation of a single machine tool or process step using tools such as Shewart charts. The traditional approach works well for simple small to medium size processes. For more complex processes a number of multivariate SPC techniques have been developed in recent decades. These advanced methods suffer from several disadvantages compared to univariate techniques: they tend to be statistically less powerful, and they tend to complicate process diagnosis when a disturbance is detected. This research introduces a general method for simplifying multivariate process monitoring in such a manner as to allow the use of traditional SPC tools while facilitating process diagnosis. Latent variable representations of complex processes are developed which directly relate disturbances with process steps or segments. The method models disturbances in the process rather than the process itself. The basic tool used is Independent Component Analysis (ICA). The methodology is illustrated on the problem of monitoring Electrical Test (E-Test) data from a semiconductor manufacturing process. Development and production data from a working semiconductor plant are used to estimate a factor model that is then used to develop univariate control charts for particular types of process disturbances. Detection and false alarm rates for data with known disturbances are given. The charts correctly detect and classify all the disturbance cases with a very low false alarm rate. A secondary contribution is the introduction of a method for performing an ICA like analysis using possibilistic data instead of probabilistic data. This technique extends the general ICA framework to apply to a broader range of uncertainty types. Further development of this technique could lead to the capability to use extremely sparse data to estimate ICA process models

Quasimodularity and large genus limits of Siegel-Veech constants

Quasimodular forms were first studied in the context of counting torus coverings. Here we show that a weighted version of these coverings with Siegel-Veech weights also provides quasimodular forms. We apply this to prove conjectures of Eskin and Zorich on the large genus limits of Masur-Veech volumes and of Siegel-Veech constants. In Part I we connect the geometric definition of Siegel-Veech constants both with a combinatorial counting problem and with intersection numbers on Hurwitz spaces. We introduce modified Siegel-Veech weights whose generating functions will later be shown to be quasimodular. Parts II and III are devoted to the study of the quasimodularity of the generating functions arising from weighted counting of torus coverings. The starting point is the theorem of Bloch and Okounkov saying that q-brackets of shifted symmetric functions are quasimodular forms. In Part II we give an expression for their growth polynomials in terms of Gaussian integrals and use this to obtain a closed formula for the generating series of cumulants that is the basis for studying large genus asymptotics. In Part III we show that the even hook-length moments of partitions are shifted symmetric polynomials and prove a formula for the q-bracket of the product of such a hook-length moment with an arbitrary shifted symmetric polynomial. This formula proves quasimodularity also for the (-2)-nd hook-length moments by extrapolation, and implies the quasimodularity of the Siegel-Veech weighted counting functions. Finally, in Part IV these results are used to give explicit generating functions for the volumes and Siegel-Veech constants in the case of the principal stratum of abelian differentials. To apply these exact formulas to the Eskin-Zorich conjectures we provide a general framework for computing the asymptotics of rapidly divergent power series.Comment: 107 pages, final version, to appear in J. of the AM

Applied stochastic eigen-analysis

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution February 2007The first part of the dissertation investigates the application of the theory of large random matrices to high-dimensional inference problems when the samples are drawn from a multivariate normal distribution. A longstanding problem in sensor array processing is addressed by designing an estimator for the number of signals in white noise that dramatically outperforms that proposed by Wax and Kailath. This methodology is extended to develop new parametric techniques for testing and estimation. Unlike techniques found in the literature, these exhibit robustness to high-dimensionality, sample size constraints and eigenvector misspecification. By interpreting the eigenvalues of the sample covariance matrix as an interacting particle system, the existence of a phase transition phenomenon in the largest (“signal”) eigenvalue is derived using heuristic arguments. This exposes a fundamental limit on the identifiability of low-level signals due to sample size constraints when using the sample eigenvalues alone. The analysis is extended to address a problem in sensor array processing, posed by Baggeroer and Cox, on the distribution of the outputs of the Capon-MVDR beamformer when the sample covariance matrix is diagonally loaded. The second part of the dissertation investigates the limiting distribution of the eigenvalues and eigenvectors of a broader class of random matrices. A powerful method is proposed that expands the reach of the theory beyond the special cases of matrices with Gaussian entries; this simultaneously establishes a framework for computational (non-commutative) “free probability” theory. The class of “algebraic” random matrices is defined and the generators of this class are specified. Algebraicity of a random matrix sequence is shown to act as a certificate of the computability of the limiting eigenvalue distribution and, for a subclass, the limiting conditional “eigenvector distribution.” The limiting moments of algebraic random matrix sequences, when they exist, are shown to satisfy a finite depth linear recursion so that they may often be efficiently enumerated in closed form. The method is applied to predict the deterioration in the quality of the sample eigenvectors of large algebraic empirical covariance matrices due to sample size constraints.I am grateful to the National Science Foundation for supporting this work via grant DMS-0411962 and the Office of Naval Research Graduate Traineeship awar

Entropy-based parametric estimation of spike train statistics

We consider the evolution of a network of neurons, focusing on the asymptotic behavior of spikes dynamics instead of membrane potential dynamics. The spike response is not sought as a deterministic response in this context, but as a conditional probability : "Reading out the code" consists of inferring such a probability. This probability is computed from empirical raster plots, by using the framework of thermodynamic formalism in ergodic theory. This gives us a parametric statistical model where the probability has the form of a Gibbs distribution. In this respect, this approach generalizes the seminal and profound work of Schneidman and collaborators. A minimal presentation of the formalism is reviewed here, while a general algorithmic estimation method is proposed yielding fast convergent implementations. It is also made explicit how several spike observables (entropy, rate, synchronizations, correlations) are given in closed-form from the parametric estimation. This paradigm does not only allow us to estimate the spike statistics, given a design choice, but also to compare different models, thus answering comparative questions about the neural code such as : "are correlations (or time synchrony or a given set of spike patterns, ..) significant with respect to rate coding only ?" A numerical validation of the method is proposed and the perspectives regarding spike-train code analysis are also discussed.Comment: 37 pages, 8 figures, submitte

Approximation and inference methods for stochastic biochemical kinetics-a tutorial review

Stochastic fluctuations of molecule numbers are ubiquitous in biological systems. Important examples include gene expression and enzymatic processes in living cells. Such systems are typically modelled as chemical reaction networks whose dynamics are governed by the chemical master equation. Despite its simple structure, no analytic solutions to the chemical master equation are known for most systems. Moreover, stochastic simulations are computationally expensive, making systematic analysis and statistical inference a challenging task. Consequently, significant effort has been spent in recent decades on the development of efficient approximation and inference methods. This article gives an introduction to basic modelling concepts as well as an overview of state of the art methods. First, we motivate and introduce deterministic and stochastic methods for modelling chemical networks, and give an overview of simulation and exact solution methods. Next, we discuss several approximation methods, including the chemical Langevin equation, the system size expansion, moment closure approximations, time-scale separation approximations and hybrid methods. We discuss their various properties and review recent advances and remaining challenges for these methods. We present a comparison of several of these methods by means of a numerical case study and highlight some of their respective advantages and disadvantages. Finally, we discuss the problem of inference from experimental data in the Bayesian framework and review recent methods developed the literature. In summary, this review gives a self-contained introduction to modelling, approximations and inference methods for stochastic chemical kinetics