Missing data imputation through generative topographic mapping as a mixture of t-distributions: Theoretical developments

The Generative Topographic Mapping (GTM) was originally conceived as a probabilistic alternative to the well-known, neural network-inspired, Self-Organizing Map (SOM). The GTM can also be interpreted as a constrained mixture of distributions model. In recent years, much attention has been directed towards Student t-distributions as an alternative to Gaussians in mixture models due to their robustness towards outliers. In this report, the GTM is redefined as a constrained mixture of t-distributions: the t-GTM, and the Expectation-Maximization algorithm that is used to fit the model to the data is modified to provide missing data imputation.Postprint (published version

Multiple Resolution Nonparametric Classifiers

Bayesian discriminant functions provide optimal classification decision boundaries in the sense of minimizing the average error rate. An operational assumption is that the probability density functions for the individual classes are either known a priori or can be estimated from the data through the use of estimating techniques. The use of Parzen- windows is a popular and theoretically sound choice for such estimation. However, while the minimal average error rate can be achieved when combining Bayes Rule with Parzen-window density estimation, the latter is computationally costly to the point where it may lead to unacceptable run-time performance. We present the Multiple Resolution Nonparametric (MRN) classifier as a new approach for significantly reducing the computational cost of using Parzen-window density estimates without sacrificing the virtues of Bayesian discriminant functions. Performance is evaluated against a standard Parzen-window classifier on several common datasets

Connectionist-Symbolic Machine Intelligence using Cellular Automata based Reservoir-Hyperdimensional Computing

We introduce a novel framework of reservoir computing, that is capable of both connectionist machine intelligence and symbolic computation. Cellular automaton is used as the reservoir of dynamical systems. Input is randomly projected onto the initial conditions of automaton cells and nonlinear computation is performed on the input via application of a rule in the automaton for a period of time. The evolution of the automaton creates a space-time volume of the automaton state space, and it is used as the reservoir. The proposed framework is capable of long short-term memory and it requires orders of magnitude less computation compared to Echo State Networks. We prove that cellular automaton reservoir holds a distributed representation of attribute statistics, which provides a more effective computation than local representation. It is possible to estimate the kernel for linear cellular automata via metric learning, that enables a much more efficient distance computation in support vector machine framework. Also, binary reservoir feature vectors can be combined using Boolean operations as in hyperdimensional computing, paving a direct way for concept building and symbolic processing.Comment: Corrected Typos. Responded some comments on section 8. Added appendix for details. Recurrent architecture emphasize

Stochastic model reduction for robust dynamical characterization of structures with random parameters

International audienceIn this paper, we characterize random eigenspaces with a non-intrusive method based on the decoupling of random eigenvalues from their corresponding random eigenvectors. This method allows us to estimate the first statistical moments of the random eigenvalues of the system with a reduced number of deterministic finite element computations. The originality of this work is to adapt the method used to estimate each random eigenvalue depending on a global accuracy requirement. This allows us to ensure a minimal computational cost. The stochastic model of the structure is thus reduced by exploiting specific properties of random eigenvectors associated with the random eigenfrequencies being sought. An indicator with no additional computation cost is proposed to identify when the method needs to be enhanced. Finally, a simple three-beam frame and an industrial structure illustrate the proposed approach

Flexible and Robust Bayesian Classification by Finite Mixture Models

The regularized Mahalanobis distance is proposed in the framework of nite mixture models to avoid commonly faced numerical difficulties encountered with EM. Its principle is applied to Gaussian and Student-t mixtures, resulting in reliable density estimates, the model complexity being kept low. Besides, the regularized models are robust to various noise types. Finally, it is shown that the quality of the associated Bayesian classication is near optimal on Ripley's synthetic data set

Flexible and Robust Bayesian Classification by Finite Mixture Models

The regularized Mahalanobis distance is proposed in the framework of finite mixture models to avoid commonly faced numerical difficulties encountered with EM. Its principle is applied to Gaussian and Student-t mixtures, resulting in reliable density estimates, the model complexity being kept low. Besides, the regularized models are robust to various noise types. Finally, it is shown..