Design space reduction in optimization using generative topographic mapping

Dimension reduction in design optimization is an extensively researched area. The need arises in design problems dealing with very high dimensions, which increase the computational burden of the design process because the sample space required for the design search varies exponentially with the dimensions. This work describes the application of a latent variable method called Generative Topographic Mapping (GTM) in dimension reduction of a data set by transformation into a low-dimensional latent space. The attraction it presents is that the variables are not removed, but only transformed and hence there is no risk of missing out on information relating to all the variables. The method has been tested on the Branin test function initially and then on an aircraft wing weight problem. Ongoing work involves finding a suitable update strategy for adding infill points to the trained GTM in order to converge to the global optimum effectively. Three update methods tested on GTM so far are discussed

Compositional generative mapping for tree-structured data - Part II: Topographic projection model

We introduce GTM-SD (Generative Topographic Mapping for Structured Data), which is the first compositional generative model for topographic mapping of tree-structured data. GTM-SD exploits a scalable bottom-up hidden-tree Markov model that was introduced in Part I of this paper to achieve a recursive topographic mapping of hierarchical information. The proposed model allows efficient exploitation of contextual information from shared substructures by a recursive upward propagation on the tree structure which distributes substructure information across the topographic map. Compared to its noncompositional generative counterpart, GTM-SD is shown to allow the topographic mapping of the full sample tree, which includes a projection onto the lattice of all the distinct subtrees rooted in each of its nodes. Experimental results show that the continuous projection space generated by the smooth topographic mapping of GTM-SD yields a finer grained discrimination of the sample structures with respect to the state-of-the-art recursive neural network approach

The effect of noise and sample size on an unsupervised feature selection method for manifold learning

The research on unsupervised feature selection is scarce in comparison to that for supervised models, despite the fact that this is an important issue for many clustering problems. An unsupervised feature selection method for general Finite Mixture Models was recently proposed and subsequently extended to Generative Topographic Mapping (GTM), a manifold learning constrained mixture model that provides data visualization. Some of the results of a previous partial assessment of this unsupervised feature selection method for GTM suggested that its performance may be affected by insufficient sample size and by noisy data. In this brief study, we test in some detail such limitations of the method.Postprint (published version

Clustering with alternative similarity functions

We [6, 7] have recently investigated several families of clustering algorithms. In this paper, we show how a novel similarity function can be integrated into one of our algorithms as a method of performing clustering and show that the resulting method is superior to existing methods in that it canbe shown to reliably find a globally optimal clustering rather than local optima which other methods often find. We also extend the method to perform topology preserving mappings and show the results of such mappings on artificial and real data

Block GTM: Incorporating prior knowledge of covariance structure in data visualisation

Visualising data for exploratory analysis is a big challenge in scientific and engineering domains where there is a need to gain insight into the structure and distribution of the data. Typically, visualisation methods like principal component analysis and multi-dimensional scaling are used, but it is difficult to incorporate prior knowledge about structure of the data into the analysis. In this technical report we discuss a complementary approach based on an extension of a well known non-linear probabilistic model, the Generative Topographic Mapping. We show that by including prior information of the covariance structure into the model, we are able to improve both the data visualisation and the model fit

Data visualisation and exploration with prior knowledge

Visualising data for exploratory analysis is a major challenge in many applications. Visualisation allows scientists to gain insight into the structure and distribution of the data, for example finding common patterns and relationships between samples as well as variables. Typically, visualisation methods like principal component analysis and multi-dimensional scaling are employed. These methods are favoured because of their simplicity, but they cannot cope with missing data and it is difficult to incorporate prior knowledge about properties of the variable space into the analysis; this is particularly important in the high-dimensional, sparse datasets typical in geochemistry. In this paper we show how to utilise a block-structured correlation matrix using a modification of a well known non-linear probabilistic visualisation model, the Generative Topographic Mapping (GTM), which can cope with missing data. The block structure supports direct modelling of strongly correlated variables. We show that including prior structural information it is possible to improve both the data visualisation and the model fit. These benefits are demonstrated on artificial data as well as a real geochemical dataset used for oil exploration, where the proposed modifications improved the missing data imputation results by 3 to 13%