Magnification Control in Winner Relaxing Neural Gas

An important goal in neural map learning, which can conveniently be accomplished by magnification control, is to achieve information optimal coding in the sense of information theory. In the present contribution we consider the winner relaxing approach for the neural gas network. Originally, winner relaxing learning is a slight modification of the self-organizing map learning rule that allows for adjustment of the magnification behavior by an a priori chosen control parameter. We transfer this approach to the neural gas algorithm. The magnification exponent can be calculated analytically for arbitrary dimension from a continuum theory, and the entropy of the resulting map is studied numerically conf irming the theoretical prediction. The influence of a diagonal term, which can be added without impacting the magnification, is studied numerically. This approach to maps of maximal mutual information is interesting for applications as the winner relaxing term only adds computational cost of same order and is easy to implement. In particular, it is not necessary to estimate the generally unknown data probability density as in other magnification control approaches.Comment: 14pages, 2 figure

Multidimensional Urban Segregation - Toward A Neural Network Measure

We introduce a multidimensional, neural-network approach to reveal and measure urban segregation phenomena, based on the Self-Organizing Map algorithm (SOM). The multidimensionality of SOM allows one to apprehend a large number of variables simultaneously, defined on census or other types of statistical blocks, and to perform clustering along them. Levels of segregation are then measured through correlations between distances on the neural network and distances on the actual geographical map. Further, the stochasticity of SOM enables one to quantify levels of heterogeneity across census blocks. We illustrate this new method on data available for the city of Paris.Comment: NCAA S.I. WSOM+ 201

Magnification Control in Self-Organizing Maps and Neural Gas

We consider different ways to control the magnification in self-organizing maps (SOM) and neural gas (NG). Starting from early approaches of magnification control in vector quantization, we then concentrate on different approaches for SOM and NG. We show that three structurally similar approaches can be applied to both algorithms: localized learning, concave-convex learning, and winner relaxing learning. Thereby, the approach of concave-convex learning in SOM is extended to a more general description, whereas the concave-convex learning for NG is new. In general, the control mechanisms generate only slightly different behavior comparing both neural algorithms. However, we emphasize that the NG results are valid for any data dimension, whereas in the SOM case the results hold only for the one-dimensional case.Comment: 24 pages, 4 figure

Forecasting the CATS benchmark with the Double Vector Quantization method

The Double Vector Quantization method, a long-term forecasting method based on the SOM algorithm, has been used to predict the 100 missing values of the CATS competition data set. An analysis of the proposed time series is provided to estimate the dimension of the auto-regressive part of this nonlinear auto-regressive forecasting method. Based on this analysis experimental results using the Double Vector Quantization (DVQ) method are presented and discussed. As one of the features of the DVQ method is its ability to predict scalars as well as vectors of values, the number of iterative predictions needed to reach the prediction horizon is further observed. The method stability for the long term allows obtaining reliable values for a rather long-term forecasting horizon.Comment: Accepted for publication in Neurocomputing, Elsevie

Reducing Catastrophic Forgetting in Self-Organizing Maps

An agent that is capable of continual or lifelong learning is able to continuously learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents capable of such learning is that neural systems struggle to retain previously-acquired knowledge when learning from new data samples. This problem is known as catastrophic forgetting and remains an unsolved problem in the domain of machine learning to this day. To overcome catastrophic forgetting, different approaches have been proposed. One major line of thought advocates the use of memory buffers to store data where the stored data is then used to randomly retrain the model to improve memory retention. However, storing and giving access to previous physical data points results in a variety of practical difficulties particularly with respect to growing memory storage costs. In this work, we propose an alternative way to tackle the problem of catastrophic forgetting, inspired by and building on top of a classical neural model, the self-organizing map (SOM) which is a form of unsupervised clustering. Although the SOM has the potential to combat forgetting through the use of pattern-specializing units, we uncover that it too suffers from the same problem and this forgetting becomes worse when the SOM is trained in a task incremental fashion. To mitigate this, we propose a generalization of the SOM, the continual SOM (c-SOM), which introduces several novel mechanisms to improve its memory retention -- new decay functions and generative resampling schemes to facilitate generative replay in the model. We perform extensive experiments using split-MNIST with these approaches, demonstrating that the c-SOM significantly improves over the classical SOM. Additionally, we come up with a new performance metric alpha_mem to measure the efficacy of SOMs trained in a task incremental fashion, providing a benchmark for other competitive learning models

Study of the magnification effect on self-organizing maps

Self-Organizing Maps (SOM), are a type of neuronal network (Kohonen, 1982b) that has been used mainly in data clustering problems, using unsupervised learning. Among the multiple areas of application, SOM has been used in various problems of direct interest to the Navy (V. J. Lobo, 2009), including route planning and the location of critical infrastructures. The SOM has also been used to sample large databases. In this sort of application, they have a behaviour called the magnification effect (Bauer & Der, 1996), which causes areas of the attribute space of data with less density to be overrepresented or magnified. This dissertation uses an experimental approach to mitigate the lack of theoretical explanation for this effect except for one-dimensional and quite simple cases. From experimental evidence obtained for carefully designed problems we infer a relationship between input data densities and output neuron densities that can be applied universally, or at least in a broad set of situations. A large number of experiments were conducted using one-dimensional to one-dimensional mappings followed by 2D to 2D, 3D to 1, 2 and 3D. We derived an empirical relationship whereby the density in the output space is equal to a constant times the density of the input space raised to the power of (alpha) which although depending on a number of factors can be approximated by the root index n of 2/3 where n is the input space dimension. The correlation that we found in our experiments, for both the well-known 1- dimensional case and for more general 2 to 3-dimensional cases is a useful guide to predict the magnification effect in practical situations.Therefore, in chapter 4 we produce a populational cartogram of Angola and we prove that our relation can be used to correct the magnification effect on 2-dimensional cases.Os mapas auto-organizados ou SOM (Self Organizing Maps), são um tipo de rede neuronal (Kohonen, 1982) que tem sido utilizada sobretudo em problemas agrupamento de dados (clustering), usando aprendizagem não supervisionada. Entre as múltiplas áreas de aplicação, os SOM têm sido usados em vários problemas com interesse direto para a Marinha (Lobo, 2009), incluindo o planeamento de rotas e a localização de infraestruturas críticas. Os SOM também têm sido usados para fazer amostragem de grandes bases de dados, e nesse tipo de aplicações têm um comportamento, denominado efeito de magnificação (Bauer & R. Der, 1996), que faz com que zonas do espaço de atributos dos dados com menor densidade sejam sobre representadas, ou seja magnificadas. Esta dissertação traz uma abordagem experimental para mitigar a falta de explicação teórica para este efeito, com exceção de casos unidimensionais e bastante simples. A partir de provas experimentais obtidas para problemas cuidadosamente concebidos, inferimos uma relação entre densidades de dados de entrada e densidades de neuronios à saída que podem ser aplicadas universalmente, ou pelo menos num conjunto alargado de situações. Foram realizadas um grande numero de experiências usando mapeamentos unidimensionais para mapeamentos unidimensionais seguidos por 2D para 2D, 3D para 1, 2 e 3D. Derivamos uma relação empírica em que a densidade no espaço de saída é igual a uma constante vezes a densidade do espaço de entrada elevada a (alpha) que, embora dependendo de uma série de fatores, pode ser aproximado pela raiz de índice n de 2/3 onde n é a dimensão do espaço de entrada. A correlação que encontramos nas nossas experiências, tanto para o caso unidimensional bem como para casos mais gerais de 2 a 3 dimensões é um guia útil para prever o efeito de magnificação em situações práticas. No capítulo 4 produzimos um cartograma populacional de Angola e provamos que a nossa relação pode ser usada para corrigir o efeito de magnificação em casos bidimensionais

Medical imaging analysis with artificial neural networks

Given that neural networks have been widely reported in the research community of medical imaging, we provide a focused literature survey on recent neural network developments in computer-aided diagnosis, medical image segmentation and edge detection towards visual content analysis, and medical image registration for its pre-processing and post-processing, with the aims of increasing awareness of how neural networks can be applied to these areas and to provide a foundation for further research and practical development. Representative techniques and algorithms are explained in detail to provide inspiring examples illustrating: (i) how a known neural network with fixed structure and training procedure could be applied to resolve a medical imaging problem; (ii) how medical images could be analysed, processed, and characterised by neural networks; and (iii) how neural networks could be expanded further to resolve problems relevant to medical imaging. In the concluding section, a highlight of comparisons among many neural network applications is included to provide a global view on computational intelligence with neural networks in medical imaging

Performance and storage requirements of topology-conserving maps for robot manipulator control

A new programming paradigm for the control of a robot manipulator by learning the mapping between the Cartesian space and the joint space (inverse Kinematic) is discussed. It is based on a Neural Network model of optimal mapping between two high-dimensional spaces by Kohonen. This paper describes the approach and presents the optimal mapping, based on the principle of maximal information gain. It is shown that Kohonens mapping in the 2-dimensional case is optimal in this sense. Furthermore, the principal control error made by the learned mapping is evaluated for the example of the commonly used PUMA robot, the trade-off between storage resources and positional error is discussed and an optimal position encoding resolution is proposed

Visualization of clusters in geo-referenced data using three-dimensional self-organizing maps

Dissertação apresentada como requisito parcial para obtenção do grau de Mestre em Estatística e Gestão de InformaçãoThe Self-Organizing Map (SOM) is an artificial neural network that performs simultaneously vector quantization and vector projection. Due to this characteristic, the SOM is an effective method for clustering analysis via visualization. The SOM can be visualized through the output space, generally a regular two-dimensional grid of nodes, and through the input space, emphasizing the vector quantization process. Among all the strategies for visualizing the SOM, we are particularly interested in those that allow dealing with spatial dependency, linking the SOM to the geographic visualization with color. One possible approach, commonly used, is the cartographic representation of data with label colors defined from the output space of a two-dimensional SOM. However, in the particular case of geo-referenced data, it is possible to consider the use of a three-dimensional SOM for this purpose, thus adding one more dimension in the analysis. In this dissertation is presented a method for clustering geo-referenced data that integrates the visualization of both perspectives of a three dimensional SOM: linking its output space to the cartographic representation through a ordered set of colors; and exploring the use of frontiers among geo-referenced elements, computed according to the distances in the input space between their Best Matching Units