15,992 research outputs found
Multiorder neurons for evolutionary higher-order clustering and growth
This letter proposes to use multiorder neurons for clustering irregularly shaped data arrangements. Multiorder neurons are an evolutionary extension of the use of higher-order neurons in clustering. Higher-order neurons parametrically model complex neuron shapes by replacing the classic synaptic weight by higher-order tensors. The multiorder neuron goes one step further and eliminates two problems associated with higher-order neurons. First, it uses evolutionary algorithms to select the best neuron order for a given problem. Second, it obtains more information about the underlying data distribution by identifying the correct order for a given cluster of patterns. Empirically we observed that when the correlation of clusters found with ground truth information is used in measuring clustering accuracy, the proposed evolutionary multiorder neurons method can be shown to outperform other related clustering methods. The simulation results from the Iris, Wine, and Glass data sets show significant improvement when compared to the results obtained using self-organizing maps and higher-order neurons. The letter also proposes an intuitive model by which multiorder neurons can be grown, thereby determining the number of clusters in data
A Multi-signal Variant for the GPU-based Parallelization of Growing Self-Organizing Networks
Among the many possible approaches for the parallelization of self-organizing
networks, and in particular of growing self-organizing networks, perhaps the
most common one is producing an optimized, parallel implementation of the
standard sequential algorithms reported in the literature. In this paper we
explore an alternative approach, based on a new algorithm variant specifically
designed to match the features of the large-scale, fine-grained parallelism of
GPUs, in which multiple input signals are processed at once. Comparative tests
have been performed, using both parallel and sequential implementations of the
new algorithm variant, in particular for a growing self-organizing network that
reconstructs surfaces from point clouds. The experimental results show that
this approach allows harnessing in a more effective way the intrinsic
parallelism that the self-organizing networks algorithms seem intuitively to
suggest, obtaining better performances even with networks of smaller size.Comment: 17 page
Somoclu: An Efficient Parallel Library for Self-Organizing Maps
Somoclu is a massively parallel tool for training self-organizing maps on
large data sets written in C++. It builds on OpenMP for multicore execution,
and on MPI for distributing the workload across the nodes in a cluster. It is
also able to boost training by using CUDA if graphics processing units are
available. A sparse kernel is included, which is useful for high-dimensional
but sparse data, such as the vector spaces common in text mining workflows.
Python, R and MATLAB interfaces facilitate interactive use. Apart from fast
execution, memory use is highly optimized, enabling training large emergent
maps even on a single computer.Comment: 26 pages, 9 figures. The code is available at
https://peterwittek.github.io/somoclu
Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization
Artificial autonomous agents and robots interacting in complex environments
are required to continually acquire and fine-tune knowledge over sustained
periods of time. The ability to learn from continuous streams of information is
referred to as lifelong learning and represents a long-standing challenge for
neural network models due to catastrophic forgetting. Computational models of
lifelong learning typically alleviate catastrophic forgetting in experimental
scenarios with given datasets of static images and limited complexity, thereby
differing significantly from the conditions artificial agents are exposed to.
In more natural settings, sequential information may become progressively
available over time and access to previous experience may be restricted. In
this paper, we propose a dual-memory self-organizing architecture for lifelong
learning scenarios. The architecture comprises two growing recurrent networks
with the complementary tasks of learning object instances (episodic memory) and
categories (semantic memory). Both growing networks can expand in response to
novel sensory experience: the episodic memory learns fine-grained
spatiotemporal representations of object instances in an unsupervised fashion
while the semantic memory uses task-relevant signals to regulate structural
plasticity levels and develop more compact representations from episodic
experience. For the consolidation of knowledge in the absence of external
sensory input, the episodic memory periodically replays trajectories of neural
reactivations. We evaluate the proposed model on the CORe50 benchmark dataset
for continuous object recognition, showing that we significantly outperform
current methods of lifelong learning in three different incremental learning
scenario
Fast training of self organizing maps for the visual exploration of molecular compounds
Visual exploration of scientific data in life science
area is a growing research field due to the large amount of
available data. The Kohonen’s Self Organizing Map (SOM) is
a widely used tool for visualization of multidimensional data.
In this paper we present a fast learning algorithm for SOMs
that uses a simulated annealing method to adapt the learning
parameters. The algorithm has been adopted in a data analysis
framework for the generation of similarity maps. Such maps
provide an effective tool for the visual exploration of large and
multi-dimensional input spaces. The approach has been applied
to data generated during the High Throughput Screening
of molecular compounds; the generated maps allow a visual
exploration of molecules with similar topological properties.
The experimental analysis on real world data from the
National Cancer Institute shows the speed up of the proposed
SOM training process in comparison to a traditional approach.
The resulting visual landscape groups molecules with similar
chemical properties in densely connected regions
Advances in Self Organising Maps
The Self-Organizing Map (SOM) with its related extensions is the most popular
artificial neural algorithm for use in unsupervised learning, clustering,
classification and data visualization. Over 5,000 publications have been
reported in the open literature, and many commercial projects employ the SOM as
a tool for solving hard real-world problems. Each two years, the "Workshop on
Self-Organizing Maps" (WSOM) covers the new developments in the field. The WSOM
series of conferences was initiated in 1997 by Prof. Teuvo Kohonen, and has
been successfully organized in 1997 and 1999 by the Helsinki University of
Technology, in 2001 by the University of Lincolnshire and Humberside, and in
2003 by the Kyushu Institute of Technology. The Universit\'{e} Paris I
Panth\'{e}on Sorbonne (SAMOS-MATISSE research centre) organized WSOM 2005 in
Paris on September 5-8, 2005.Comment: Special Issue of the Neural Networks Journal after WSOM 05 in Pari
Fast Algorithm and Implementation of Dissimilarity Self-Organizing Maps
In many real world applications, data cannot be accurately represented by
vectors. In those situations, one possible solution is to rely on dissimilarity
measures that enable sensible comparison between observations. Kohonen's
Self-Organizing Map (SOM) has been adapted to data described only through their
dissimilarity matrix. This algorithm provides both non linear projection and
clustering of non vector data. Unfortunately, the algorithm suffers from a high
cost that makes it quite difficult to use with voluminous data sets. In this
paper, we propose a new algorithm that provides an important reduction of the
theoretical cost of the dissimilarity SOM without changing its outcome (the
results are exactly the same as the ones obtained with the original algorithm).
Moreover, we introduce implementation methods that result in very short running
times. Improvements deduced from the theoretical cost model are validated on
simulated and real world data (a word list clustering problem). We also
demonstrate that the proposed implementation methods reduce by a factor up to 3
the running time of the fast algorithm over a standard implementation
S-TREE: Self-Organizing Trees for Data Clustering and Online Vector Quantization
This paper introduces S-TREE (Self-Organizing Tree), a family of models that use unsupervised learning to construct hierarchical representations of data and online tree-structured vector quantizers. The S-TREE1 model, which features a new tree-building algorithm, can be implemented with various cost functions. An alternative implementation, S-TREE2, which uses a new double-path search procedure, is also developed. S-TREE2 implements an online procedure that approximates an optimal (unstructured) clustering solution while imposing a tree-structure constraint. The performance of the S-TREE algorithms is illustrated with data clustering and vector quantization examples, including a Gauss-Markov source benchmark and an image compression application. S-TREE performance on these tasks is compared with the standard tree-structured vector quantizer (TSVQ) and the generalized Lloyd algorithm (GLA). The image reconstruction quality with S-TREE2 approaches that of GLA while taking less than 10% of computer time. S-TREE1 and S-TREE2 also compare favorably with the standard TSVQ in both the time needed to create the codebook and the quality of image reconstruction.Office of Naval Research (N00014-95-10409, N00014-95-0G57
- …