16,163 research outputs found
Dynamic learning rates for continual unsupervised learning.
The dilemma between stability and plasticity is crucial in machine learning, especially when non-stationary input
distributions are considered. This issue can be addressed by continual learning in order to alleviate catastrophic forgetting. This
strategy has been previously proposed for supervised and reinforcement learning models. However, little attention has been devoted
to unsupervised learning. This work presents a dynamic learning rate framework for unsupervised neural networks that can handle
non-stationary distributions. In order for the model to adapt to the input as it changes its characteristics, a varying learning rate
that does not merely depend on the training step but on the reconstruction error has been proposed. In the experiments, different
configurations for classical competitive neural networks, self-organizing maps and growing neural gas with either per-neuron or
per-network dynamic learning rate have been tested. Experimental results on document clustering tasks demonstrate the suitability
of the proposal for real-world problems
A new self-organizing neural gas model based on Bregman divergences
In this paper, a new self-organizing neural gas model that we call Growing Hierarchical Bregman Neural
Gas (GHBNG) has been proposed. Our proposal is based on the Growing Hierarchical Neural Gas (GHNG) in which Bregman divergences are incorporated in order to compute the winning neuron. This model has been applied to anomaly detection in video sequences together with a Faster R-CNN as an object detector module. Experimental results not only confirm the effectiveness of the GHBNG for the detection of anomalous object in video sequences but also its selforganization
capabilities.Universidad de Málaga. Campus de Excelencia Internacional AndalucĂa Tec
Hierarchical growing cell structures: TreeGCS
We propose a hierarchical clustering algorithm (TreeGCS) based upon the Growing Cell Structure (GCS) neural network of Fritzke. Our algorithm refines and builds upon the GCS base, overcoming an inconsistency in the original GCS algorithm, where the network topology is susceptible to the ordering of the input vectors. Our algorithm is unsupervised, flexible, and dynamic and we have imposed no additional parameters on the underlying GCS algorithm. Our ultimate aim is a hierarchical clustering neural network that is both consistent and stable and identifies the innate hierarchical structure present in vector-based data. We demonstrate improved stability of the GCS foundation and evaluate our algorithm against the hierarchy generated by an ascendant hierarchical clustering dendogram. Our approach emulates the hierarchical clustering of the dendogram. It demonstrates the importance of the parameter settings for GCS and how they affect the stability of the clustering
Evolutionary Neural Gas (ENG): A Model of Self Organizing Network from Input Categorization
Despite their claimed biological plausibility, most self organizing networks
have strict topological constraints and consequently they cannot take into
account a wide range of external stimuli. Furthermore their evolution is
conditioned by deterministic laws which often are not correlated with the
structural parameters and the global status of the network, as it should happen
in a real biological system. In nature the environmental inputs are noise
affected and fuzzy. Which thing sets the problem to investigate the possibility
of emergent behaviour in a not strictly constrained net and subjected to
different inputs. It is here presented a new model of Evolutionary Neural Gas
(ENG) with any topological constraints, trained by probabilistic laws depending
on the local distortion errors and the network dimension. The network is
considered as a population of nodes that coexist in an ecosystem sharing local
and global resources. Those particular features allow the network to quickly
adapt to the environment, according to its dimensions. The ENG model analysis
shows that the net evolves as a scale-free graph, and justifies in a deeply
physical sense- the term gas here used.Comment: 16 pages, 8 figure
PAC: A Novel Self-Adaptive Neuro-Fuzzy Controller for Micro Aerial Vehicles
There exists an increasing demand for a flexible and computationally
efficient controller for micro aerial vehicles (MAVs) due to a high degree of
environmental perturbations. In this work, an evolving neuro-fuzzy controller,
namely Parsimonious Controller (PAC) is proposed. It features fewer network
parameters than conventional approaches due to the absence of rule premise
parameters. PAC is built upon a recently developed evolving neuro-fuzzy system
known as parsimonious learning machine (PALM) and adopts new rule growing and
pruning modules derived from the approximation of bias and variance. These rule
adaptation methods have no reliance on user-defined thresholds, thereby
increasing the PAC's autonomy for real-time deployment. PAC adapts the
consequent parameters with the sliding mode control (SMC) theory in the
single-pass fashion. The boundedness and convergence of the closed-loop control
system's tracking error and the controller's consequent parameters are
confirmed by utilizing the LaSalle-Yoshizawa theorem. Lastly, the controller's
efficacy is evaluated by observing various trajectory tracking performance from
a bio-inspired flapping-wing micro aerial vehicle (BI-FWMAV) and a rotary wing
micro aerial vehicle called hexacopter. Furthermore, it is compared to three
distinctive controllers. Our PAC outperforms the linear PID controller and
feed-forward neural network (FFNN) based nonlinear adaptive controller.
Compared to its predecessor, G-controller, the tracking accuracy is comparable,
but the PAC incurs significantly fewer parameters to attain similar or better
performance than the G-controller.Comment: This paper has been accepted for publication in Information Science
Journal 201
S-TREE: Self-Organizing Trees for Data Clustering and Online Vector Quantization
This paper introduces S-TREE (Self-Organizing Tree), a family of models that use unsupervised learning to construct hierarchical representations of data and online tree-structured vector quantizers. The S-TREE1 model, which features a new tree-building algorithm, can be implemented with various cost functions. An alternative implementation, S-TREE2, which uses a new double-path search procedure, is also developed. S-TREE2 implements an online procedure that approximates an optimal (unstructured) clustering solution while imposing a tree-structure constraint. The performance of the S-TREE algorithms is illustrated with data clustering and vector quantization examples, including a Gauss-Markov source benchmark and an image compression application. S-TREE performance on these tasks is compared with the standard tree-structured vector quantizer (TSVQ) and the generalized Lloyd algorithm (GLA). The image reconstruction quality with S-TREE2 approaches that of GLA while taking less than 10% of computer time. S-TREE1 and S-TREE2 also compare favorably with the standard TSVQ in both the time needed to create the codebook and the quality of image reconstruction.Office of Naval Research (N00014-95-10409, N00014-95-0G57
Application of artificial neural network in market segmentation: A review on recent trends
Despite the significance of Artificial Neural Network (ANN) algorithm to
market segmentation, there is a need of a comprehensive literature review and a
classification system for it towards identification of future trend of market
segmentation research. The present work is the first identifiable academic
literature review of the application of neural network based techniques to
segmentation. Our study has provided an academic database of literature between
the periods of 2000-2010 and proposed a classification scheme for the articles.
One thousands (1000) articles have been identified, and around 100 relevant
selected articles have been subsequently reviewed and classified based on the
major focus of each paper. Findings of this study indicated that the research
area of ANN based applications are receiving most research attention and self
organizing map based applications are second in position to be used in
segmentation. The commonly used models for market segmentation are data mining,
intelligent system etc. Our analysis furnishes a roadmap to guide future
research and aid knowledge accretion and establishment pertaining to the
application of ANN based techniques in market segmentation. Thus the present
work will significantly contribute to both the industry and academic research
in business and marketing as a sustainable valuable knowledge source of market
segmentation with the future trend of ANN application in segmentation.Comment: 24 pages, 7 figures,3 Table
- …