1,301 research outputs found
Rule Extraction and Insertion to Improve the Performance of a Dynamic Cell Structure Neural Network
Artificial Neural Networks are extremely useful machine learning tools. They are used for many purposes, such as prediction, classification, pattern recognition, etc. Although neural networks have been used for decades, they are still often not completely understood or trusted, especially in safety and mission critical situations. Typically, neural networks are trained on data sets that are representative of what needs to be learned. Sometimes training sets are constructed in order to train the neural network in a certain way, in order to embed appropriate knowledge. The purpose of this research is to determine if there is another method that can be used to embed specific knowledge in a neural network before training and if this improves the performance of a neural network.
This research develops and tests a new method of embedding pre-knowledge into the Dynamic Cell Structure (DCS) neural network. The DCS is a type of self-organizing map neural network that has been used for many purposes, including classification. In the research presented here, the method used for embedding pre-knowledge into the neural network is to start by converting the knowledge to a set of IF/THEN rules, that can be easily understood and/or validated by a human expert. Once the rules are constructed and validated, then they are converted to a beginning neural network structure. This allows pre-knowledge to be embedded before training the neural network. This conversion and embedding process is called Rule Insertion.
In order to determine whether this process improves performance, the neural network was trained with and without pre-knowledge embedded. After the training, the neural network structure was again converted to rules, Rule Extraction, and then the neural network accuracy and the rule accuracy were computed. Also, the agreement between the neural network and the extracted rules was computed.
The findings of this research show that using Rule Insertion to embed pre-knowledge into a DCS neural network can increase the accuracy of the neural network. An expert can create the rules to be embedded and can also examine and validate the rules extracted to give more confidence in what the neural network has learned during training. The extracted rules are also a refinement of the inserted rules, meaning the neural network was able to improve upon the expert knowledge based on the data presented
Rough Neural Networks Architecture For Improving Generalization In Pattern Recognition
Neural networks are found to be attractive trainable machines for pattern recognition.
The capability of these models to accommodate wide variety and variability of
conditions, and the ability to imitate brain functions, make them popular research
area.
This research focuses on developing hybrid rough neural networks. These novel
approaches are assumed to provide superior performance with respect to detection
and automatic target recognition.In this thesis, hybrid architectures of rough set theory and neural networks have been
investigated, developed, and implemented. The first hybrid approach provides novel
neural network referred to as Rough Shared weight Neural Networks (RSNN). It uses
the concept of approximation based on rough neurons to feature extraction, and
experiences the methodology of weight sharing. The network stages are a feature
extraction network, and a classification network. The extraction network is
composed of rough neurons that accounts for the upper and lower approximations
and embeds a membership function to replace ordinary activation functions. The
neural network learns the rough set’s upper and lower approximations as feature
extractors simultaneously with classification. The RSNN implements a novel
approximation transform. The basic design for the network is provided together with
the learning rules. The architecture provides a novel method to pattern recognition
and is expected to be robust to any pattern recognition problem.
The second hybrid approach is a two stand alone subsystems, referred to as Rough
Neural Networks (RNN). The extraction network extracts detectors that represent
pattern’s classes to be supplied to the classification network. It works as a filter for
original distilled features based on equivalence relations and rough set reduction,
while the second is responsible for classification of the outputs from the first system.
The two approaches were applied to image pattern recognition problems. The RSNN
was applied to automatic target recognition problem. The data is Synthetic Aperture
Radar (SAR) image scenes of tanks, and background. The RSNN provides a novel
methodology for designing nonlinear filters without prior knowledge of the problem domain. The RNN was used to detect patterns present in satellite image. A novel
feature extraction algorithm was developed to extract the feature vectors. The
algorithm enhances the recognition ability of the system compared to manual
extraction and labeling of pattern classes. The performance of the rough
backpropagation network is improved compared to backpropagation of the same
architecture. The network has been designed to produce detection plane for the
desired pattern.
The hybrid approaches developed in this thesis provide novel techniques to
recognition static and dynamic representation of patterns. In both domains the rough
set theory improved generalization of the neural networks paradigms. The
methodologies are theoretically robust to any pattern recognition problem, and are
proved practically for image environments
Neural Networks for Complex Data
Artificial neural networks are simple and efficient machine learning tools.
Defined originally in the traditional setting of simple vector data, neural
network models have evolved to address more and more difficulties of complex
real world problems, ranging from time evolving data to sophisticated data
structures such as graphs and functions. This paper summarizes advances on
those themes from the last decade, with a focus on results obtained by members
of the SAMM team of Universit\'e Paris
Hybrid quantum-classical unsupervised data clustering based on the Self-Organizing Feature Map
Unsupervised machine learning is one of the main techniques employed in
artificial intelligence. Quantum computers offer opportunities to speed up such
machine learning techniques. Here, we introduce an algorithm for quantum
assisted unsupervised data clustering using the self-organizing feature map, a
type of artificial neural network. We make a proof-of-concept realization of
one of the central components on the IBM Q Experience and show that it allows
us to reduce the number of calculations in a number of clusters. We compare the
results with the classical algorithm on a toy example of unsupervised text
clustering
Chaotic Time Series Forecasting Using Higher Order Neural Networks
This study presents a novel application and comparison of higher order neural networks (HONNs) to forecast benchmark chaotic time series. Two models of HONNs were implemented, namely functional link neural network (FLNN) and pi-sigma neural network (PSNN). These models were tested on two benchmark time series; the monthly smoothed sunspot numbers and the Mackey-Glass time-delay differential equation time series. The forecasting performance of the HONNs is compared against the performance of different models previously used in the literature such as fuzzy and neural networks models. Simulation results showed that FLNN and PSNN offer good performance compared to many previously used hybrid models
Metaheuristic design of feedforward neural networks: a review of two decades of research
Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era
An Overview of the Use of Neural Networks for Data Mining Tasks
In the recent years the area of data mining has experienced a considerable demand for technologies that extract knowledge from large and complex data sources. There is a substantial commercial interest as well as research investigations in the area that aim to develop new and improved approaches for extracting information, relationships, and patterns from datasets. Artificial Neural Networks (NN) are popular biologically inspired intelligent methodologies, whose classification, prediction and pattern recognition capabilities have been utilised successfully in many areas, including science, engineering, medicine, business, banking, telecommunication, and many other fields. This paper highlights from a data mining perspective the implementation of NN, using supervised and unsupervised learning, for pattern recognition, classification, prediction and cluster analysis, and focuses the discussion on their usage in bioinformatics and financial data analysis tasks
A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications
This survey samples from the ever-growing family of adaptive resonance theory
(ART) neural network models used to perform the three primary machine learning
modalities, namely, unsupervised, supervised and reinforcement learning. It
comprises a representative list from classic to modern ART models, thereby
painting a general picture of the architectures developed by researchers over
the past 30 years. The learning dynamics of these ART models are briefly
described, and their distinctive characteristics such as code representation,
long-term memory and corresponding geometric interpretation are discussed.
Useful engineering properties of ART (speed, configurability, explainability,
parallelization and hardware implementation) are examined along with current
challenges. Finally, a compilation of online software libraries is provided. It
is expected that this overview will be helpful to new and seasoned ART
researchers
- …