414 research outputs found
Generation of Explicit Knowledge from Empirical Data through Pruning of Trainable Neural Networks
This paper presents a generalized technology of extraction of explicit
knowledge from data. The main ideas are 1) maximal reduction of network
complexity (not only removal of neurons or synapses, but removal all the
unnecessary elements and signals and reduction of the complexity of elements),
2) using of adjustable and flexible pruning process (the pruning sequence
shouldn't be predetermined - the user should have a possibility to prune
network on his own way in order to achieve a desired network structure for the
purpose of extraction of rules of desired type and form), and 3) extraction of
rules not in predetermined but any desired form. Some considerations and notes
about network architecture and training process and applicability of currently
developed pruning techniques and rule extraction algorithms are discussed. This
technology, being developed by us for more than 10 years, allowed us to create
dozens of knowledge-based expert systems. In this paper we present a
generalized three-step technology of extraction of explicit knowledge from
empirical data.Comment: 9 pages, The talk was given at the IJCNN '99 (Washington DC, July
1999
Distributed Activation, Search, and Learning by ART and ARTMAP Neural Networks
Adaptive resonance theory (ART) models have been used for learning and prediction in a wide variety of applications. Winner-take-all coding allows these networks to maintain stable memories, but this type of code representation can cause problems such as category proliferation with fast learning and a noisy training set. A new class of ART models with an arbitrarily distributed code representation is outlined here. With winner-take-all coding, the unsupervised distributed ART model (dART) reduces to fuzzy ART and the supervised distributed ARTMAP model (dARTMAP) reduces to fuzzy ARTMAP. dART automatically apportions learned changes according to the degree of activation of each node, which permits fast as well as slow learning with compressed or distributed codes. Distributed ART models replace the traditional neural network path weight with a dynamic weight equal to the rectified difference between coding node activation and an adaptive threshold. Dynamic weights that project to coding nodes obey a distributed instar leaning law and those that originate from coding nodes obey a distributed outstar learning law. Inputs activate distributed codes through phasic and tonic signal components with dual computational properties, and a parallel distributed match-reset-search process helps stabilize memory.National Science Foundation (IRI 94-0 1659); Office of Naval Research (N00014-95-1-0409, N00014-95-0657
Artificial Neural Network Pruning to Extract Knowledge
Artificial Neural Networks (NN) are widely used for solving complex problems
from medical diagnostics to face recognition. Despite notable successes, the
main disadvantages of NN are also well known: the risk of overfitting, lack of
explainability (inability to extract algorithms from trained NN), and high
consumption of computing resources. Determining the appropriate specific NN
structure for each problem can help overcome these difficulties: Too poor NN
cannot be successfully trained, but too rich NN gives unexplainable results and
may have a high chance of overfitting. Reducing precision of NN parameters
simplifies the implementation of these NN, saves computing resources, and makes
the NN skills more transparent. This paper lists the basic NN simplification
problems and controlled pruning procedures to solve these problems. All the
described pruning procedures can be implemented in one framework. The developed
procedures, in particular, find the optimal structure of NN for each task,
measure the influence of each input signal and NN parameter, and provide a
detailed verbal description of the algorithms and skills of NN. The described
methods are illustrated by a simple example: the generation of explicit
algorithms for predicting the results of the US presidential election.Comment: IJCNN 202
A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications
This survey samples from the ever-growing family of adaptive resonance theory
(ART) neural network models used to perform the three primary machine learning
modalities, namely, unsupervised, supervised and reinforcement learning. It
comprises a representative list from classic to modern ART models, thereby
painting a general picture of the architectures developed by researchers over
the past 30 years. The learning dynamics of these ART models are briefly
described, and their distinctive characteristics such as code representation,
long-term memory and corresponding geometric interpretation are discussed.
Useful engineering properties of ART (speed, configurability, explainability,
parallelization and hardware implementation) are examined along with current
challenges. Finally, a compilation of online software libraries is provided. It
is expected that this overview will be helpful to new and seasoned ART
researchers
Open Vocabulary Learning on Source Code with a Graph-Structured Cache
Machine learning models that take computer program source code as input
typically use Natural Language Processing (NLP) techniques. However, a major
challenge is that code is written using an open, rapidly changing vocabulary
due to, e.g., the coinage of new variable and method names. Reasoning over such
a vocabulary is not something for which most NLP methods are designed. We
introduce a Graph-Structured Cache to address this problem; this cache contains
a node for each new word the model encounters with edges connecting each word
to its occurrences in the code. We find that combining this graph-structured
cache strategy with recent Graph-Neural-Network-based models for supervised
learning on code improves the models' performance on a code completion task and
a variable naming task --- with over relative improvement on the latter
--- at the cost of a moderate increase in computation time.Comment: Published in the International Conference on Machine Learning (ICML
2019), 13 page
Multi-agent knowledge integration mechanism using particle swarm optimization
This is the post-print version of the final paper published in Technological Forecasting and Social Change. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2011 Elsevier B.V.Unstructured group decision-making is burdened with several central difficulties: unifying the knowledge of multiple experts in an unbiased manner and computational inefficiencies. In addition, a proper means of storing such unified knowledge for later use has not yet been established. Storage difficulties stem from of the integration of the logic underlying multiple experts' decision-making processes and the structured quantification of the impact of each opinion on the final product. To address these difficulties, this paper proposes a novel approach called the multiple agent-based knowledge integration mechanism (MAKIM), in which a fuzzy cognitive map (FCM) is used as a knowledge representation and storage vehicle. In this approach, we use particle swarm optimization (PSO) to adjust causal relationships and causality coefficients from the perspective of global optimization. Once an optimized FCM is constructed an agent based model (ABM) is applied to the inference of the FCM to solve real world problem. The final aggregate knowledge is stored in FCM form and is used to produce proper inference results for other target problems. To test the validity of our approach, we applied MAKIM to a real-world group decision-making problem, an IT project risk assessment, and found MAKIM to be statistically robust.Ministry of Education, Science and Technology (Korea
Multidimensional flow mapping for proportional valves
Inverse, multidimensional input-output flow mapping is very important for use of valves in precision motion control applications. Due to the highly nonlinear characteristic and uncertain model structure of the cartridge valves, it is hard to formulate the modelling of their flow mappings into simple parameter estimation problems. This contribution conducts a comprehensive analysis and validation of three- and four-dimensional input-output-mapping approaches for a proportional pilot operated seat valves. Therefore, a virtual and a physical test-rig setup are utilized for initial measurement, implementation and assessment. After modeling and validating the valve under consideration, as a function of flow, pressure and temperature different mapping methods are investigated. More specifically, state of the art approaches, deep-learning methods and a newly developed approach (extPoly) are examined. Especially ANNs and Polynomials show reasonable approximation results even for more than two inputs. However, the results are strongly dependent on the structure and distribution of the input data points. Besides identification effort, the invertibility was investigated
A Novel Hybrid Particle Swarm Optimization and Sine Cosine Algorithm for Seismic Optimization of Retaining Structures
This study introduces an effective hybrid optimization algorithm, namely Particle Swarm Sine Cosine Algorithm (PSSCA) for numerical function optimization and automating optimum design of retaining structures under seismic loads. The new algorithm employs the dynamic behavior of sine and cosine functions in the velocity updating operation of particle swarm optimization (PSO) to achieve faster convergence and better accuracy of final solution without getting trapped in local minima. The proposed algorithm is tested over a set of 16 benchmark functions and the results are compared with other well-known algorithms in the field of optimization. For seismic optimization of retaining structure, Mononobe-Okabe method is employed for dynamic loading condition and total construction cost of the structure is considered as the objective function. Finally, optimization of two retaining structures under static and seismic loading are considered from the literature. As results demonstrate, the PSSCA is superior and it could generate better optimal solutions compared with other competitive algorithms
- …