6,635 research outputs found
Automatic generation of initial weights and target outputs of multilayer neural networks and its application to pattern classification
金沢大学理工研究域 電子情報学
PROPOSED METHODOLOGY FOR OPTIMIZING THE TRAINING PARAMETERS OF A MULTILAYER FEED-FORWARD ARTIFICIAL NEURAL NETWORKS USING A GENETIC ALGORITHM
An artificial neural network (ANN), or shortly "neural network" (NN), is a powerful
mathematical or computational model that is inspired by the structure and/or
functional characteristics of biological neural networks. Despite the fact that ANN has
been developing rapidly for many years, there are still some challenges concerning
the development of an ANN model that performs effectively for the problem at hand.
ANN can be categorized into three main types: single layer, recurrent network and
multilayer feed-forward network. In multilayer feed-forward ANN, the actual
performance is highly dependent on the selection of architecture and training
parameters. However, a systematic method for optimizing these parameters is still an
active research area. This work focuses on multilayer feed-forward ANNs due to their
generalization capability, simplicity from the viewpoint of structure, and ease of
mathematical analysis. Even though, several rules for the optimization of multilayer
feed-forward ANN parameters are available in the literature, most networks are still
calibrated via a trial-and-error procedure, which depends mainly on the type of
problem, and past experience and intuition of the expert. To overcome these
limitations, there have been attempts to use genetic algorithm (GA) to optimize some
of these parameters. However most, if not all, of the existing approaches are focused
partially on the part of architecture and training parameters. On the contrary, the GAANN
approach presented here has covered most aspects of multilayer feed-forward
ANN in a more comprehensive way. This research focuses on the use of binaryencoded
genetic algorithm (GA) to implement efficient search strategies for the
optimal architecture and training parameters of a multilayer feed-forward ANN.
Particularly, GA is utilized to determine the optimal number of hidden layers, number
of neurons in each hidden layer, type of training algorithm, type of activation function
of hidden and output neurons, initial weight, learning rate, momentum term, and
epoch size of a multilayer feed-forward ANN. In this thesis, the approach has been
analyzed and algorithms that simulate the new approach have been mapped out
Strategies for neural networks in ballistocardiography with a view towards hardware implementation
A thesis submitted for the degree of Doctor of Philosophy
at the University of LutonThe work described in this thesis is based on the results of a clinical trial conducted by the research team at the Medical Informatics Unit of the University of Cambridge, which show that the Ballistocardiogram (BCG) has prognostic value in detecting impaired left ventricular function before it becomes clinically overt as myocardial infarction leading to sudden death. The objective of this study is to develop and demonstrate a framework for realising an on-line BCG signal classification model in a portable device that would have the potential to find pathological signs as early as possible for home health care.
Two new on-line automatic BeG classification models for time domain BeG classification are proposed. Both systems are based on a two stage process: input feature extraction followed by a neural classifier. One system uses a principal component analysis neural network, and the other a discrete wavelet transform, to reduce the input dimensionality. Results of the classification, dimensionality reduction, and comparison are presented. It is indicated that the combined wavelet transform and MLP system has a more reliable performance than the combined neural networks system, in situations where the data available to determine the network parameters is limited. Moreover, the wavelet transfonn requires no prior knowledge of the statistical distribution of data samples and the computation complexity and training time are reduced. Overall, a methodology for realising an automatic BeG classification system for a portable instrument is presented.
A fully paralJel neural network design for a low cost platform using field programmable gate arrays (Xilinx's XC4000 series) is explored. This addresses the potential speed requirements in the biomedical signal processing field. It also demonstrates a flexible hardware design approach so that an instrument's parameters can be updated as data expands with time. To reduce the hardware design complexity and to increase the system performance, a hybrid learning algorithm using random optimisation and the backpropagation rule is developed to achieve an efficient weight update mechanism in low weight precision learning. The simulation results show that the hybrid learning algorithm is effective in solving the network paralysis problem and the convergence is much faster than by the standard backpropagation rule. The hidden and output layer nodes have been mapped on Xilinx FPGAs with automatic placement and routing tools. The static time analysis results suggests that the proposed network implementation could generate 2.7 billion connections per second performance
Query-Based Learning for Aerospace Applications
Models of real-world applications often include a large number of parameters with a wide dynamic range, which contributes to the difficulties of neural network training. Creating the training data set for such applications becomes costly, if not impossible. In order to overcome the challenge, one can employ an active learning technique known as query-based learning (QBL) to add performance-critical data to the training set during the learning phase, thereby efficiently improving the overall learning/generalization. The performance-critical data can be obtained using an inverse mapping called network inversion (discrete network inversion and continuous network inversion) followed by oracle query. This paper investigates the use of both inversion techniques for QBL learning, and introduces an original heuristic to select the inversion target values for continuous network inversion method. Efficiency and generalization was further enhanced by employing node decoupled extended Kalman filter (NDEKF) training and a causality index (CI) as a means to reduce the input search dimensionality. The benefits of the overall QBL approach are experimentally demonstrated in two aerospace applications: a classification problem with large input space and a control distribution problem
A CASE STUDY ON SUPPORT VECTOR MACHINES VERSUS ARTIFICIAL NEURAL NETWORKS
The capability of artificial neural networks for pattern recognition of real world problems is well known. In recent years, the support vector machine has been advocated for its structure risk minimization leading to tolerance margins of decision boundaries. Structures and performances of these pattern classifiers depend on the feature dimension and training data size. The objective of this research is to compare these pattern recognition systems based on a case study. The particular case considered is on classification of hypertensive and normotensive right ventricle (RV) shapes obtained from Magnetic Resonance Image (MRI) sequences. In this case, the feature dimension is reasonable, but the available training data set is small, however, the decision surface is highly nonlinear.For diagnosis of congenital heart defects, especially those associated with pressure and volume overload problems, a reliable pattern classifier for determining right ventricle function is needed. RV¡¦s global and regional surface to volume ratios are assessed from an individual¡¦s MRI heart images. These are used as features for pattern classifiers. We considered first two linear classification methods: the Fisher linear discriminant and the linear classifier trained by the Ho-Kayshap algorithm. When the data are not linearly separable, artificial neural networks with back-propagation training and radial basis function networks were then considered, providing nonlinear decision surfaces. Thirdly, a support vector machine was trained which gives tolerance margins on both sides of the decision surface. We have found in this case study that the back-propagation training of an artificial neural network depends heavily on the selection of initial weights, even though randomized. The support vector machine where radial basis function kernels are used is easily trained and provides decision tolerance margins, in spite of only small margins
Personalized Health Monitoring Using Evolvable Block-based Neural Networks
This dissertation presents personalized health monitoring using evolvable block-based neural networks. Personalized health monitoring plays an increasingly important role in modern society as the population enjoys longer life. Personalization in health monitoring considers physiological variations brought by temporal, personal or environmental differences, and demands solutions capable to reconfigure and adapt to specific requirements. Block-based neural networks (BbNNs) consist of 2-D arrays of modular basic blocks that can be easily implemented using reconfigurable digital hardware such as field programmable gate arrays (FPGAs) that allow on-line partial reorganization. The modular structure of BbNNs enables easy expansion in size by adding more blocks. A computationally efficient evolutionary algorithm is developed that simultaneously optimizes structure and weights of BbNNs. This evolutionary algorithm increases optimization speed by integrating a local search operator. An adaptive rate update scheme removing manual tuning of operator rates enhances the fitness trend compared to pre-determined fixed rates. A fitness scaling with generalized disruptive pressure reduces the possibility of premature convergence. The BbNN platform promises an evolvable solution that changes structures and parameters for personalized health monitoring. A BbNN evolved with the proposed evolutionary algorithm using the Hermite transform coefficients and a time interval between two neighboring R peaks of ECG signal, provides a patient-specific ECG heartbeat classification system. Experimental results using the MIT-BIH Arrhythmia database demonstrate a potential for significant performance enhancements over other major techniques
Improving malware detection with neuroevolution : a study with the semantic learning machine
Project Work presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business IntelligenceMachine learning has become more attractive over the years due to its remarkable adaptation and
problem-solving abilities. Algorithms compete amongst each other to claim the best possible results
for every problem, being one of the most valued characteristics their generalization ability.
A recently proposed methodology of Genetic Programming (GP), called Geometric Semantic Genetic
Programming (GSGP), has seen its popularity rise over the last few years, achieving great results
compared to other state-of-the-art algorithms, due to its remarkable feature of inducing a fitness
landscape with no local optima solutions. To any supervised learning problem, where a metric is used
as an error function, GSGP’s landscape will be unimodal, therefore allowing for genetic algorithms to
behave much more efficiently and effectively.
Inspired by GSGP’s features, Gonçalves developed a new mutation operator to be applied to the Neural
Networks (NN) domain, creating the Semantic Learning Machine (SLM). Despite GSGP’s good results
already proven, there are still research opportunities for improvement, that need to be performed to
empirically prove GSGP as a state-of-the-art framework.
In this case, the study focused on applying SLM to NNs with multiple hidden layers and compare its
outputs to a very popular algorithm, Multilayer Perceptron (MLP), on a considerably large classification
dataset about Android malware. Findings proved that SLM, sharing common parametrization with
MLP, in order to have a fair comparison, is able to outperform it, with statistical significance
- …