Development of neural units with higher-order synaptic operations and their applications to logic circuits and control problems

Neural networks play an important role in the execution of goal-oriented paradigms. They offer flexibility, adaptability and versatility, so that a variety of approaches may be used to meet a specific goal, depending upon the circumstances and the requirements of the design specifications. Development of higher-order neural units with higher-order synaptic operations will open a new window for some complex problems such as control of aerospace vehicles, pattern recognition, and image processing. The neural models described in this thesis consider the behavior of a single neuron as the basic computing unit in neural information processing operations. Each computing unit in the network is based on the concept of an idealized neuron in the central nervous system (CNS). Most recent mathematical models and their architectures for neuro-control systems have generated many theoretical and industrial interests. Recent advances in static and dynamic neural networks have created a profound impact in the field of neuro-control. Neural networks consisting of several layers of neurons, with linear synaptic operation, have been extensively used in different applications such as pattern recognition, system identification and control of complex systems such as flexible structures, and intelligent robotic systems. The conventional linear neural models are highly simplified models of the biological neuron. Using this model, many neural morphologies, usually referred to as multilayer feedforward neural networks (MFNNs), have been reported in the literature. The performance of the neurons is greatly affected when a layer of neurons are implemented for system identification, pattern recognition and control problems. Through simulation studies of the XOR logic it was concluded that the neurons with linear synaptic operation are limited to only linearly separable forms of pattern distribution. However, they perform a variety of complex mathematical operations when they are implemented in the form of a network structure. These networks suffer from various limitations such as computational efficiency and learning capabilities and moreover, these models ignore many salient features of the biological neurons such as time delays, cross and self correlations, and feedback paths which are otherwise very important in the neural activity. In this thesis an effort is made to develop new mathematical models of neurons that belong to the class of higher-order neural units (HONUs) with higher-order synaptic operations such as quadratic and cubic synaptic operations. The advantage of using this type of neural unit is associated with performance of the neurons but the performance comes at the cost of exponential increase in parameters that hinders the speed of the training process. In this context, a novel method of representation of weight parameters without sacrificing the neural performance has been introduced. A generalised representation of the higher-order synaptic operation for these neural structures was proposed. It was shown that many existing neural structures can be derived from this generalized representation of the higher-order synaptic operation. In the late 1960’s, McCulloch and Pitts modeled the stimulation-response of the primitive neuron using the threshold logic. Since then, it has become a practice to implement the logic circuits using neural structures. In this research, realization of the logic circuits such as OR, AND, and XOR were implemented using the proposed neural structures. These neural structures were also implemented as neuro-controllers for the control problems such as satellite attitude control and model reference adaptive control. A comparative study of the performance of these neural structures compared to that of the conventional linear controllers has been presented. The simulation results obtained in this research were applicable only for the simplified model presented in the simulation studies

Dynamic construction of back-propagation artificial neural networks.

by Korris Fu-lai Chung.Thesis (M.Phil.) -- Chinese University of Hong Kong, 1991.Bibliography: leaves R-1 - R-5.LIST OF FIGURES --- p.viLIST OF TABLES --- p.viiiChapter 1 --- INTRODUCTIONChapter 1.1 --- Recent Resurgence of Artificial Neural Networks --- p.1-1Chapter 1.2 --- A Design Problem in Applying Back-Propagation Networks --- p.1-4Chapter 1.3 --- Related Works --- p.1-6Chapter 1.4 --- Objective of the Research --- p.1-8Chapter 1.5 --- Thesis Organization --- p.1-9Chapter 2 --- MULTILAYER FEEDFORWARD NETWORKS (MFNs) AND BACK-PRO- PAGATION (BP) LEARNING ALGORITHMChapter 2.1 --- Introduction --- p.2-1Chapter 2.2 --- From Perceptrons to MFNs --- p.2-2Chapter 2.3 --- From Delta Rule to BP Algorithm --- p.2-6Chapter 2.4 --- A Variant of BP Algorithm --- p.2-12Chapter 3 --- INTERPRETATIONS AND PROPERTIES OF BP NETWORKSChapter 3.1 --- Introduction --- p.3-1Chapter 3.2 --- A Pattern Classification View on BP Networks --- p.3-2Chapter 3.2.1 --- Pattern Space Interpretation of BP Networks --- p.3-2Chapter 3.2.2 --- Weight Space Interpretation of BP Networks --- p.3-3Chapter 3.3 --- Local Minimum --- p.3-5Chapter 3.4 --- Generalization --- p.3-6Chapter 4 --- GROWTH OF BP NETWORKSChapter 4.1 --- Introduction --- p.4-1Chapter 4.2 --- Problem Formulation --- p.4-1Chapter 4.3 --- Learning an Additional Pattern --- p.4-2Chapter 4.4 --- A Progressive Training Algorithm --- p.4-4Chapter 4.5 --- Experimental Results and Performance Analysis --- p.4-7Chapter 4.6 --- Concluding Remarks --- p.4-16Chapter 5 --- PRUNING OF BP NETWORKSChapter 5.1 --- Introduction --- p.5-1Chapter 5.2 --- Characteristics of Hidden Nodes in Oversized Networks --- p.5-2Chapter 5.2.1 --- Observations from an Empirical Study --- p.5-2Chapter 5.2.2 --- Four Categories of Excessive Nodes --- p.5-3Chapter 5.2.3 --- Why are they excessive ？ --- p.5-6Chapter 5.3 --- Pruning of Excessive Nodes --- p.5-9Chapter 5.4 --- Experimental Results and Performance Analysis --- p.5-13Chapter 5.5 --- Concluding Remarks --- p.5-19Chapter 6 --- DYNAMIC CONSTRUCTION OF BP NETWORKSChapter 6.1 --- A Hybrid Approach --- p.6-1Chapter 6.2 --- Experimental Results and Performance Analysis --- p.6-2Chapter 6.3 --- Concluding Remarks --- p.6-7Chapter 7 --- CONCLUSIONS --- p.7-1Chapter 7.1 --- Contributions --- p.7-1Chapter 7.2 --- Limitations and Suggestions for Further Research --- p.7-2REFERENCES --- p.R-lAPPENDIXChapter A.1 --- A Handwriting Numeral Recognition Experiment: Feature Extraction Technique and Sampling Process --- p.A-1Chapter A.2 --- Determining the distance d= δ2/2r in Lemma 1 --- p.A-

Use of Multivariate Techniques to Validate and Improved the Current USAF Pilot Candidate Selection Model

The Pilot Candidate Selection Method (PCSM) seeks to ensure the highest possible probability of success at UPT. PCSM applies regression weights to a candidate\u27s Air Force Officer Qualification Test (AFOQT) Pilot composite score, self-reported flying hours, and five Basic Attributes Test (BAT) score composites. PCSM scores range between 0 and 99 and is interpreted as a candidate\u27s probability of passing UPT. The goal of this study is to apply multivariate data analysis techniques to validate PCSM and determine appropriate changes to the model\u27s weights. Performance of the updated weights is compared to the current PCSM model via Receiver Operating Curves (ROC). In addition, two independent models are developed using multi-layer perceptron neural networks and discriminant analysis. Both linear and logistic regression is used to investigate possible updates to PCSM\u27s current linear regression weights. An independent test set is used to estimate the generalized performance of the regressions and independent models. Validation of the current PCSM model demonstrated in the first phase of this research is enhanced by the fact that PCSM outperforms all other models developed in the research

Razvoj metod strojnega učenja za identifikacijo kozmičnih delcev ekstremnih energij ter njihova implementacija pri iskanju fotonov ekstremnih energij s površinskimi detektorji Observatorija Pierre Auger

Despite their discovery already more than a century ago, Cosmic Rays (CRs) still did not divulge all their properties yet. Theories about the origin of ultra-high energy (UHE, > 10^18 eV) CRs predict accompanying primary photons. The existence of UHE photons can be investigated with the world’s largest ground-based experiment for detection of CR-induced extensive air showers (EAS), the Pierre Auger Observatory, which offers an unprecedented exposure to rare UHE cosmic particles. The discovery of photons in the UHE regime would open a new observational window to the Universe, improve our understanding of the origin of CRs, and potentially uncloak new physics beyond the standard model. The novelty of the presented work is the development of a "real-time" photon candidate event stream to a global network of observatories, the Astrophysical Multimessenger Observatory Network (AMON). The stream classifies CR events observed by the Auger surface detector (SD) array as regards their probability to be photon nominees, by feeding to advanced machine learning (ML) methods observational air shower parameters of individual CR events combined in a multivariate analysis (MVA). The described straightforward classification procedure further increases the Pierre Auger Observatory’s endeavour to contribute to the global effort of multi-messenger (MM) studies of the highest energy astrophysical phenomena, by supplying AMON partner observatories the possibility to follow-up detected UHE events, live or in their archival data

Practical approaches to mining of clinical datasets : from frameworks to novel feature selection

Research has investigated clinical data that have embedded within them numerous complexities and uncertainties in the form of missing values, class imbalances and high dimensionality. The research in this thesis was motivated by these challenges to minimise these problems whilst, at the same time, maximising classification performance of data and also selecting the significant subset of variables. As such, this led to the proposal of a data mining framework and feature selection method. The proposed framework has a simple algorithmic framework and makes use of a modified form of existing frameworks to address a variety of different data issues, called the Handling Clinical Data Framework (HCDF). The assessment of data mining techniques reveals that missing values imputation and resampling data for class balancing can improve the performance of classification. Next, the proposed feature selection method was introduced; it involves projecting onto principal component method (FS-PPC) and draws on ideas from both feature extraction and feature selection to select a significant subset of features from the data. This method selects features that have high correlation with the principal component by applying symmetrical uncertainty (SU). However, irrelevant and redundant features are removed by using mutual information (MI). However, this method provides confidence in the selected subset of features that will yield realistic results with less time and effort. FS-PPC is able to retain classification performance and meaningful features while consisting of non-redundant features. The proposed methods have been practically applied to analysis of real clinical data and their effectiveness has been assessed. The results show that the proposed methods are enable to minimise the clinical data problems whilst, at the same time, maximising classification performance of data