Integrating Symbolic and Neural Processing in a Self-Organizing Architechture for Pattern Recognition and Prediction

British Petroleum (89A-1204); Defense Advanced Research Projects Agency (N00014-92-J-4015); National Science Foundation (IRI-90-00530); Office of Naval Research (N00014-91-J-4100); Air Force Office of Scientific Research (F49620-92-J-0225

A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers

Learning, Categorization, Rule Formation, and Prediction by Fuzzy Neural Networks

National Science Foundation (IRI 94-01659); Office of Naval Research (N00014-91-J-4100, N00014-92-J-4015) Air Force Office of Scientific Research (90-0083, N00014-92-J-4015

Automatic Diagnosis of Schizophrenia and Attention Deficit Hyperactivity Disorder in rs-fMRI Modality using Convolutional Autoencoder Model and Interval Type-2 Fuzzy Regression

Nowadays, many people worldwide suffer from brain disorders, and their health is in danger. So far, numerous methods have been proposed for the diagnosis of Schizophrenia (SZ) and attention deficit hyperactivity disorder (ADHD), among which functional magnetic resonance imaging (fMRI) modalities are known as a popular method among physicians. This paper presents an SZ and ADHD intelligent detection method of resting-state fMRI (rs-fMRI) modality using a new deep learning method. The University of California Los Angeles dataset, which contains the rs-fMRI modalities of SZ and ADHD patients, has been used for experiments. The FMRIB software library toolbox first performed preprocessing on rs-fMRI data. Then, a convolutional Autoencoder model with the proposed number of layers is used to extract features from rs-fMRI data. In the classification step, a new fuzzy method called interval type-2 fuzzy regression (IT2FR) is introduced and then optimized by genetic algorithm, particle swarm optimization, and gray wolf optimization (GWO) techniques. Also, the results of IT2FR methods are compared with multilayer perceptron, k-nearest neighbors, support vector machine, random forest, and decision tree, and adaptive neuro-fuzzy inference system methods. The experiment results show that the IT2FR method with the GWO optimization algorithm has achieved satisfactory results compared to other classifier methods. Finally, the proposed classification technique was able to provide 72.71% accuracy

A Survey on the Project in title

In this paper we present a survey of work that has been done in the project ldquo;Unsupervised Adaptive P300 BCI in the framework of chaotic theory and stochastic theoryrdquo;we summarised the following papers, (Mohammed J Alhaddad amp; 2011), (Mohammed J. Alhaddad amp; Kamel M, 2012), (Mohammed J Alhaddad, Kamel, amp; Al-Otaibi, 2013), (Mohammed J Alhaddad, Kamel, amp; Bakheet, 2013), (Mohammed J Alhaddad, Kamel, amp; Al-Otaibi, 2014), (Mohammed J Alhaddad, Kamel, amp; Bakheet, 2014), (Mohammed J Alhaddad, Kamel, amp; Kadah, 2014), (Mohammed J Alhaddad, Kamel, Makary, Hargas, amp; Kadah, 2014), (Mohammed J Alhaddad, Mohammed, Kamel, amp; Hagras, 2015).We developed a new pre-processing method for denoising P300-based brain-computer interface data that allows better performance with lower number of channels and blocks. The new denoising technique is based on a modified version of the spectral subtraction denoising and works on each temporal signal channel independently thus offering seamless integration with existing pre-processing and allowing low channel counts to be used. We also developed a novel approach for brain-computer interface data that requires no prior training. The proposed approach is based on interval type-2 fuzzy logic based classifier which is able to handle the usersrsquo; uncertainties to produce better prediction accuracies than other competing classifiers such as BLDA or RFLDA. In addition, the generated type-2 fuzzy classifier is learnt from data via genetic algorithms to produce a small number of rules with a rule length of only one antecedent to maximize the transparency and interpretability for the normal clinician. We also employ a feature selection system based on an ensemble neural networks recursive feature selection which is able to find the effective time instances within the effective sensors in relation to given P300 event. The basic principle of this new class of techniques is that the trial with true activation signal within each block has to be different from the rest of the trials within that block. Hence, a measure that is sensitive to this dissimilarity can be used to make a decision based on a single block without any prior training. The new methods were verified using various experiments which were performed on standard data sets and using real-data sets obtained from real subjects experiments performed in the BCI lab in King Abdulaziz University. The results were compared to the classification results of the same data using previous methods. Enhanced performance in different experiments as quantitatively assessed using classification block accuracy as well as bit rate estimates was confirmed. It will be shown that the produced type-2 fuzzy logic based classifier will learn simple rules which are easy to understand explaining the events in question. In addition, the produced type-2 fuzzy logic classifier will be able to give better accuracies when compared to BLDA or RFLDA on various human subjects on the standard and real-world data sets

Informational Paradigm, management of uncertainty and theoretical formalisms in the clustering framework: A review

Fifty years have gone by since the publication of the first paper on clustering based on fuzzy sets theory. In 1965, L.A. Zadeh had published “Fuzzy Sets” [335]. After only one year, the first effects of this seminal paper began to emerge, with the pioneering paper on clustering by Bellman, Kalaba, Zadeh [33], in which they proposed a prototypal of clustering algorithm based on the fuzzy sets theory

MECHANICAL ENERGY HARVESTER FOR POWERING RFID SYSTEMS COMPONENTS: MODELING, ANALYSIS, OPTIMIZATION AND DESIGN

Finding alternative power sources has been an important topic of study worldwide. It is vital to find substitutes for finite fossil fuels. Such substitutes may be termed renewable energy sources and infinite supplies. Such limitless sources are derived from ambient energy like wind energy, solar energy, sea waves energy; on the other hand, smart cities megaprojects have been receiving enormous amounts of funding to transition our lives into smart lives. Smart cities heavily rely on smart devices and electronics, which utilize small amounts of energy to run. Using batteries as the power source for such smart devices imposes environmental and labor cost issues. Moreover, in many cases, smart devices are in hard-to-access places, making accessibility for disposal and replacement difficult. Finally, battery waste harms the environment. To overcome these issues, vibration-based energy harvesters have been proposed and implemented. Vibration-based energy harvesters convert the dynamic or kinetic energy which is generated due to the motion of an object into electric energy. Energy transduction mechanisms can be delivered based on piezoelectric, electromagnetic, or electrostatic methods; the piezoelectric method is generally preferred to the other methods, particularly if the frequency fluctuations are considerable. In response, piezoelectric vibration-based energy harvesters (PVEHs), have been modeled and analyzed widely. However, there are two challenges with PVEH: the maximum amount of extractable voltage and the effective (operational) frequency bandwidth are often insufficient. In this dissertation, a new type of integrated multiple system comprised of a cantilever and spring-oscillator is proposed to improve and develop the performance of the energy harvester in terms of extractable voltage and effective frequency bandwidth. The new energy harvester model is proposed to supply sufficient energy to power low-power electronic devices like RFID components. Due to the temperature fluctuations, the thermal effect over the performance of the harvester is initially studied. To alter the resonance frequency of the harvester structure, a rotating element system is considered and analyzed. In the analytical-numerical analysis, Hamilton’s principle along with Galerkin’s decomposition approach are adopted to derive the governing equations of the harvester motion and corresponding electric circuit. It is observed that integration of the spring-oscillator subsystem alters the boundary condition of the cantilever and subsequently reforms the resulting characteristic equation into a more complicated nonlinear transcendental equation. To find the resonance frequencies, this equation is solved numerically in MATLAB. It is observed that the inertial effects of the oscillator rendered to the cantilever via the restoring force effects of the spring significantly alter vibrational features of the harvester. Finally, the voltage frequency response function is analytically and numerically derived in a closed-from expression. Variations in parameter values enable the designer to mutate resonance frequencies and mode shape functions as desired. This is particularly important, since the generated energy from a PVEH is significant only if the excitation frequency coming from an external source matches the resonance (natural) frequency of the harvester structure. In subsequent sections of this work, the oscillator mass and spring stiffness are considered as the design parameters to maximize the harvestable voltage and effective frequency bandwidth, respectively. For the optimization, a genetic algorithm is adopted to find the optimal values. Since the voltage frequency response function cannot be implemented in a computer algorithm script, a suitable function approximator (regressor) is designed using fuzzy logic and neural networks. The voltage function requires manual assistance to find the resonance frequency and cannot be done automatically using computer algorithms. Specifically, to apply the numerical root-solver, one needs to manually provide the solver with an initial guess. Such an estimation is accomplished using a plot of the characteristic equation along with human visual inference. Thus, the entire process cannot be automated. Moreover, the voltage function encompasses several coefficients making the process computationally expensive. Thus, training a supervised machine learning regressor is essential. The trained regressor using adaptive-neuro-fuzzy-inference-system (ANFIS) is utilized in the genetic optimization procedure. The optimization problem is implemented, first to find the maximum voltage and second to find the maximum widened effective frequency bandwidth, which yields the optimal oscillator mass value along with the optimal spring stiffness value. As there is often no control over the external excitation frequency, it is helpful to design an adaptive energy harvester. This means that, considering a specific given value of the excitation frequency, energy harvester system parameters (oscillator mass and spring stiffness) need to be adjusted so that the resulting natural (resonance) frequency of the system aligns with the given excitation frequency. To do so, the given excitation frequency value is considered as the input and the system parameters are assumed as outputs which are estimated via the neural network fuzzy logic regressor. Finally, an experimental setup is implemented for a simple pure cantilever energy harvester triggered by impact excitations. Unlike the theoretical section, the experimental excitation is considered to be an impact excitation, which is a random process. The rationale for this is that, in the real world, the external source is a random trigger. Harmonic base excitations used in the theoretical chapters are to assess the performance of the energy harvester per standard criteria. To evaluate the performance of a proposed energy harvester model, the input excitation type consists of harmonic base triggers. In summary, this dissertation discusses several case studies and addresses key issues in the design of optimized piezoelectric vibration-based energy harvesters (PVEHs). First, an advanced model of the integrated systems is presented with equation derivations. Second, the proposed model is decomposed and analyzed in terms of mechanical and electrical frequency response functions. To do so, analytic-numeric methods are adopted. Later, influential parameters of the integrated system are detected. Then the proposed model is optimized with respect to the two vital criteria of maximum amount of extractable voltage and widened effective (operational) frequency bandwidth. Corresponding design (influential) parameters are found using neural network fuzzy logic along with genetic optimization algorithms, i.e., a soft computing method. The accuracy of the trained integrated algorithms is verified using the analytical-numerical closed-form expression of the voltage function. Then, an adaptive piezoelectric vibration-based energy harvester (PVEH) is designed. This final design pertains to the cases where the excitation (driving) frequency is given and constant, so the desired goal is to match the natural frequency of the system with the given driving frequency. In this response, a regressor using neural network fuzzy logic is designed to find the proper design parameters. Finally, the experimental setup is implemented and tested to report the maximum voltage harvested in each test execution

Novel Art-Based Neural Network Models For Pattern Classification, Rule Extraction And Data Regression

This thesis is concerned with the development of novel neural network models for tackling pattern classification, rule extraction, and data regression problems. The research focuses on one of the advanced features of neural networks, i.e., the incremental learning ability. This ability relates to continuous learning of new knowledge without disturbing the existing knowledge base and without re-iterating through the training samples. The Adaptive Resonance Theory (ART) and Generalized Regression Neural Network (GRNN) models are employed as the backbone in this research