Multi-objective evolutionary algorithms of spiking neural networks

Spiking neural network (SNN) is considered as the third generation of artificial neural networks. Although there are many models of SNN, Evolving Spiking Neural Network (ESNN) is widely used in many recent research works. Among the many important issues that need to be explored in ESNN are determining the optimal pre-synaptic neurons and parameters values for a given data set. Moreover, previous studies have not investigated the performance of the multi-objective approach with ESNN. In this study, the aim is to find the optimal pre-synaptic neurons and parameter values for ESNN simultaneously by proposing several integrations between ESNN and differential evolution (DE). The proposed algorithms applied to address these problems include DE with evolving spiking neural network (DE-ESNN) and DE for parameter tuning with evolving spiking neural network (DEPT-ESNN). This study also utilized the approach of multi-objective (MOO) with ESNN for better learning structure and classification accuracy. Harmony Search (HS) and memetic approach was used to improve the performance of MOO with ESNN. Consequently, Multi- Objective Differential Evolution with Evolving Spiking Neural Network (MODE-ESNN), Harmony Search Multi-Objective Differential Evolution with Evolving Spiking Neural Network (HSMODE-ESNN) and Memetic Harmony Search Multi-Objective Differential Evolution with Evolving Spiking Neural Network (MEHSMODE-ESNN) were applied to improve ESNN structure and accuracy rates. The hybrid methods were tested by using seven benchmark data sets from the machine learning repository. The performance was evaluated using different criteria such as accuracy (ACC), geometric mean (GM), sensitivity (SEN), specificity (SPE), positive predictive value (PPV), negative predictive value (NPV) and average site performance (ASP) using k-fold cross validation. Evaluation analysis shows that the proposed methods demonstrated better classification performance as compared to the standard ESNN especially in the case of imbalanced data sets. The findings revealed that the MEHSMODE-ESNN method statistically outperformed all the other methods using the different data sets and evaluation criteria. It is concluded that multi objective proposed methods have been evinced as the best proposed methods for most of the data sets used in this study. The findings have proven that the proposed algorithms attained the optimal presynaptic neurons and parameters values and MOO approach was applicable for the ESNN

Granular computing approach for the design of medical data classification systems

Granular computing is a computation theory that imitates human thinking and reasoning by dealing with information at different levels of abstraction/precision. The adoption of granular computing approach in the design of data classification systems improves their performance in dealing with data uncertainty and facilitates handling large volumes of data. In this paper, a new approach for the design of medical data classification systems is proposed. The proposed approach makes use of data granulation in training the classifier. Training data is granulated at different levels and data from each level is used for constructing the classification system. To evaluate performance of the proposed approach, a classification system based on neural network is implemented. Four medical datasets are used to compare performance of the proposed approach to other classifiers: neural network classifier, ANFIS classifier and SVM classifier. Results show that the proposed approach improves classification performance of neural network classifier and produces better accuracy and area under curve than other classifiers for most of the datasets used

Spiking neurons in 3D growing self-organising maps

In Kohonen’s Self-Organising Maps (SOM) learning, preserving the map topology to simulate the actual input features appears to be a significant process. Misinterpretation of the training samples can lead to failure in identifying the important features that may affect the outcomes generated by the SOM model. Nonetheless, it is a challenging task as most of the real problems are composed of complex and insufficient data. Spiking Neural Network (SNN) is the third generation of Artificial Neural Network (ANN), in which information can be transferred from one neuron to another using spike, processed, and trigger response as output. This study, hence, embedded spiking neurons for SOM learning in order to enhance the learning process. The proposed method was divided into five main phases. Phase 1 investigated issues related to SOM learning algorithm, while in Phase 2; datasets were collected for analyses carried out in Phase 3, wherein neural coding scheme for data representation process was implemented in the classification task. Next, in Phase 4, the spiking SOM model was designed, developed, and evaluated using classification accuracy rate and quantisation error. The outcomes showed that the proposed model had successfully attained exceptional classification accuracy rate with low quantisation error to preserve the quality of the generated map based on original input data. Lastly, in the final phase, a Spiking 3D Growing SOM is proposed to address the surface reconstruction issue by enhancing the spiking SOM using 3D map structure in SOM algorithm with a growing grid mechanism. The application of spiking neurons to enhance the performance of SOM is relevant in this study due to its ability to spike and to send a reaction when special features are identified based on its learning of the presented datasets. The study outcomes contribute to the enhancement of SOM in learning the patterns of the datasets, as well as in proposing a better tool for data analysis

Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks

Throughout history, the development of artificial intelligence, particularly artificial neural networks, has been open to and constantly inspired by the increasingly deepened understanding of the brain, such as the inspiration of neocognitron, which is the pioneering work of convolutional neural networks. Per the motives of the emerging field: NeuroAI, a great amount of neuroscience knowledge can help catalyze the next generation of AI by endowing a network with more powerful capabilities. As we know, the human brain has numerous morphologically and functionally different neurons, while artificial neural networks are almost exclusively built on a single neuron type. In the human brain, neuronal diversity is an enabling factor for all kinds of biological intelligent behaviors. Since an artificial network is a miniature of the human brain, introducing neuronal diversity should be valuable in terms of addressing those essential problems of artificial networks such as efficiency, interpretability, and memory. In this Primer, we first discuss the preliminaries of biological neuronal diversity and the characteristics of information transmission and processing in a biological neuron. Then, we review studies of designing new neurons for artificial networks. Next, we discuss what gains can neuronal diversity bring into artificial networks and exemplary applications in several important fields. Lastly, we discuss the challenges and future directions of neuronal diversity to explore the potential of NeuroAI

Bio-Inspired Systems: Computational and Ambient Intelligence. 10th International Work-Conference on Artificial Neural Networks, IWANN 2009, Salamanca, Spain, June 10-12, 2009. Proceedings, Part I

This book constitutes the refereed proceedings of the 10th International Work-Conference on Artificial Neural Networks, IWANN 2009, held in Salamanca, Spain in June 2009. The 167 revised full papers presented together with 3 invited lectures were carefully reviewed and selected from over 230 submissions. The papers are organized in thematic sections on theoretical foundations and models; learning and adaptation; self-organizing networks, methods and applications; fuzzy systems; evolutionary computation and genetic algoritms; pattern recognition; formal languages in linguistics; agents and multi-agent on intelligent systems; brain-computer interfaces (bci); multiobjetive optimization; robotics; bioinformatics; biomedical applications; ambient assisted living (aal) and ambient intelligence (ai); other applications

Statistical Data Modeling and Machine Learning with Applications

The modeling and processing of empirical data is one of the main subjects and goals of statistics. Nowadays, with the development of computer science, the extraction of useful and often hidden information and patterns from data sets of different volumes and complex data sets in warehouses has been added to these goals. New and powerful statistical techniques with machine learning (ML) and data mining paradigms have been developed. To one degree or another, all of these techniques and algorithms originate from a rigorous mathematical basis, including probability theory and mathematical statistics, operational research, mathematical analysis, numerical methods, etc. Popular ML methods, such as artificial neural networks (ANN), support vector machines (SVM), decision trees, random forest (RF), among others, have generated models that can be considered as straightforward applications of optimization theory and statistical estimation. The wide arsenal of classical statistical approaches combined with powerful ML techniques allows many challenging and practical problems to be solved. This Special Issue belongs to the section “Mathematics and Computer Science”. Its aim is to establish a brief collection of carefully selected papers presenting new and original methods, data analyses, case studies, comparative studies, and other research on the topic of statistical data modeling and ML as well as their applications. Particular attention is given, but is not limited, to theories and applications in diverse areas such as computer science, medicine, engineering, banking, education, sociology, economics, among others. The resulting palette of methods, algorithms, and applications for statistical modeling and ML presented in this Special Issue is expected to contribute to the further development of research in this area. We also believe that the new knowledge acquired here as well as the applied results are attractive and useful for young scientists, doctoral students, and researchers from various scientific specialties

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Computational Intelligence in Healthcare

This book is a printed edition of the Special Issue Computational Intelligence in Healthcare that was published in Electronic

Recurrent error-based ridge polynomial neural networks for time series forecasting

Time series forecasting has attracted much attention due to its impact on many practical applications. Neural networks (NNs) have been attracting widespread interest as a promising tool for time series forecasting. The majority of NNs employ only autoregressive (AR) inputs (i.e., lagged time series values) when forecasting time series. Moving-average (MA) inputs (i.e., errors) however have not adequately considered. The use of MA inputs, which can be done by feeding back forecasting errors as extra network inputs, alongside AR inputs help to produce more accurate forecasts. Among numerous existing NNs architectures, higher order neural networks (HONNs), which have a single layer of learnable weights, were considered in this research work as they have demonstrated an ability to deal with time series forecasting and have an simple architecture. Based on two HONNs models, namely the feedforward ridge polynomial neural network (RPNN) and the recurrent dynamic ridge polynomial neural network (DRPNN), two recurrent error-based models were proposed. These models were called the ridge polynomial neural network with error feedback (RPNN-EF) and the ridge polynomial neural network with error-output feedbacks (RPNN-EOF). Extensive simulations covering ten time series were performed. Besides RPNN and DRPNN, a pi-sigma neural network and a Jordan pi-sigma neural network were used for comparison. Simulation results showed that introducing error feedback to the models lead to significant forecasting performance improvements. Furthermore, it was found that the proposed models outperformed many state-of-the-art models. It was concluded that the proposed models have the capability to efficiently forecast time series and that practitioners could benefit from using these forecasting models

Computational Intelligence in Healthcare

The number of patient health data has been estimated to have reached 2314 exabytes by 2020. Traditional data analysis techniques are unsuitable to extract useful information from such a vast quantity of data. Thus, intelligent data analysis methods combining human expertise and computational models for accurate and in-depth data analysis are necessary. The technological revolution and medical advances made by combining vast quantities of available data, cloud computing services, and AI-based solutions can provide expert insight and analysis on a mass scale and at a relatively low cost. Computational intelligence (CI) methods, such as fuzzy models, artificial neural networks, evolutionary algorithms, and probabilistic methods, have recently emerged as promising tools for the development and application of intelligent systems in healthcare practice. CI-based systems can learn from data and evolve according to changes in the environments by taking into account the uncertainty characterizing health data, including omics data, clinical data, sensor, and imaging data. The use of CI in healthcare can improve the processing of such data to develop intelligent solutions for prevention, diagnosis, treatment, and follow-up, as well as for the analysis of administrative processes. The present Special Issue on computational intelligence for healthcare is intended to show the potential and the practical impacts of CI techniques in challenging healthcare applications