Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Machine Learning and Integrative Analysis of Biomedical Big Data.

Recent developments in high-throughput technologies have accelerated the accumulation of massive amounts of omics data from multiple sources: genome, epigenome, transcriptome, proteome, metabolome, etc. Traditionally, data from each source (e.g., genome) is analyzed in isolation using statistical and machine learning (ML) methods. Integrative analysis of multi-omics and clinical data is key to new biomedical discoveries and advancements in precision medicine. However, data integration poses new computational challenges as well as exacerbates the ones associated with single-omics studies. Specialized computational approaches are required to effectively and efficiently perform integrative analysis of biomedical data acquired from diverse modalities. In this review, we discuss state-of-the-art ML-based approaches for tackling five specific computational challenges associated with integrative analysis: curse of dimensionality, data heterogeneity, missing data, class imbalance and scalability issues

Enforcement of the principal component analysis - extreme learning machine algorithm by linear discriminant analysis

In the majority of traditional extreme learning machine (ELM) approaches, the parameters of the basis functions are randomly generated and do not need to be tuned, while the weights connecting the hidden layer to the output layer are analytically estimated. The determination of the optimal number of basis functions to be included in the hidden layer is still an open problem. Cross-validation and heuristic approaches (constructive and destructive) are some of the methodologies used to perform this task. Recently, a deterministic algorithm based on the principal component analysis (PCA) and ELM has been proposed to assess the number of basis functions according to the number of principal components necessary to explain the 90 % of the variance in the data. In this work, the PCA part of the PCA–ELM algorithm is joined to the linear discriminant analysis (LDA) as a hybrid means to perform the pruning of the hidden nodes. This is justified by the fact that the LDA approach is outperforming the PCA one on a set of problems. Hence, the idea of combining the two approaches in a LDA–PCA–ELM algorithm is shown to be in average better than its PCA–ELM and LDA–ELM counterparts. Moreover, the performance in classification and the number of basis functions selected by the algorithm, on a set of benchmark problems, have been compared and validated in the experimental section using nonparametric tests against a set of existing ELM techniques

Application of machine learning techniques to analyse the effects of physical exercise in ventricular fibrillation

This work presents the application of machine learning techniques to analyze the influence of physical exercise in the heart's physiological properties, during ventricular fibrillation. With that purpose, different kinds of classifiers (linear and neural models) were used to classify between trained and sedentary rabbit hearts. These classifiers were used to perform knowledge extraction through a wrapper feature selection algorithm. The obtained results showed the higher performance of the neural models compared to the linear classifier (higher performance measures and higher dimensionality reduction). The most relevant features to describe the benefits of physical exercise are those related to myocardial heterogeneity, mean activation rate and activation complexity

ELM ZA KLASIFIKACIJU TUMORA MOZGA KOD 3D MR SNIMAKA

Extreme Learning machine (ELM) a widely adopted algorithm in machine learning field is proposed for the use of pattern classification model using 3D MRI images for identifying tissue abnormalities in brain histology. The four class classification includes gray matter, white matter, cerebrospinal-fluid and tumor. The 3D MRI assessed by a pathologist indicates the ROI and the images are normalized. Texture features for each of the sub-regions is based on the Run-length Matrix, Co-occurence Matrix, Intensity, Euclidean distance, Gradient vector and neighbourhood statistics. Genetic Algorithm is custom designed to extract and sub-select a decisive optimal bank of features which are then used to model the ELM classifier and best selection of ELM algorithm parameters to handle sparse image data. The algorithm is explored using different activation function and the effect of number of neurons in the hidden layer by using different ratios of the number of features in the training and test data. The ELM classification outperformed in terms of accuracy, sensitivity and specificity as 93.20 %, 91.6 %, and 97.98% for discrimination of brain and pathological tumor tissue classification against state-of-the-art feature extraction methods and classifiers in the literature for publicly available SPL dataset.ELM, široko prihvaćen algoritam strojnog učenja se predlaže za korištenje u uzorkovanju pomoću klasifikacijskog modela 3D MRI slika za identifikaciju abnormalnosti tkiva u histologiji mozga. Četiri klase obuhvaćaju sive, bijele tvari, cerebrospinalne tekućine-i tumore. 3D MRI koji ocjenjuje patolog, ukazuje na ROI, a slike su normalizirane. Značajke tekstura za svaku od podregija se temelje na Run-length matrici, ponovnom pojavljivanju matrice, intenzitet, euklidska udaljenost, gradijent vektora i statistike susjedstva. Genetski algoritam je obično dizajniran za izdvajanje i sub-optimalan odabir odlučujući o značajkama koje se onda koriste za model ELM klasifikatora i najbolji izbor ELM parametra algoritama za obradu rijetkih slikovnih podataka. Algoritam se istražuje koristeći različite aktivacijske funkcije i utjecaj broja neurona u skrivenom sloju pomoću različitih omjera broja značajki kod trening i test podataka. ELM klasifikacija je nadmašila u smislu točnosti, osjetljivosti i specifičnosti, kao 93,20%, 91,6% i 97,98% za diskriminaciju mozga i patološki kod tumora i sistematizacije metode za prikupljanje podataka i klasifikatore u literaturi za javno dostupne SPL skup podataka

Moving Learning Machine Towards Fast Real-Time Applications: A High-Speed FPGA-based Implementation of the OS-ELM Training Algorithm

Currently, there are some emerging online learning applications handling data streams in real-time. The On-line Sequential Extreme Learning Machine (OS-ELM) has been successfully used in real-time condition prediction applications because of its good generalization performance at an extreme learning speed, but the number of trainings by a second (training frequency) achieved in these continuous learning applications has to be further reduced. This paper proposes a performance-optimized implementation of the OS-ELM training algorithm when it is applied to real-time applications. In this case, the natural way of feeding the training of the neural network is one-by-one, i.e., training the neural network for each new incoming training input vector. Applying this restriction, the computational needs are drastically reduced. An FPGA-based implementation of the tailored OS-ELMalgorithm is used to analyze, in a parameterized way, the level of optimization achieved. We observed that the tailored algorithm drastically reduces the number of clock cycles consumed for the training execution up to approximately the 1%. This performance enables high-speed sequential training ratios, such as 14 KHz of sequential training frequency for a 40 hidden neurons SLFN, or 180 Hz of sequential training frequency for a 500 hidden neurons SLFN. In practice, the proposed implementation computes the training almost 100 times faster, or more, than other applications in the bibliography. Besides, clock cycles follows a quadratic complexity O(N 2), with N the number of hidden neurons, and are poorly influenced by the number of input neurons. However, it shows a pronounced sensitivity to data type precision even facing small-size problems, which force to use double floating-point precision data types to avoid finite precision arithmetic effects. In addition, it has been found that distributed memory is the limiting resource and, thus, it can be stated that current FPGA devices can support OS-ELM-based on-chip learning of up to 500 hidden neurons. Concluding, the proposed hardware implementation of the OS-ELM offers great possibilities for on-chip learning in portable systems and real-time applications where frequent and fast training is required

µG2-ELM: an upgraded implementation of µ G-ELM

µG-ELM is a multiobjective evolutionary algorithm which looks for the best (in terms of the MSE) and most compact artificial neural network using the ELM methodology. In this work we present the µG2-ELM, an upgraded version of µG-ELM, previously presented by the authors. The upgrading is based on three key elements: a specifically designed approach for the initialization of the weights of the initial artificial neural networks, the introduction of a re-sowing process when selecting the population to be evolved and a change of the process used to modify the weights of the artificial neural networks. To test our proposal we consider several state-of-the-art Extreme Learning Machine (ELM) algorithms and we confront them using a wide and well-known set of continuous, regression and classification problems. From the conducted experiments it is proved that the µG2-ELM shows a better general performance than the previous version and also than other competitors. Therefore, we can guess that the combination of evolutionary algorithms with the ELM methodology is a promising subject of study since both together allow for the design of better training algorithms for artificial neural networks