Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Modeling Financial Time Series with Artificial Neural Networks

Financial time series convey the decisions and actions of a population of human actors over time. Econometric and regressive models have been developed in the past decades for analyzing these time series. More recently, biologically inspired artificial neural network models have been shown to overcome some of the main challenges of traditional techniques by better exploiting the non-linear, non-stationary, and oscillatory nature of noisy, chaotic human interactions. This review paper explores the options, benefits, and weaknesses of the various forms of artificial neural networks as compared with regression techniques in the field of financial time series analysis.CELEST, a National Science Foundation Science of Learning Center (SBE-0354378); SyNAPSE program of the Defense Advanced Research Project Agency (HR001109-03-0001

Bibliometric Mapping of the Computational Intelligence Field

In this paper, a bibliometric study of the computational intelligence field is presented. Bibliometric maps showing the associations between the main concepts in the field are provided for the periods 1996–2000 and 2001–2005. Both the current structure of the field and the evolution of the field over the last decade are analyzed. In addition, a number of emerging areas in the field are identified. It turns out that computational intelligence can best be seen as a field that is structured around four important types of problems, namely control problems, classification problems, regression problems, and optimization problems. Within the computational intelligence field, the neural networks and fuzzy systems subfields are fairly intertwined, whereas the evolutionary computation subfield has a relatively independent position.neural networks;bibliometric mapping;fuzzy systems;bibliometrics;computational intelligence;evolutionary computation

Two neural network algorithms for designing optimal terminal controllers with open final time

Multilayer neural networks, trained by the backpropagation through time algorithm (BPTT), have been used successfully as state-feedback controllers for nonlinear terminal control problems. Current BPTT techniques, however, are not able to deal systematically with open final-time situations such as minimum-time problems. Two approaches which extend BPTT to open final-time problems are presented. In the first, a neural network learns a mapping from initial-state to time-to-go. In the second, the optimal number of steps for each trial run is found using a line-search. Both methods are derived using Lagrange multiplier techniques. This theoretical framework is used to demonstrate that the derived algorithms are direct extensions of forward/backward sweep methods used in N-stage optimal control. The two algorithms are tested on a Zermelo problem and the resulting trajectories compare favorably to optimal control results

Neural networks for small scale ORC optimization

This study concerns a thermodynamic and technical optimization of a small scale Organic Rankine Cycle system for waste heat recovery applications. An Artificial Neural Network (ANN) has been used to develop a thermodynamic model to be used for the maximization of the production of power while keeping the size of the heat exchangers and hence the cost of the plant at its minimum. R1234yf has been selected as the working fluid. The results show that the use of ANN is promising in solving complex nonlinear optimization problems that arise in the field of thermodynamics

Integrating Evolutionary Computation with Neural Networks

There is a tremendous interest in the development of the evolutionary computation techniques as they are well suited to deal with optimization of functions containing a large number of variables. This paper presents a brief review of evolutionary computing techniques. It also discusses briefly the hybridization of evolutionary computation and neural networks and presents a solution of a classical problem using neural computing and evolutionary computing technique

Kajian terhadap ketahanan hentaman ke atas konkrit berbusa yang diperkuat dengan serat kelapa sawit

Konkrit berbusa merupakan sejenis konkrit ringan yang mempunyai kebolehkerjaan yang baik dan tidak memerlukan pengetaran untuk proses pemadatan. Umum mengenali konkrit berbusa sebagai bahan binaan yang mempunyai sifat kekuatan yang rendah dan lemah terutama apabila bahan binaan ini dikenakan tenaga hentaman yang tinggi. Namun begitu, konkrit berbusa merupakan bahan yang berpotensi untuk dijadikan sebagai bahan binaan yang berkonsepkan futuristik. Binaan futuristik adalah binaan yang bercirikan ringan, ekonomi, mudah dari segi kerja pembinaan dan yang paling penting adalah mesra alam. Dalam kajian ini, konkrit berbusa ditambah serat buangan pokok kelapa sawit untuk untuk meningkatkan sifat kekuatan atau rapuh. Serat kelapa sawit juga berfungsi mempertingkatkan ketahanan hentaman terutamanya aspek nilai penyerapan tenaga hentaman dan nilai tenaga hentaman. Kandungan peratusan serat kelapa sawit yang digunakan adalah 10%, 20% dan 30% dengan dua ketumpatan konkrit berbusa iaitu 1000kg/m3 dan 1400kg/m3 . Untuk menentukan nilai penyerapan tenaga hentaman dan nilai tenaga hentaman, ujikaji Indentasi dan ujikaji hentaman dilakukan ke atas sampel�sampel yang telah diawet selama 28 hari. Luas bawah graf tegasan-terikan yang diperolehi daripada ujikaji Indentasi merupakan nilai penyerapan tenaga hentaman bagi sampel konkrit berbusa. Untuk ujikaji hentaman, keputusan ujikaji dinilai berdasarkan nilai tenaga hentaman untuk meretakkan sampel yang diperolehi daripada mesin ujikaji dynatup. Secara keseluruhannya, hasil dapatan utama bagi kedua-dua ujikaji menunjukkan sampel yang mengandungi peratusan serat kelapa sawit sebanyak 20% mempunyai nilai penyerapan tenaga hentaman dan nilai tenaga hentaman yang tinggi. Serapan tenaga maksimum adalah sebanyak 4.517MJ/m3 untuk ketumpatan 1400kg/m3 . Ini menunjukkan ketumpatan 1400kg/m3 berupaya menyerap tenaga lebih baik berbanding ketumpatan 1000kg/m3 . Manakala untuk nilai tenaga hentaman maksimum adalah sebanyak 27.229J untuk ketumpatan 1400kg/m3 . Hasil dapatan tersebut menunjukkan ketumpatan 1400kg/m3 dengan peratusan serat sebanyak 20% berupaya mengalas tenaga hentaman yang lebih banyak sebelum sampel retak. Kesimpulannya, peningkatan ketumpatan konkrit berbusa dan pertambahan serat buangan kelapa sawit ke dalam konkrit berbusa dapat meningkatkan ciri ketahanan hentaman konkrit berbusa khususnya aspek nilai penyerapan tenaga hentaman dan nilai tenaga hentaman

Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data

Subsequence clustering of multivariate time series is a useful tool for discovering repeated patterns in temporal data. Once these patterns have been discovered, seemingly complicated datasets can be interpreted as a temporal sequence of only a small number of states, or clusters. For example, raw sensor data from a fitness-tracking application can be expressed as a timeline of a select few actions (i.e., walking, sitting, running). However, discovering these patterns is challenging because it requires simultaneous segmentation and clustering of the time series. Furthermore, interpreting the resulting clusters is difficult, especially when the data is high-dimensional. Here we propose a new method of model-based clustering, which we call Toeplitz Inverse Covariance-based Clustering (TICC). Each cluster in the TICC method is defined by a correlation network, or Markov random field (MRF), characterizing the interdependencies between different observations in a typical subsequence of that cluster. Based on this graphical representation, TICC simultaneously segments and clusters the time series data. We solve the TICC problem through alternating minimization, using a variation of the expectation maximization (EM) algorithm. We derive closed-form solutions to efficiently solve the two resulting subproblems in a scalable way, through dynamic programming and the alternating direction method of multipliers (ADMM), respectively. We validate our approach by comparing TICC to several state-of-the-art baselines in a series of synthetic experiments, and we then demonstrate on an automobile sensor dataset how TICC can be used to learn interpretable clusters in real-world scenarios.Comment: This revised version fixes two small typos in the published versio