Ensembles of Biologically Inspired Optimization Algorithms for Training Multilayer Perceptron Neural Networks

Artificial neural networks have proven to be effective in a wide range of fields, providing solutions to various problems. Training artificial neural networks using evolutionary algorithms is known as neuroevolution. The idea of finding not only the optimal weights and biases of a neural network but also its architecture has drawn the attention of many researchers. In this paper, we use different biologically inspired optimization algorithms to train multilayer perceptron neural networks for generating regression models. Specifically, our contribution involves analyzing and finding a strategy for combining several algorithms into a hybrid ensemble optimizer, which we apply for the optimization of a fully connected neural network. The goal is to obtain good regression models for studying and making predictions for the process of free radical polymerization of methyl methacrylate (MMA). In the first step, we use a search procedure to find the best parameter values for seven biologically inspired optimization algorithms. In the second step, we use a subset of the best-performing algorithms and improve the search capability by combining the chosen algorithms into an ensemble of optimizers. We propose three ensemble strategies that do not involve changes in the logic of optimization algorithms: hybrid cascade, hybrid single elite solution, and hybrid multiple elite solutions. The proposed strategies inherit the advantages of each individual optimizer and have faster convergence at a computational effort very similar to an individual optimizer. Our experimental results show that the hybrid multiple elite strategy ultimately produces neural networks which constitute the most dependable regression models for the aforementioned process

Ensembles of Biologically Inspired Optimization Algorithms for Training Multilayer Perceptron Neural Networks

Artificial neural networks have proven to be effective in a wide range of fields, providing solutions to various problems. Training artificial neural networks using evolutionary algorithms is known as neuroevolution. The idea of finding not only the optimal weights and biases of a neural network but also its architecture has drawn the attention of many researchers. In this paper, we use different biologically inspired optimization algorithms to train multilayer perceptron neural networks for generating regression models. Specifically, our contribution involves analyzing and finding a strategy for combining several algorithms into a hybrid ensemble optimizer, which we apply for the optimization of a fully connected neural network. The goal is to obtain good regression models for studying and making predictions for the process of free radical polymerization of methyl methacrylate (MMA). In the first step, we use a search procedure to find the best parameter values for seven biologically inspired optimization algorithms. In the second step, we use a subset of the best-performing algorithms and improve the search capability by combining the chosen algorithms into an ensemble of optimizers. We propose three ensemble strategies that do not involve changes in the logic of optimization algorithms: hybrid cascade, hybrid single elite solution, and hybrid multiple elite solutions. The proposed strategies inherit the advantages of each individual optimizer and have faster convergence at a computational effort very similar to an individual optimizer. Our experimental results show that the hybrid multiple elite strategy ultimately produces neural networks which constitute the most dependable regression models for the aforementioned process

A Hybrid Competitive Evolutionary Neural Network Optimization Algorithm for a Regression Problem in Chemical Engineering

Neural networks have demonstrated their usefulness for solving complex regression problems in circumstances where alternative methods do not provide satisfactory results. Finding a good neural network model is a time-consuming task that involves searching through a complex multidimensional hyperparameter and weight space in order to find the values that provide optimal convergence. We propose a novel neural network optimizer that leverages the advantages of both an improved evolutionary competitive algorithm and gradient-based backpropagation. The method consists of a modified, hybrid variant of the Imperialist Competitive Algorithm (ICA). We analyze multiple strategies for initialization, assimilation, revolution, and competition, in order to find the combination of ICA steps that provides optimal convergence and enhance the algorithm by incorporating a backpropagation step in the ICA loop, which, together with a self-adaptive hyperparameter adjustment strategy, significantly improves on the original algorithm. The resulting hybrid method is used to optimize a neural network to solve a complex problem in the field of chemical engineering: the synthesis and swelling behavior of the semi- and interpenetrated multicomponent crosslinked structures of hydrogels, with the goal of predicting the yield in a crosslinked polymer and the swelling degree based on several reaction-related input parameters. We show that our approach has better performance than other biologically inspired optimization algorithms and generates regression models capable of making predictions that are better correlated with the desired outputs

Obtaining Bricks Using Silicon-Based Materials: Experiments, Modeling and Optimization with Artificial Intelligence Tools

In the brick manufacturing industry, there is a growing concern among researchers to find solutions to reduce energy consumption. An industrial process for obtaining bricks was approached, with the manufacturing mix modified via the introduction of sunflower seed husks and sawdust. The process was analyzed with artificial intelligence tools, with the goal of minimizing the exhaust emissions of CO and CH4. Optimization algorithms inspired by human and virus behaviors were applied in this approach, which were associated with neural network models. A series of feed-forward neural networks have been developed, with 6 inputs corresponding to the working conditions, one or two intermediate layers and one output (CO or CH4, respectively). The results for ten biologically inspired algorithms and a search grid method were compared successfully within a single objective optimization procedure. It was established that by introducing 1.9% sunflower seed husks and 0.8% sawdust in the brick manufacturing mix, a minimum quantity of CH4 emissions was obtained, while 0% sunflower seed husks and 0.5% sawdust were the minimum quantities for CO emissions