Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Development of an Algorithm for Multicriteria Optimization of Deep Learning Neural Networks

Nowadays, machine learning methods are actively used to process big data. A promising direction is neural networks, in which structure optimization occurs on the principles of self-configuration. Genetic algorithms are applied to solve this nontrivial problem. Most multicriteria evolutionary algorithms use a procedure known as non-dominant sorting to rank decisions. However, the efficiency of procedures for adding points and updating rank values in non-dominated sorting (incremental non-dominated sorting) remains low. In this regard, this research improves the performance of these algorithms, including the condition of an asynchronous calculation of the fitness of individuals. The relevance of the research is determined by the fact that although many scholars and specialists have studied the self-tuning of neural networks, they have not yet proposed a comprehensive solution to this problem. In particular, algorithms for efficient non-dominated sorting under conditions of incremental and asynchronous updates when using evolutionary methods of multicriteria optimization have not been fully developed to date. To achieve this goal, a hybrid co-evolutionary algorithm was developed that significantly outperforms all algorithms included in it, including error-back propagation and genetic algorithms that operate separately. The novelty of the obtained results lies in the fact that the developed algorithms have minimal asymptotic complexity. The practical value of the developed algorithms is associated with the fact that they make it possible to solve applied problems of increased complexity in a practically acceptable time. Doi: 10.28991/HIJ-2023-04-01-011 Full Text: PD

Selected Issues of Design and Implementation od Decision Support Systems

The paper presents selected issues related to design and implementation of model based Decision Support Systems (DSS). For over ten years the SDS Program has been involved in cooperation with various projects at IIASA and in collaborating research institutes. This cooperation has resulted in the development of many DSS, which in turn stimulated research on the theory and methodology of decision analysis. An overview of selected DDS developed within the cooperation with IIASA is presented. Different concepts of DSS are briefly discussed and one specific type of DSS, namely model based, aspiration-led DSS is characterized. Finally, selected problems of designing and implementation of a DSS are discussed in more detail. A short description of software packages developed within the cooperation with the MDA Project is provided. The paper also gives a short summary of recent activities of the Methodology of Decision Analysis Project and of the DSS software available from the MDA Project

Efficient Learning Machines

Computer scienc

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Multiobjective Evolutionary Optimization for Prototype-Based Fuzzy Classifiers

Evolving intelligent systems (EISs), particularly, the zero-order ones have demonstrated strong performance on many real-world problems concerning data stream classification, while offering high model transparency and interpretability thanks to their prototype-based nature. Zero-order EISs typically learn prototypes by clustering streaming data online in a “one pass” manner for greater computation efficiency. However, such identified prototypes often lack optimality, resulting in less precise classification boundaries, thereby hindering the potential classification performance of the systems. To address this issue, a commonly adopted strategy is to minimise the training error of the models on historical training data or alternatively, to iteratively minimise the intra-cluster variance of the clusters obtained via online data partitioning. This recognises the fact that the ultimate classification performance of zero-order EISs is driven by the positions of prototypes in the data space. Yet, simply minimising the training error may potentially lead to overfitting, whilst minimising the intra-cluster variance does not necessarily ensure the optimised prototype-based models to attain improved classification outcomes. To achieve better classification performance whilst avoiding overfitting for zero-order EISs, this paper presents a novel multi-objective optimisation approach, enabling EISs to obtain optimal prototypes via involving these two disparate but complementary strategies simultaneously. Five decision-making schemes are introduced for selecting a suitable solution to deploy from the final non-dominated set of the resulting optimised models. Systematic experimental studies are carried out to demonstrate the effectiveness of the proposed optimisation approach in improving the classification performance of zero-order EISs

Multiobjective Design Optimization Of Gas Turbine Blade With Emphasis On Internal Cooling

In the design of mechanical components, numerical simulations and experimental methods are commonly used for design creation (or modification) and design optimization. However, a major challenge of using simulation and experimental methods is that they are timeconsuming and often cost-prohibitive for the designer. In addition, the simultaneous interactions between aerodynamic, thermodynamic and mechanical integrity objectives for a particular component or set of components are difficult to accurately characterize, even with the existing simulation tools and experimental methods. The current research and practice of using numerical simulations and experimental methods do little to address the simultaneous “satisficing” of multiple and often conflicting design objectives that influence the performance and geometry of a component. This is particularly the case for gas turbine systems that involve a large number of complex components with complicated geometries. Numerous experimental and numerical studies have demonstrated success in generating effective designs for mechanical components; however, their focus has been primarily on optimizing a single design objective based on a limited set of design variables and associated values. In this research, a multiobjective design optimization framework to solve a set of userspecified design objective functions for mechanical components is proposed. The framework integrates a numerical simulation and a nature-inspired optimization procedure that iteratively perturbs a set of design variables eventually converging to a set of tradeoff design solutions. In this research, a gas turbine engine system is used as the test application for the proposed framework. More specifically, the optimization of the gas turbine blade internal cooling channel configuration is performed. This test application is quite relevant as gas turbine engines serve a iv critical role in the design of the next-generation power generation facilities around the world. Furthermore, turbine blades require better cooling techniques to increase their cooling effectiveness to cope with the increase in engine operating temperatures extending the useful life of the blades. The performance of the proposed framework is evaluated via a computational study, where a set of common, real-world design objectives and a set of design variables that directly influence the set of objectives are considered. Specifically, three objectives are considered in this study: (1) cooling channel heat transfer coefficient, which measures the rate of heat transfer and the goal is to maximize this value; (2) cooling channel air pressure drop, where the goal is to minimize this value; and (3) cooling channel geometry, specifically the cooling channel cavity area, where the goal is to maximize this value. These objectives, which are conflicting, directly influence the cooling effectiveness of a gas turbine blade and the material usage in its design. The computational results show the proposed optimization framework is able to generate, evaluate and identify thousands of competitive tradeoff designs in a fraction of the time that it would take designers using the traditional simulation tools and experimental methods commonly used for mechanical component design generation. This is a significant step beyond the current research and applications of design optimization to gas turbine blades, specifically, and to mechanical components, in general