4,563 research outputs found
Metaheuristic design of feedforward neural networks: a review of two decades of research
Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era
Recommended from our members
Combinatorial optimization and metaheuristics
Today, combinatorial optimization is one of the youngest and most active areas of discrete mathematics. It is a branch of optimization in applied mathematics and computer science, related to operational research, algorithm theory and computational complexity theory. It sits at the intersection of several fields, including artificial intelligence, mathematics and software engineering. Its increasing interest arises for the fact that a large number of scientific and industrial problems can be formulated as abstract combinatorial optimization problems, through graphs and/or (integer) linear programs. Some of these problems have polynomial-time (“efficient”) algorithms, while most of them are NP-hard, i.e. it is not proved that they can be solved in polynomial-time. Mainly, it means that it is not possible to guarantee that an exact solution to the problem can be found and one has to settle for an approximate solution with known performance guarantees. Indeed, the goal of approximate methods is to find “quickly” (reasonable run-times), with “high” probability, provable “good” solutions (low error from the real optimal solution). In the last 20 years, a new kind of algorithm commonly called metaheuristics have emerged in this class, which basically try to combine heuristics in high level frameworks aimed at efficiently and effectively exploring the search space. This report briefly outlines the components, concepts, advantages and disadvantages of different metaheuristic approaches from a conceptual point of view, in order to analyze their similarities and differences. The two very significant forces of intensification and diversification, that mainly determine the behavior of a metaheuristic, will be pointed out. The report concludes by exploring the importance of hybridization and integration methods
Review of Metaheuristics and Generalized Evolutionary Walk Algorithm
Metaheuristic algorithms are often nature-inspired, and they are becoming
very powerful in solving global optimization problems. More than a dozen of
major metaheuristic algorithms have been developed over the last three decades,
and there exist even more variants and hybrid of metaheuristics. This paper
intends to provide an overview of nature-inspired metaheuristic algorithms,
from a brief history to their applications. We try to analyze the main
components of these algorithms and how and why they works. Then, we intend to
provide a unified view of metaheuristics by proposing a generalized
evolutionary walk algorithm (GEWA). Finally, we discuss some of the important
open questions.Comment: 14 page
Firefly Algorithm: Recent Advances and Applications
Nature-inspired metaheuristic algorithms, especially those based on swarm
intelligence, have attracted much attention in the last ten years. Firefly
algorithm appeared in about five years ago, its literature has expanded
dramatically with diverse applications. In this paper, we will briefly review
the fundamentals of firefly algorithm together with a selection of recent
publications. Then, we discuss the optimality associated with balancing
exploration and exploitation, which is essential for all metaheuristic
algorithms. By comparing with intermittent search strategy, we conclude that
metaheuristics such as firefly algorithm are better than the optimal
intermittent search strategy. We also analyse algorithms and their implications
for higher-dimensional optimization problems.Comment: 15 page
Efficiency Analysis of Swarm Intelligence and Randomization Techniques
Swarm intelligence has becoming a powerful technique in solving design and
scheduling tasks. Metaheuristic algorithms are an integrated part of this
paradigm, and particle swarm optimization is often viewed as an important
landmark. The outstanding performance and efficiency of swarm-based algorithms
inspired many new developments, though mathematical understanding of
metaheuristics remains partly a mystery. In contrast to the classic
deterministic algorithms, metaheuristics such as PSO always use some form of
randomness, and such randomization now employs various techniques. This paper
intends to review and analyze some of the convergence and efficiency associated
with metaheuristics such as firefly algorithm, random walks, and L\'evy
flights. We will discuss how these techniques are used and their implications
for further research.Comment: 10 pages. arXiv admin note: substantial text overlap with
arXiv:1212.0220, arXiv:1208.0527, arXiv:1003.146
Bat Algorithm: Literature Review and Applications
Bat algorithm (BA) is a bio-inspired algorithm developed by Yang in 2010 and
BA has been found to be very efficient. As a result, the literature has
expanded significantly in the last 3 years. This paper provides a timely review
of the bat algorithm and its new variants. A wide range of diverse applications
and case studies are also reviewed and summarized briefly here. Further
research topics are also discussed.Comment: 10 page
- …