Metaheuristic Algorithms for Convolution Neural Network

A typical modern optimization technique is usually either heuristic or metaheuristic. This technique has managed to solve some optimization problems in the research area of science, engineering, and industry. However, implementation strategy of metaheuristic for accuracy improvement on convolution neural networks (CNN), a famous deep learning method, is still rarely investigated. Deep learning relates to a type of machine learning technique, where its aim is to move closer to the goal of artificial intelligence of creating a machine that could successfully perform any intellectual tasks that can be carried out by a human. In this paper, we propose the implementation strategy of three popular metaheuristic approaches, that is, simulated annealing, differential evolution, and harmony search, to optimize CNN. The performances of these metaheuristic methods in optimizing CNN on classifying MNIST and CIFAR dataset were evaluated and compared. Furthermore, the proposed methods are also compared with the original CNN. Although the proposed methods show an increase in the computation time, their accuracy has also been improved (up to 7.14 percent).Comment: Article ID 1537325, 13 pages. Received 29 January 2016; Revised 15 April 2016; Accepted 10 May 2016. Academic Editor: Martin Hagan. in Hindawi Publishing. Computational Intelligence and Neuroscience Volume 2016 (2016

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Using 2-Opt based evolution strategy for travelling salesman problem

Harmony search algorithm that matches the (µ+ 1) evolution strategy, is a heuristic method simulated by the process of music improvisation. In this paper, a harmony search algorithm is directly used for the travelling salesman problem. Instead of conventional selection operators such as roulette wheel, the transformation of real number values of harmony search algorithm to order index of vertex representation and improvement of solutions are obtained by using the 2-Opt local search algorithm. Then, the obtained algorithm is tested on two different parameter groups of TSPLIB. The proposed method is compared with classical 2-Opt which randomly started at each step and best known solutions of test instances from TSPLIB. It is seen that the proposed algorithm offers valuable solutions

Enhancing Harmony Search Parameters Based On Step And Linear Function For Bus Driver Scheduling And Rostering Problems

Optimization is a major challenge in numerous practical world problems.According to the “No Free Lunch (NFL)” theorem,there is no existing single optimizer algorithm that is able to resolve all issues in an effective and efficient manner.It is varied and need to be solved according to the specific capabilities inherent to certain algorithms making it hard to foresee the algorithm that is best suited for each problem.As a result,the heuristic technique is adopted for this research as it has been identified as a potentially suitable algorithm.Alternative heuristic algorithms are also suggested to obtain optimal solutions with reasonable computational effort.However,the heuristic approach failed to produce a solution that nears optimum when the complexity of a problem increases;therefore a type of nature-inspired algorithm known as meta-euristics which utilises an intelligent searching mechanism over a population is considered and consequently used.The meta-heuristic approach is widely used to substitute heuristic terms and is broadly applied to address problems with regards to driver scheduling.However,this meta-heuristic technique is still unable to address the fairness issue in the scheduling and rostering problems.Hence,this research proposes a strategy to adopt an amendment of the harmony search algorithm in order to address the fairness issue which in turn will escalate the level of fairness in driver scheduling and rostering.The harmony search algorithm is classified as a meta-heuristics algorithm that is capable of solving hard and combinatorial or discrete optimisation problems.In this respect,the three main operators in harmony search,namely the Harmony Memory Consideration Rate (HMCR),Pitch Adjustment Rate (PAR) and Bandwidth (BW) play a vital role in balancing local exploitation and global exploration.These parameters influence the overall performance of the HS algorithm,and therefore it is crucial to fine-tune them. Therefore,it is of great interest that we find adjustments for these parameters in this research.There are two contributions to this research.The first one is having HMCR parameter using step function and the linear increase function while the second is applying the fret spacing concept on guitars that is associated with mathematical formulae is also applied in the BW parameter.There are three proposed models on the alteration of HMCR parameters based on the use of the fundamental step function;namely,the constant interval of step function, and its dynamic increase and decrease interval functions.The experimental results revealed that our proposed approach is superior to other state of the art harmony searches either in specific or generic cases. This is achieved by using a first type of association between the linear increase function with a second model of the dynamic increase of step function being remarkable in other combinations and also other models.In conclusion,this proposed approach managed to generate a fairer roster and was thus capable of maximising the balancing distribution of shifts and routes among drivers,which contributed to the lowering of illness, incidents,absenteeism and accidents

Is swarm intelligence able to create mazes?

In this paper, the idea of applying Computational Intelligence in the process of creation board games, in particular mazes, is presented. For two different algorithms the proposed idea has been examined. The results of the experiments are shown and discussed to present advantages and disadvantages

Bat Algorithm: Literature Review and Applications

Bat algorithm (BA) is a bio-inspired algorithm developed by Yang in 2010 and BA has been found to be very efficient. As a result, the literature has expanded significantly in the last 3 years. This paper provides a timely review of the bat algorithm and its new variants. A wide range of diverse applications and case studies are also reviewed and summarized briefly here. Further research topics are also discussed.Comment: 10 page