50 research outputs found
Metaheuristic Algorithms for Convolution Neural Network
A typical modern optimization technique is usually either heuristic or
metaheuristic. This technique has managed to solve some optimization problems
in the research area of science, engineering, and industry. However,
implementation strategy of metaheuristic for accuracy improvement on
convolution neural networks (CNN), a famous deep learning method, is still
rarely investigated. Deep learning relates to a type of machine learning
technique, where its aim is to move closer to the goal of artificial
intelligence of creating a machine that could successfully perform any
intellectual tasks that can be carried out by a human. In this paper, we
propose the implementation strategy of three popular metaheuristic approaches,
that is, simulated annealing, differential evolution, and harmony search, to
optimize CNN. The performances of these metaheuristic methods in optimizing CNN
on classifying MNIST and CIFAR dataset were evaluated and compared.
Furthermore, the proposed methods are also compared with the original CNN.
Although the proposed methods show an increase in the computation time, their
accuracy has also been improved (up to 7.14 percent).Comment: Article ID 1537325, 13 pages. Received 29 January 2016; Revised 15
April 2016; Accepted 10 May 2016. Academic Editor: Martin Hagan. in Hindawi
Publishing. Computational Intelligence and Neuroscience Volume 2016 (2016
Training Neural Networks with Stochastic Hessian-Free Optimization
Hessian-free (HF) optimization has been successfully used for training deep
autoencoders and recurrent networks. HF uses the conjugate gradient algorithm
to construct update directions through curvature-vector products that can be
computed on the same order of time as gradients. In this paper we exploit this
property and study stochastic HF with gradient and curvature mini-batches
independent of the dataset size. We modify Martens' HF for these settings and
integrate dropout, a method for preventing co-adaptation of feature detectors,
to guard against overfitting. Stochastic Hessian-free optimization gives an
intermediary between SGD and HF that achieves competitive performance on both
classification and deep autoencoder experiments.Comment: 11 pages, ICLR 201