A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have been proposed to connect the current layer with its forward layers apart from its next layer, which has been proved to be able to facilitate the training process of deep CNNs. However, there are various ways to build the shortcut connections, it is hard to manually design the best shortcut connections when solving a particular problem, especially given the design of the network architecture is already very challenging. In this paper, a hybrid evolutionary computation (EC) method is proposed to \textit{automatically} evolve both the architecture of deep CNNs and the shortcut connections. Three major contributions of this work are: Firstly, a new encoding strategy is proposed to encode a CNN, where the architecture and the shortcut connections are encoded separately; Secondly, a hybrid two-level EC method, which combines particle swarm optimisation and genetic algorithms, is developed to search for the optimal CNNs; Lastly, an adjustable learning rate is introduced for the fitness evaluations, which provides a better learning rate for the training process given a fixed number of epochs. The proposed algorithm is evaluated on three widely used benchmark datasets of image classification and compared with 12 peer Non-EC based competitors and one EC based competitor. The experimental results demonstrate that the proposed method outperforms all of the peer competitors in terms of classification accuracy

Clustering search

This paper presents the Clustering Search (CS) as a new hybrid metaheuristic, which works in conjunction with other metaheuristics, managing the implementation of local search algorithms for optimization problems. Usually the local search is costly and should be used only in promising regions of the search space. The CS assists in the discovery of these regions by dividing the search space into clusters. The CS and its applications are reviewed and a case study for a problem of capacitated clustering is presented.Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Universidade Federal do MaranhãoUniversidade Federal de São Paulo (UNIFESP)Instituto Nacional de Pesquisas EspaciaisUNIFESPSciEL

Knowledge management for computational intelligence systems

Proceedings of the 8th IEEE International Symposium on High Assurance Systems Engineering, HASE 2004, pp. 116–125.Computer systems do not learn from previous experiences unless they are designed for this purpose. Computational intelligence systems (CIS) are inherently capable of dealing with imprecise contexts, creating a new solution in each new execution. Therefore, every execution of a CIS is valuable to be learned. We describe an architecture for designing CIS that includes a knowledge management (KM) framework, allowing the system to learn from its own experiences, and those learned in external contexts. This framework makes the system flexible and adaptable so it evolves, guaranteeing high levels of reliability when performing in a dynamic world. This KM framework is being incorporated into the computational intelligence tool for software testing at National Institute for Systems Test and Productivity. This paper introduces the framework describing the two underlying methodologies it uses, i.e. case-based reasoning and monitored distribution; it also details the motivation and requirements for incorporating the framework into CIS

Fuel-efficient driving strategies

This thesis is concerned with fuel-efficient driving strategies for vehicles driving on roads with varying topography, as well as estimation of road grade\ua0and vehicle mass for vehicles utilizing such strategies. A framework referred\ua0to as speed profile optimization (SPO), is introduced for reducing the fuel\ua0or energy consumption of single vehicles (equipped with either combustion\ua0or electric engines) and platoons of several vehicles. Using the SPO-based\ua0methods, average reductions of 11.5% in fuel consumption for single trucks,\ua07.5 to 12.6% energy savings in electric vehicles, and 15.8 to 17.4% average\ua0fuel consumption reductions for platoons of trucks were obtained. Moreover,\ua0SPO-based methods were shown to achieve higher savings compared to\ua0the commonly used methods for fuel-efficient driving. Furthermore, it was\ua0demonstrated that the simulations are sufficiently accurate to be transferred\ua0to real trucks. In the SPO-based methods, the optimized speed profiles were\ua0generated using a genetic algorithm for which it was demonstrated, in a\ua0discretized case, that it is able to produce speed profiles whose fuel consumption\ua0is within 2% of the theoretical optimum.A feedforward neural network (FFNN) approach, with a simple feedback\ua0mechanism, is introduced and evaluated in simulations, for simultaneous estimation of the road grade and vehicle mass. The FFNN provided road grade\ua0estimates with root mean square (RMS) error of around 0.10 to 0.14 degrees,\ua0as well as vehicle mass estimates with an average RMS error of 1%, relative\ua0to the actual value. The estimates obtained with the FFNN outperform road\ua0grade and mass estimates obtained with other approaches

Energy Harvesting and Energy Storage Systems

This book discuss the recent developments in energy harvesting and energy storage systems. Sustainable development systems are based on three pillars: economic development, environmental stewardship, and social equity. One of the guiding principles for finding the balance between these pillars is to limit the use of non-renewable energy sources

Algoritmos genéticos aplicados na otimização de antenas refletoras

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-Graduação em Engenharia Elétrica.Muito se tem estudado sobre Otimização Estocástica com Algoritmos Genéticos. Sua aplicação em problemas eletromagnéticos complexos tem sido bem explorada e devidamente qualificada como de grande utilidade pela comunidade científica. Esta dissertação apresenta uma breve revisão dos Algoritmos Genéticos, descrevendo os conceitos básicos e as ferramentas usadas para a melhoria da convergência deste método de otimização. Como contribuição desta dissertação foi desenvolvida uma nova metodologia para os operadores genéticos, utilizando codificação real, com o objetivo de melhorar a varredura do espaço de busca da solução ótima. A eficácia dos Algoritmos Genéticos e destes novos operadores genéticos é verificada através de sua aplicação em diversas funções teste. Para aplicação deste método de otimização em um problema eletromagnético, optou-se pela conformação da superfície do refletor de uma antena refletora offset. O objetivo desta otimização é obter uma antena de satélite que produza um diagrama de radiação que cubra uniformemente o território brasileiro. Conforme demonstrado pelos resultados obtidos, tanto para a antena como para as funções teste, pode-se afirmar que os Algoritmos Genéticos são um método eficiente e confiável para a otimização de problemas complexos

A study of search neighbourhood in the bees algorithm

The Bees Algorithm, a heuristic optimisation procedure that mimics bees foraging behaviour, is becoming more popular among swarm intelligence researchers. The algorithm involves neighbourhood and global search and is able to find promising solutions to complex multimodal optimisation problems. The purpose of neighbourhood search is to intensify the search effort around promising solutions, while global search is to enable avoidance of local optima. Despite numerous studies aimed at enhancing the Bees Algorithm, there have not been many attempts at studying neighbourhood search. This research investigated different kinds of neighbourhoods and their effects on neighbourhood search. First, the adaptive enlargement of the search neighbourhood was proposed. This idea was implemented in the Bees Algorithm and tested on a set of mathematical benchmarks. The modified algorithm was also tested on single objective engineering design problems. The experimental results obtained confirmed that the adaptive enlargement of the search neighbourhood improved the performance of the proposed algorithm. Normally, a symmetrical search neighbourhood is employed in the Bees Algorithm. As opposed to this practice, an asymmetrical search neighbourhood was tried in this work to determine the significance of neighbourhood symmetry. In addition to the mathematical benchmarks, the algorithm with an asymmetrical search neighbourhood was also tested on an engineering design problem. The analysis verified that under certain measurements of asymmetry, the proposed ii algorithm produced a similar performance as that of the Bees Algorithm. For this reason, it was concluded that users were free to employ either a symmetrical or an asymmetrical search neighbourhood in the Bees Algorithm. Finally, the combination of adaptive enlargement and reduction of the search neighbourhood was presented. In addition to the above mathematical benchmarks and engineering design problems, a multi-objective design optimisation exercise with constraints was selected to demonstrate the performance of the modified algorithm. The experimental results obtained showed that this combination was beneficial to the proposed algorithm.EThOS - Electronic Theses Online ServiceGBUnited Kingdo