GAdaboost: Accelerating adaboost feature selection with genetic algorithms

Throughout recent years Machine Learning has acquired attention, due to the abundant data. Thus, devising techniques to reduce the dimensionality of data has been on going. Object detection is one of the Machine Learning techniques which suffer from this draw back. As an example, one of the most famous object detection frameworks is the Viola-Jones Rapid Object Detector, which suffers from a lengthy training process due to the vast search space, which can reach more than 160,000 features for a 24X24 image. The Viola-Jones Rapid Object Detector also uses Adaboost, which is a brute force method, and is required to pass by the set of all possible features in order to train the classifiers. Consequently, ways for reducing the whole feature set into a smaller representative one, eliminating those features that have non relevant information, were devised. The most commonly used technique for this is Feature Selection with its three categories: Filters, Wrappers and Embedded. Feature Selection has proven its success in providing fast and accurate classifiers. Wrapper methods harvest the power of evolutionary computing, most commonly Genetic Algorithms, in finding the set of representative features. This is mostly due to the Advantage of Genetic Algorithms and their power in finding adequate solutions more efficiently. In this thesis we propose GAdaboost: A Genetic Algorithm to accelerate the training procedure of the Viola-Jones Rapid Object Detector through Feature Selection. Specifically, we propose to limit the Adaboost search within a sub-set of the huge feature space, while evolving this subset following a Genetic Algorithm. Experiments demonstrate that our proposed GAdaboost is up to 3.7 times faster than Adaboost. We also demonstrate that the price of this speedup is a mere decrease (3%, 4%) in detection accuracy when tested on FDDB benchmark face detection set, and Caltech Web Faces respectivel

Bio-Inspired Optimization of Ultra-Wideband Patch Antennas Using Graphics Processing Unit Acceleration

Ultra-wideband (UWB) wireless systems have recently gained considerable attention as effective communications platforms with the properties of low power and high data rates. Applications of UWB such as wireless USB put size constraints on the antenna, however, which can be very dicult to meet using typical narrow band antenna designs. The aim of this thesis is to show how bio-inspired evolutionary optimization algorithms, in particular genetic algorithm (GA), particle swarm optimization (PSO) and biogeography-based optimization (BBO) can produce novel UWB planar patch antenna designs that meet a size constraint of a 10 mm 10 mm patch. Each potential antenna design is evaluated with the nite dierence time domain (FDTD) technique, which is accurate but time-consuming. Another aspect of this thesis is the modication of FDTD to run on a graphics processing unit (GPU) to obtain nearly a 20 speedup. With the combination of GA, PSO, BBO and GPU-accelerated FDTD, three novel antenna designs are produced that meet the size and bandwidth requirements applicable to UWB wireless USB system

Computer aided analysis of skin lesions

Effective screening to detect the skin cancer accurately in the early stage is essential for reducing the mortality of skin cancer. Surface features, such as texture and pigmentation area from the surface, epi-illumination images of the skin lesions have been well correlated to detect skin cancer. An increase in the lesion\u27s subsurface blood volume has been correlated to early diagnosis of malignant melanoma. A method for estimating the optimal features is obtained. The optimal features help in accurately classify the skin lesion in various grades. To make the process faster these optimal features are clustered. The optimal clusters are obtained by genetic algorithm. The optimal cluster centers act as input to the SVM classifier and the kernel parameters are obtained. Finally, parameters of the kernel function are optimized by genetic algorithm, which help in classifying the skin lesions into various grades leading to early diagnosis of skin cancer

Progressive insular cooperative genetic programming algorithm for multiclass classification

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsIn contrast to other types of optimisation algorithms, Genetic Programming (GP) simultaneously optimises a group of solutions for a given problem. This group is named population, the algorithm iterations are named generations and the optimisation is named evolution as a reference o the algorithm’s inspiration in Darwin’s theory on the evolution of species. When a GP algorithm uses a one-vs-all class comparison for a multiclass classification (MCC) task, the classifiers for each target class (specialists) are evolved in a subpopulation and the final solution of the GP is a team composed of one specialist classifier of each class. In this scenario, an important question arises: should these subpopulations interact during the evolution process or should they evolve separately? The current thesis presents the Progressively Insular Cooperative (PIC) GP, a MCC GP in which the level of interaction between specialists for different classes changes through the evolution process. In the first generations, the different specialists can interact more, but as the algorithm evolves, this level of interaction decreases. At a later point in the evolution process, controlled through algorithm parameterisation, these interactions can be eliminated. Thus, in the beginning of the algorithm there is more cooperation among specialists of different classes, favouring search space exploration. With elimination of cooperation, search space exploitation is favoured. In this work, different parameters of the proposed algorithm were tested using the Iris dataset from the UCI Machine Learning Repository. The results showed that cooperation among specialists of different classes helps the improvement of classifiers specialised in classes that are more difficult to discriminate. Moreover, the independent evolution of specialist subpopulations further benefits the classifiers when they already achieved good performance. A combination of the two approaches seems to be beneficial when starting with subpopulations of differently performing classifiers. The PIC GP also presented great performance for the more complex Thyroid and Yeast datasets of the same repository, achieving similar accuracy to the best values found in literature for other MCC models.Diferente de outros algoritmos de otimiação computacional, o algoritmo de Programação Genética PG otimiza simultaneamente um grupo de soluções para um determinado problema. Este grupo de soluções é chamado população, as iterações do algoritmo são chamadas de gerações e a otimização é chamada de evolução em alusão à inspiração do algoritmo na teoria da evolução das espécies de Darwin. Quando o algoritmo GP utiliza a abordagem de comparação de classes um-vs-todos para uma classificação multiclasses (CMC), os classificadores específicos para cada classe (especialistas) são evoluídos em subpopulações e a solução final do PG é uma equipe composta por um especialista de cada classe. Neste cenário, surge uma importante questão: estas subpopulações devem interagir durante o processo evolutivo ou devem evoluir separadamente? A presente tese apresenta o algoritmo Cooperação Progressivamente Insular (CPI) PG, um PG CMC em que o grau de interação entre especialistas em diferentes classes varia ao longo do processo evolutivo. Nas gerações iniciais, os especialistas de diferentes classes interagem mais. Com a evolução do algoritmo, estas interações diminuem e mais tarde, dependendo da parametriação do algoritmo, elas podem ser eliminadas. Assim, no início do processo evolutivo há mais cooperação entre os especialistas de diferentes classes, o que favorece uma exploração mais ampla do espaço de busca. Com a eliminação da cooperação, favorece-se uma exploração mais local e detalhada deste espaço. Foram testados diferentes parâmetros do PG CPl utilizando o conjunto de dados iris do UCI Machine Learning Repository. Os resultados mostraram que a cooperação entre especialistas de diferentes classes ajudou na melhoria dos classificadores de classes mais difíceis de modelar. Além disso, que a evolução sem a interação entre as classes de diferentes especialidades beneficiou os classificadores quando eles já apresentam boa performance Uma combinação destes dois modos pode ser benéfica quando o algoritmo começa com classificadores que apresentam qualidades diferentes. O PG CPI também apresentou ótimos resultados para outros dois conjuntos de dados mais complexos o thyroid e o yeast, do mesmo repositório, alcançando acurácia similar aos melhores valores encontrados na literatura para outros modelos de CMC

Efficient Reduced BIAS Genetic Algorithm for Generic Community Detection Objectives

The problem of community structure identification has been an extensively investigated area for biology, physics, social sciences, and computer science in recent years for studying the properties of networks representing complex relationships. Most traditional methods, such as K-means and hierarchical clustering, are based on the assumption that communities have spherical configurations. Lately, Genetic Algorithms (GA) are being utilized for efficient community detection without imposing sphericity. GAs are machine learning methods which mimic natural selection and scale with the complexity of the network. However, traditional GA approaches employ a representation method that dramatically increases the solution space to be searched by introducing redundancies. They also utilize a crossover operator which imposes a linear ordering that is not suitable for community detection. The algorithm presented here is a framework to detect communities for complex biological networks that removes both redundancies and linearity. We also introduce a novel operator, named Gene Repair. This algorithm is unique as it is a flexible community detection technique aimed at maximizing the value of any given mathematical objective for the network. We reduce the memory requirements by representing chromosomes as a 3-dimensional bit array. Furthermore, in order to increase diversity while retaining promising chromosomes, we use natural selection process based on tournament selection with elitism. Additionally, our approach doesn’t require prior information about the number of true communities in the network. We apply our novel algorithm to benchmark datasets and also to a network representing a large cohort of AD cases and controls. By utilizing this efficient and flexible implementation that is cognizant of characteristics for networks representing complex disease genetics, we sift out communities representing patterns of interacting genetic variants that are associated with this enigmatic disease

Class decomposition for GA-based classifier agents – A Pitt approach

Incremental learning has been widely addressed in the machine learning literature to cope with learning tasks where the learning environment is ever changing or training samples become available over time. However, most research work explores incremental learning with statistical algorithms or neural networks, rather than evolutionary algorithms. The work in this paper employs genetic algorithms (GAs) as basic learning algorithms for incremental learning within one or more classifier agents in a multi-agent environment. Four new approaches with different initialization schemes are proposed. They keep the old solutions and use an “integration” operation to integrate them with new elements to accommodate new attributes, while biased mutation and crossover operations are adopted to further evolve a reinforced solution. The simulation results on benchmark classification data sets show that the proposed approaches can deal with the arrival of new input attributes and integrate them with the original input space. It is also shown that the proposed approaches can be successfully used for incremental learning and improve classification rates as compared to the retraining GA. Possible applications for continuous incremental training and feature selection are also discussed

A niching memetic algorithm for simultaneous clustering and feature selection

Clustering is inherently a difficult task, and is made even more difficult when the selection of relevant features is also an issue. In this paper we propose an approach for simultaneous clustering and feature selection using a niching memetic algorithm. Our approach (which we call NMA_CFS) makes feature selection an integral part of the global clustering search procedure and attempts to overcome the problem of identifying less promising locally optimal solutions in both clustering and feature selection, without making any a priori assumption about the number of clusters. Within the NMA_CFS procedure, a variable composite representation is devised to encode both feature selection and cluster centers with different numbers of clusters. Further, local search operations are introduced to refine feature selection and cluster centers encoded in the chromosomes. Finally, a niching method is integrated to preserve the population diversity and prevent premature convergence. In an experimental evaluation we demonstrate the effectiveness of the proposed approach and compare it with other related approaches, using both synthetic and real data

An Elitist Non-Dominated Multi-Objective Genetic Algorithm Based Temperature Aware Circuit Synthesis

At sub-nanometre technology, temperature is one of the important design parameters to be taken care of during the target implementation for the circuit for its long term and reliable operation. High device package density leads to high power density that generates high temperatures. The temperature of a chip is directly proportional to the power density of the chip. So, the power density of a chip can be minimized to reduce the possibility of the high temperature generation. Temperature minimization approaches are generally addressed at the physical design level but it incurs high cooling cost. To reduce the cooling cost, the temperature minimization approaches can be addressed at the logic level. In this work, the Non-Dominated Sorting Genetic Algorithm-II (NSGA-II) based multi-objective heuristic approach is proposed to select the efficient input variable polarity of Mixed Polarity Reed-Muller (MPRM) expansion for simultaneous optimization of area, power, and temperature. A Pareto optimal solution set is obtained from the vast solution set of 3n (‘n’ is the number of input variables) different polarities of MPRM. Tabular technique is used for input polarity conversion from Sum-of-Product (SOP) form to MPRM form. Finally, using CADENCE and HotSpot tool absolute temperature, silicon area and power consumption of the synthesized circuits are calculated and are reported. The proposed algorithm saves around 76.20% silicon area, 29.09% power dissipation and reduces 17.06% peak temperature in comparison with the reported values in the literature

Comprehensible credit scoring models using rule extraction from support vector machines.

In recent years, Support Vector Machines (SVMs) were successfully applied to a wide range of applications. Their good performance is achieved by an implicit non-linear transformation of the original problem to a high-dimensional (possibly infinite) feature space in which a linear decision hyperplane is constructed that yields a nonlinear classifier in the input space. However, since the classifier is described as a complex mathematical function, it is rather incomprehensible for humans. This opacity property prevents them from being used in many real- life applications where both accuracy and comprehensibility are required, such as medical diagnosis and credit risk evaluation. To overcome this limitation, rules can be extracted from the trained SVM that are interpretable by humans and keep as much of the accuracy of the SVM as possible. In this paper, we will provide an overview of the recently proposed rule extraction techniques for SVMs and introduce two others taken from the artificial neural networks domain, being Trepan and G-REX. The described techniques are compared using publicly avail- able datasets, such as Ripley's synthetic dataset and the multi-class iris dataset. We will also look at medical diagnosis and credit scoring where comprehensibility is a key requirement and even a regulatory recommendation. Our experiments show that the SVM rule extraction techniques lose only a small percentage in performance compared to SVMs and therefore rank at the top of comprehensible classification techniques.Credit; Credit scoring; Models; Model; Applications; Performance; Space; Decision; Yield; Real life; Risk; Evaluation; Rules; Neural networks; Networks; Classification; Research;