Explaining Adaptation in Genetic Algorithms With Uniform Crossover: The Hyperclimbing Hypothesis

The hyperclimbing hypothesis is a hypothetical explanation for adaptation in genetic algorithms with uniform crossover (UGAs). Hyperclimbing is an intuitive, general-purpose, non-local search heuristic applicable to discrete product spaces with rugged or stochastic cost functions. The strength of this heuristic lie in its insusceptibility to local optima when the cost function is deterministic, and its tolerance for noise when the cost function is stochastic. Hyperclimbing works by decimating a search space, i.e. by iteratively fixing the values of small numbers of variables. The hyperclimbing hypothesis holds that UGAs work by implementing efficient hyperclimbing. Proof of concept for this hypothesis comes from the use of a novel analytic technique involving the exploitation of algorithmic symmetry. We have also obtained experimental results that show that a simple tweak inspired by the hyperclimbing hypothesis dramatically improves the performance of a UGA on large, random instances of MAX-3SAT and the Sherrington Kirkpatrick Spin Glasses problem.Comment: 22 pages, 5 figure

Algoritmo transgénico aplicado al Job Shop Rescheduling Problem

Context: Job sequencing has been approached from a static perspective, without considering the occurrence of unexpected events that might require modifying the schedule, thereby affecting its performance measures. Method: This paper presents the development and application of a genetic algorithm to the Job Shop Rescheduling Problem (JSRP), a reprogramming of the traditional Job Shop Scheduling Problem. This novel approach seeks to repair the schedule in such a way that theoretical models accurately represent real manufacturing environments. Results: The experiments designed to validate the algorithm aim to apply five classes of disruptions that could impact the schedule, evaluating two performance measures. This experiment was concurrently conducted with a genetic algorithm from the literature in order to facilitate the comparison of results. It was observed that the proposed approach outperforms the genetic algorithm 65% of the time, and it provides better stability measures 98% of the time. Conclusions: The proposed algorithm showed favorable outcomes when tested with well-known benchmark instances of the Job Shop Scheduling Problem, and the possibility of enhancing the tool's performance through simulation studies remains open.Contexto: La secuenciación de trabajos ha sido abordada desde un enfoque estático, sin considerar la aparición de eventos inesperados que requieran modificar el cronograma, lo que incide en sus medidas de desempeño. Método: Este artículo expone el desarrollo y aplicación de un algoritmo transgénico al Job Shop Rescheduling Problem (JSRP), una reprogramación del tradicional Job Shop Scheduling Problem. Este enfoque novedoso busca reparar el cronograma de modo que los modelos teóricos representen los entornos de manufactura reales. Resultados: Los experimentos diseñados para validar el algoritmo pretenden aplicar cinco clases de interrupciones que pueden afectar el cronograma, evaluando dos medidas de desempeño. Este experimento se realizó simultáneamente en un algoritmo genético de la literatura para facilitar la comparación de los resultados. Se observó que el enfoque propuesto tiene un desempeño superior al del algoritmo genético el 65 % de las veces y lo supera en la medida de estabilidad el 98 % de las veces. Conclusiones: El algoritmo propuesto mostró buenos resultados al ser probado con instancias de comparación reconocidas del Job Shop Scheduling Problem (JSSP), y queda abierta la posibilidad de mejorar el desempeño de la herramienta por medio de estudios de simulación

Automatic synthesis of sorting algorithms by gene expression programming + (geometric) semantic gene expression programming + encouraging phenotype variation with a new semantic operator: semantic conditional crossover

Gene Expression Programming (GEP) is an alternative to Genetic Programming (GP). Given its characteristics compared to GP, we question if GEP should be the standard choice for evolutionary program synthesis, both as base for research and practical application. We raise the question if such a shift could increase the rate of investigation, applicability and the quality of results obtained from evolutionary techniques for code optimization. We present three distinct and unprecedented studies using GEP in an attempt to develop understanding, investigate the potential and forward the branch. Each study has an individual contribution on its own involving GEP. As a whole, the three studies try to investigate di erent aspects that might be critical to answer the questions raised in the previous paragraph. In the rst individual contribution, we investigate GEP's applicability to automatically synthesize sorting algorithms. Performance is compared against GP under similar experimental conditions. GEP is shown to be capable of producing sorting algorithms and outperforms GP in doing so. As a second experiment, we enhanced GEP's evolutionary process with semantic awareness of candidate programs, originating Semantic Gene Expression Programming (SGEP), similarly to how Semantic Genetic Programming (SGP) builds over GP. Geometric semantic concepts are then introduced to SGEP, forming Geometric Semantic Gene Expression Programming (GSGEP). A comparative experiment between GP, GEP, SGP and SGEP is performed using di erent problems and setup combinations. Results were mixed when comparing SGEP and SGP, suggesting performance is signi cantly related to the problem addressed. By out-performing the alternatives in many of the benchmarks, SGEP demonstrates practical potential. The results are analyzed in di erent perspectives, also providing insight on the potential of di erent crossover variations when applied along GP/GEP. GEP' compatibility with innovation developed to work with GP is demonstrated possible without extensive adaptation. Considerations for integration of SGEP are discussed. In the last contribution, a new semantic operator is proposed, SCC, which applies crossover conditionally only when elements are semantically di erent enough, performing mutation otherwise. The strategy attempts to encourage semantic diversity and wider the portion of the semantic-solution space searched. A practical experiment was performed alternating the integration of SCC in the evolutionary process. When using the operator, the quality of obtained solutions alternated between slight improvements and declines. The results don't show a relevant indication of possible advantage from its employment and don't con rm what was expected in the theory. We discuss ways in which further work might investigate this concept and assess if it has practical potential under di erent circumstances. On the other hand, in regards to the basilar questions of this investigation, the process of development and testing of SCC is performed completely on a GEP/SGEP base, suggesting how the latest can be used as the base for future research on evolutionary program synthesis.Programa c~ao Gen etica por Express~oes (GEP) e uma alternativa recente a Programa c~ao Gen etica (GP). Neste estudo observamos o GEP e colocamos a quest~ao se este n~ao deveria ser tratado como primeira escolha quando se trata de sintetiza c~ao autom atica de programas atrav es de m etodos evolutivos. Dadas as caracteristicas do GEP perguntamonos se esta mudan ca de perspectiva poderia aumentar a investiga c~ao, aplicabilidade e qualidade dos resultados obtidos para a optimiza c~ao de c odigo por m etodos evolutivos. Neste estudo apresentamos tr^es contribui c~oes in editas e distintas usando o algoritmo GEP. Cada uma das contribui c~oes apresenta um avan co ou investiga c~ao no campo da GEP. Como um todo, estas contribui c~oes tentam obter cohecimento e informa c~oes para se abordar a quest~ao geral apresentada no p aragrafo anterior. Na primeira contribui c~ao, investiga-mos e testamos o GEP no problema da sintese autom atica de algoritmos de ordena c~ao. Para o melhor do nosso conhecimento, esta e a primeira vez que este problema e abordado com o GEP. A performance e comparada a do GP em condi c~oes semelhantes, de modo a isolar as caracteristicas de cada algoritmo como factor de distin c~ao. As a second experiment, we enhanced GEP's evolutionary process with semantic awareness of candidate programs, originating Semantic Gene Expression Programming (SGEP), similarly to how Semantic Genetic Programming (SGP) builds over GP. Geometric semantic concepts are then introduced to SGEP, forming Geometric Semantic Gene Expression Programming (GSGEP). A comparative experiment between GP, GEP, SGP and SGEP is performed using di erent problems and setup combinations. Results were mixed when comparing SGEP and SGP, suggesting performance is signi cantly related to the problem addressed. By out-performing the alternatives in many of the benchmarks, SGEP demonstrates practical potential. The results are analyzed in di erent perspectives, also providing insight on the potential of di erent crossover variations when applied along GP/GEP. GEP's compatibility with innovation developed to work with GP is demonstrated possible without extensive adaptation. Considerations for integration of SGEP are discussed. Na segunda contribui c~ao, adicionamos ao processo evolutivo do GEP a capacidade de medir o valor sem^antico dos programas que constituem a popula c~ao. A esta variante damos o nome de Programa c~ao Gen etica por Express~oes Sem^antica (SGEP). Esta variante tr as para o GEP as mesmas caracteristicas que a Programa c~ao Gen etica Sem^antica(SGP) trouxe para o GP convencional. Conceitos geom etricos s~ao tamb em apresentados para o SGEP, extendendo assim a variante e criando a Programa c~ao Gen etica por Express~oes Geom etrica Sem^antica (GSGEP). De forma a testar estas novas variantes, efectuamos uma experi^encia onde s~ao comparados o GP, GEP, SGP e SGEP entre diferentes problemas e combina c~oes de operadores de cruzamento. Os resultados mostraram que n~ao houve um algoritmo que se destaca-se em todas as experi^encias, sugerindo que a performance est a signi cativamente relacionada com o problema a ser abordado. De qualquer modo, o SGEP obteve vantagem em bastantes dos benchmarks, dando assim ind cios de pot^encial ter utilidade pr atica. De um modo geral, esta contribui c~ao demonstra que e possivel utilizar tecnologia desenvolvida a pensar em GP no GEP sem grande esfor co na adapta c~ao. No m da contribui c~ao, s~ao discutidas algumas considera c~oes sobre o SGEP. Na terceira contribui c~ao propomos um novo operador, o Cruzamento Sem^antico Condicional (SCC). Este operador, baseado na dist^ancia sem^antica entre dois elementos propostos, decide se os elementos s~ao propostos para cruzamento, ou se um deles e mutato e ambos re-introduzidos na popula c~ao. Esta estrat egia tem como objectivo aumentar a diversidade gen etica na popula c~ao em fases cruciais do processo evolutivo e alargar a por c~ao do espa co sem^antico pesquisado. Para avaliar o pot^encial deste operador, realizamos uma experi^encia pr atica e comparamos processos evolutivos semelhantes onde o uso ou n~ao uso do SCC e o factor de distin c~ao. Os resultados obtidos n~ao demonstraram vantagens no uso do SCC e n~ao con rmam o esperado em teoria. No entanto s~ao discutidas maneiras em que o conceito pode ser reaproveitado para novos testes em que possa ter pot^encial para demonstrar resultados possitivos. Em rela c~ao a quest~ao central da tese, visto este estudo ter sido desenvolvido com base em GEP/SGEP e visto a teoria do SCC ser compativel com GP, e demonstrado que um estudo geral a area da sintese de algoritmos por meios evolutivos, pode ser conduzido com base no GEP

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era