616 research outputs found
Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges
A variety of methods have been applied to the architectural configuration and
learning or training of artificial deep neural networks (DNN). These methods
play a crucial role in the success or failure of the DNN for most problems and
applications. Evolutionary Algorithms (EAs) are gaining momentum as a
computationally feasible method for the automated optimisation and training of
DNNs. Neuroevolution is a term which describes these processes of automated
configuration and training of DNNs using EAs. While many works exist in the
literature, no comprehensive surveys currently exist focusing exclusively on
the strengths and limitations of using neuroevolution approaches in DNNs.
Prolonged absence of such surveys can lead to a disjointed and fragmented field
preventing DNNs researchers potentially adopting neuroevolutionary methods in
their own research, resulting in lost opportunities for improving performance
and wider application within real-world deep learning problems. This paper
presents a comprehensive survey, discussion and evaluation of the
state-of-the-art works on using EAs for architectural configuration and
training of DNNs. Based on this survey, the paper highlights the most pertinent
current issues and challenges in neuroevolution and identifies multiple
promising future research directions.Comment: 20 pages (double column), 2 figures, 3 tables, 157 reference
Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization
Genetic algorithms constitute a family of black-box optimization algorithms,
which take inspiration from the principles of biological evolution. While they
provide a general-purpose tool for optimization, their particular
instantiations can be heuristic and motivated by loose biological intuition. In
this work we explore a fundamentally different approach: Given a sufficiently
flexible parametrization of the genetic operators, we discover entirely new
genetic algorithms in a data-driven fashion. More specifically, we parametrize
selection and mutation rate adaptation as cross- and self-attention modules and
use Meta-Black-Box-Optimization to evolve their parameters on a set of diverse
optimization tasks. The resulting Learned Genetic Algorithm outperforms
state-of-the-art adaptive baseline genetic algorithms and generalizes far
beyond its meta-training settings. The learned algorithm can be applied to
previously unseen optimization problems, search dimensions & evaluation
budgets. We conduct extensive analysis of the discovered operators and provide
ablation experiments, which highlight the benefits of flexible module
parametrization and the ability to transfer (`plug-in') the learned operators
to conventional genetic algorithms.Comment: 14 pages, 31 figure
An adaptive and modular framework for evolving deep neural networks
Santos, F. J. J. B., Gonçalves, I., & Castelli, M. (2023). Neuroevolution with box mutation: An adaptive and modular framework for evolving deep neural networks. Applied Soft Computing, 147(November), 1-15. [110767]. https://doi.org/10.1016/j.asoc.2023.110767 --- Funding: This work is funded by national funds through the FCT - Foundation for Science and Technology, I.P., within the scope of the projects CISUC - UID/CEC/00326/2020, UIDB/04152/2020 - Centro de Investigação em Gestão de Informação (MagIC)/NOVA IMS, and by European Social Fund, through the Regional Operational Program Centro 2020 .The pursuit of self-evolving neural networks has driven the emerging field of Evolutionary Deep Learning, which combines the strengths of Deep Learning and Evolutionary Computation. This work presents a novel method for evolving deep neural networks by adapting the principles of Geometric Semantic Genetic Programming, a subfield of Genetic Programming, and Semantic Learning Machine. Our approach integrates evolution seamlessly through natural selection with the optimization power of backpropagation in deep learning, enabling the incremental growth of neural networks’ neurons across generations. By evolving neural networks that achieve nearly 89% accuracy on the CIFAR-10 dataset with relatively few parameters, our method demonstrates remarkable efficiency, evolving in GPU minutes compared to the field standard of GPU days.publishersversionpublishe
Novelty-assisted Interactive Evolution Of Control Behaviors
The field of evolutionary computation is inspired by the achievements of natural evolution, in which there is no final objective. Yet the pursuit of objectives is ubiquitous in simulated evolution because evolutionary algorithms that can consistently achieve established benchmarks are lauded as successful, thus reinforcing this paradigm. A significant problem is that such objective approaches assume that intermediate stepping stones will increasingly resemble the final objective when in fact they often do not. The consequence is that while solutions may exist, searching for such objectives may not discover them. This problem with objectives is demonstrated through an experiment in this dissertation that compares how images discovered serendipitously during interactive evolution in an online system called Picbreeder cannot be rediscovered when they become the final objective of the very same algorithm that originally evolved them. This negative result demonstrates that pursuing an objective limits evolution by selecting offspring only based on the final objective. Furthermore, even when high fitness is achieved, the experimental results suggest that the resulting solutions are typically brittle, piecewise representations that only perform well by exploiting idiosyncratic features in the target. In response to this problem, the dissertation next highlights the importance of leveraging human insight during search as an alternative to articulating explicit objectives. In particular, a new approach called novelty-assisted interactive evolutionary computation (NA-IEC) combines human intuition with a method called novelty search for the first time to facilitate the serendipitous discovery of agent behaviors. iii In this approach, the human user directs evolution by selecting what is interesting from the on-screen population of behaviors. However, unlike in typical IEC, the user can then request that the next generation be filled with novel descendants, as opposed to only the direct descendants of typical IEC. The result of such an approach, unconstrained by a priori objectives, is that it traverses key stepping stones that ultimately accumulate meaningful domain knowledge. To establishes this new evolutionary approach based on the serendipitous discovery of key stepping stones during evolution, this dissertation consists of four key contributions: (1) The first contribution establishes the deleterious effects of a priori objectives on evolution. The second (2) introduces the NA-IEC approach as an alternative to traditional objective-based approaches. The third (3) is a proof-of-concept that demonstrates how combining human insight with novelty search finds solutions significantly faster and at lower genomic complexities than fully-automated processes, including pure novelty search, suggesting an important role for human users in the search for solutions. Finally, (4) the NA-IEC approach is applied in a challenge domain wherein leveraging human intuition and domain knowledge accelerates the evolution of solutions for the nontrivial octopus-arm control task. The culmination of these contributions demonstrates the importance of incorporating human insights into simulated evolution as a means to discovering better solutions more rapidly than traditional approaches
NASCTY: Neuroevolution to Attack Side-channel Leakages Yielding Convolutional Neural Networks
Side-channel analysis (SCA) can obtain information related to the secret key
by exploiting leakages produced by the device. Researchers recently found that
neural networks (NNs) can execute a powerful profiling SCA, even on targets
protected with countermeasures. This paper explores the effectiveness of
Neuroevolution to Attack Side-channel Traces Yielding Convolutional Neural
Networks (NASCTY-CNNs), a novel genetic algorithm approach that applies genetic
operators on architectures' hyperparameters to produce CNNs for side-channel
analysis automatically. The results indicate that we can achieve performance
close to state-of-the-art approaches on desynchronized leakages with mask
protection, demonstrating that similar neuroevolution methods provide a solid
venue for further research. Finally, the commonalities among the constructed
NNs provide information on how NASCTY builds effective architectures and deals
with the applied countermeasures.Comment: 19 pages, 6 figures, 4 table
Evolutionary Reinforcement Learning: A Survey
Reinforcement learning (RL) is a machine learning approach that trains agents
to maximize cumulative rewards through interactions with environments. The
integration of RL with deep learning has recently resulted in impressive
achievements in a wide range of challenging tasks, including board games,
arcade games, and robot control. Despite these successes, there remain several
crucial challenges, including brittle convergence properties caused by
sensitive hyperparameters, difficulties in temporal credit assignment with long
time horizons and sparse rewards, a lack of diverse exploration, especially in
continuous search space scenarios, difficulties in credit assignment in
multi-agent reinforcement learning, and conflicting objectives for rewards.
Evolutionary computation (EC), which maintains a population of learning agents,
has demonstrated promising performance in addressing these limitations. This
article presents a comprehensive survey of state-of-the-art methods for
integrating EC into RL, referred to as evolutionary reinforcement learning
(EvoRL). We categorize EvoRL methods according to key research fields in RL,
including hyperparameter optimization, policy search, exploration, reward
shaping, meta-RL, and multi-objective RL. We then discuss future research
directions in terms of efficient methods, benchmarks, and scalable platforms.
This survey serves as a resource for researchers and practitioners interested
in the field of EvoRL, highlighting the important challenges and opportunities
for future research. With the help of this survey, researchers and
practitioners can develop more efficient methods and tailored benchmarks for
EvoRL, further advancing this promising cross-disciplinary research field
- …