Evolving Graphs by Graph Programming

Graphs are a ubiquitous data structure in computer science and can be used to represent solutions to difficult problems in many distinct domains. This motivates the use of Evolutionary Algorithms to search over graphs and efficiently find approximate solutions. However, existing techniques often represent and manipulate graphs in an ad-hoc manner. In contrast, rule-based graph programming offers a formal mechanism for describing relations over graphs. This thesis proposes the use of rule-based graph programming for representing and implementing genetic operators over graphs. We present the Evolutionary Algorithm Evolving Graphs by Graph Programming and a number of its extensions which are capable of learning stateful and stateless digital circuits, symbolic expressions and Artificial Neural Networks. We demonstrate that rule-based graph programming may be used to implement new and effective constraint-respecting mutation operators and show that these operators may strictly generalise others found in the literature. Through our proposal of Semantic Neutral Drift, we accelerate the search process by building plateaus into the fitness landscape using domain knowledge of equivalence. We also present Horizontal Gene Transfer, a mechanism whereby graphs may be passively recombined without disrupting their fitness. Through rigorous evaluation and analysis of over 20,000 independent executions of Evolutionary Algorithms, we establish numerous benefits of our approach. We find that on many problems, Evolving Graphs by Graph Programming and its variants may significantly outperform other approaches from the literature. Additionally, our empirical results provide further evidence that neutral drift aids the efficiency of evolutionary search

Context dependent fuzzy modelling and its applications

Fuzzy rule-based systems (FRBS) use the principle of fuzzy sets and fuzzy logic to describe vague and imprecise statements and provide a facility to express the behaviours of the system with a human-understandable language. Fuzzy information, once defined by a fuzzy system, is fixed regardless of the circumstances and therefore makes it very difficult to capture the effect of context on the meaning of the fuzzy terms. While efforts have been made to integrate contextual information into the representation of fuzzy sets, it remains the case that often the context model is very restrictive and/or problem specific. The work reported in this thesis is our attempt to create a practical frame work to integrate contextual information into the representation of fuzzy sets so as to improve the interpretability as well as the accuracy of the fuzzy system. Throughout this thesis, we have looked at the capability of the proposed context dependent fuzzy sets as a stand alone as well as in combination with other methods in various application scenarios ranging from time series forecasting to complicated car racing control systems. In all of the applications, the highly competitive performance nature of our approach has proven its effectiveness and efficiency compared with existing techniques in the literature

Context dependent fuzzy modelling and its applications

Fuzzy rule-based systems (FRBS) use the principle of fuzzy sets and fuzzy logic to describe vague and imprecise statements and provide a facility to express the behaviours of the system with a human-understandable language. Fuzzy information, once defined by a fuzzy system, is fixed regardless of the circumstances and therefore makes it very difficult to capture the effect of context on the meaning of the fuzzy terms. While efforts have been made to integrate contextual information into the representation of fuzzy sets, it remains the case that often the context model is very restrictive and/or problem specific. The work reported in this thesis is our attempt to create a practical frame work to integrate contextual information into the representation of fuzzy sets so as to improve the interpretability as well as the accuracy of the fuzzy system. Throughout this thesis, we have looked at the capability of the proposed context dependent fuzzy sets as a stand alone as well as in combination with other methods in various application scenarios ranging from time series forecasting to complicated car racing control systems. In all of the applications, the highly competitive performance nature of our approach has proven its effectiveness and efficiency compared with existing techniques in the literature

Crossover control in selection hyper-heuristics: case studies using MKP and HyFlex

Hyper-heuristics are a class of high-level search methodologies which operate over a search space of heuristics rather than a search space of solutions. Hyper-heuristic research has set out to develop methods which are more general than traditional search and optimisation techniques. In recent years, focus has shifted considerably towards cross-domain heuristic search. The intention is to develop methods which are able to deliver an acceptable level of performance over a variety of different problem domains, given a set of low-level heuristics to work with. This thesis presents a body of work investigating the use of selection hyper-heuristics in a number of different problem domains. Specifically the use of crossover operators, prevalent in many evolutionary algorithms, is explored within the context of single-point search hyper-heuristics. A number of traditional selection hyper-heuristics are applied to instances of a well-known NP-hard combinatorial optimisation problem, the multidimensional knapsack problem. This domain is chosen as a benchmark for the variety of existing problem instances and solution methods available. The results suggest that selection hyper-heuristics are a viable method to solve some instances of this problem domain. Following this, a framework is defined to describe the conceptual level at which crossover low-level heuristics are managed in single-point selection hyper-heuristics. HyFlex is an existing software framework which supports the design of heuristic search methods over multiple problem domains, i.e. cross-domain optimisation. A traditional heuristic selection mechanism is modified in order to improve results in the context of cross-domain optimisation. Finally the effect of crossover use in cross-domain optimisation is explored

Crossover control in selection hyper-heuristics: case studies using MKP and HyFlex

Hyper-heuristics are a class of high-level search methodologies which operate over a search space of heuristics rather than a search space of solutions. Hyper-heuristic research has set out to develop methods which are more general than traditional search and optimisation techniques. In recent years, focus has shifted considerably towards cross-domain heuristic search. The intention is to develop methods which are able to deliver an acceptable level of performance over a variety of different problem domains, given a set of low-level heuristics to work with. This thesis presents a body of work investigating the use of selection hyper-heuristics in a number of different problem domains. Specifically the use of crossover operators, prevalent in many evolutionary algorithms, is explored within the context of single-point search hyper-heuristics. A number of traditional selection hyper-heuristics are applied to instances of a well-known NP-hard combinatorial optimisation problem, the multidimensional knapsack problem. This domain is chosen as a benchmark for the variety of existing problem instances and solution methods available. The results suggest that selection hyper-heuristics are a viable method to solve some instances of this problem domain. Following this, a framework is defined to describe the conceptual level at which crossover low-level heuristics are managed in single-point selection hyper-heuristics. HyFlex is an existing software framework which supports the design of heuristic search methods over multiple problem domains, i.e. cross-domain optimisation. A traditional heuristic selection mechanism is modified in order to improve results in the context of cross-domain optimisation. Finally the effect of crossover use in cross-domain optimisation is explored

Event and state detection in time series by genetic programming

Event and state detection in time series has significant value in scientific areas and real-world applications. The aim of detecting time series event and state patterns is to identify particular variations of user-interest in one or more channels of time series streams. For example, dangerous driving behaviours such as sudden braking and harsh acceleration can be detected from continuous recordings from inertial sensors. However, the existing methods are highly dependent on domain knowledge such as the size of the time series pattern and a set of effective features. Furthermore, they are not directly suitable for multi-channel time series data. In this study, we establish a genetic programming based method which can perform classification on multi-channel time series data. It does not need the domain knowledge required by the existing methods. The investigation consists of four parts: the methodology, an evaluation on event detection tasks, an evaluation on state detection tasks and an analysis on the suitability for real-world applications. In the methodology, a GP based method is proposed for processing and analysing multi-channel time series streams. The function set includes basic mathematical operations. In addition, specific functions and terminals are introduced to reserve historical information, capture temporal dependency across time points and handle dependency between channels. These functions and terminals help the GP based method to automatically find the pattern size and extract features. This study also investigates two different fitness functions - accuracy and area under the curve. The proposed method is investigated on a range of event detection tasks. The investigation starts from synthetic tasks such as detecting complete sine waves. The performance of the GP based method is compared to traditional classification methods. On the raw data the GP based method achieves 100 percent accuracy, which outperforms all the non-GP methods.The performance of the non-GP methods is comparable to the GP based method only with suitable features. In addition, the GP based method is investigated on two complex real-world event detection tasks - dangerous driving behaviour detection and video shot detection. In the task of detecting three dangerous driving behaviours from 21-channel time series data, the GP based method performs consistently better than the non-GP classifiers even when features are provided. In the video shot detection task, the GP based method achieves comparable performance on 11200-channel time series to the non-GP classifiers on 28 features. The GP based method is more accurate than a commercial product. The GP based method has also been investigated on state detection tasks. This involves synthetic tasks such as detecting concurrent high values in four of five channels and a real-world activity recognition problem. The results also show that the GP based method consistently outperforms the non-GP methods even with the presence of manually constructed features. As part of the investigation, a mobile phone based activity recognition data set was collected as there was no existing publicly available data set. The suitability of the GP based method for solving real-world problems is further analysed. Our analysis shows that the GP based method can be successfully extended for multi-class classification. The analysis of the evolved programs demonstrates that they do capture time series patterns. On synthetic data sets, the injected regularities are revealed in understandable individuals. The best programs for three real-world problems are more difficult to explain but still provide some insight. The selection of relevant channels and data points by the programs are consistent with domain knowledge. In addition, the analysis shows that the proposed method still performs well for time series pattern of different sizes. The effective window sizes of the evolved GP programs are close to the pattern size. Finally, our study on execution performance of the evolved programs shows that these programs are fast in execution and are suitable for real-time applications. In summary, the GP based method is suitable for the kinds of real-world applications studied in this thesis. This thesis concludes that, with a suitable representation, genetic programming can be an effective method for event and state detection in multi-channel time series for a range of synthetic and real-world tasks. This method does not require much domain knowledge such as the pattern size and suitable features. It offers an effective classification method in similar tasks that are studied in this thesis

Parameter Tuning and Scientific Testing in Evolutionary Algorithms

Eiben, A.E. [Promotor

Implicit Incremental Model Analyses and Transformations

When models of a system change, analyses based on them have to be reevaluated in order for the results to stay meaningful. In many cases, the time to get updated analysis results is critical. This thesis proposes multiple, combinable approaches and a new formalism based on category theory for implicitly incremental model analyses and transformations. The advantages of the implementation are validated using seven case studies, partially drawn from the Transformation Tool Contest (TTC)

The Complete Reference (Volume 4)

This is the fourth volume of the successful series Robot Operating Systems: The Complete Reference, providing a comprehensive overview of robot operating systems (ROS), which is currently the main development framework for robotics applications, as well as the latest trends and contributed systems. The book is divided into four parts: Part 1 features two papers on navigation, discussing SLAM and path planning. Part 2 focuses on the integration of ROS into quadcopters and their control. Part 3 then discusses two emerging applications for robotics: cloud robotics, and video stabilization. Part 4 presents tools developed for ROS; the first is a practical alternative to the roslaunch system, and the second is related to penetration testing. This book is a valuable resource for ROS users and wanting to learn more about ROS capabilities and features.info:eu-repo/semantics/publishedVersio

Multi-Objective Reinforcement Learning

The recent surge in artificial intelligence (AI) agents assisting us in daily tasks suggests that these agents possess the ability to comprehend key aspects of our environment, thereby facilitating better decision-making. Presently, this understanding is predominantly acquired through data-driven learning methods. Notably, reinforcement learning (RL) stands out as a natural framework for agents to acquire behaviors by interacting with their environment and learning from feedback. However, despite the effectiveness of RL in training agents to optimize a single objective, such as minimizing cost or maximizing performance, it overlooks the inherent complexity of decision-making in real-world scenarios where multiple objectives may need to be considered simultaneously. Indeed, an essential aspect that remains understudied is the human tendency to make compromises in various situations, influenced by values, circumstances, or mood. This limitation underscores the need for advancements in AI methodologies to address the nuanced trade-offs inherent to human decision-making. Thus, this work aims to explore the extension of RL principles into multi-objective settings, where agents can learn behaviors that balance competing objectives, thereby enabling more adaptable and personalized AI systems. In the first part of this thesis, we explore the domain of multi-objective reinforcement learning (MORL), a recent technique aimed at enabling AI agents to acquire diverse behaviors associated with different trade-offs from multiple feedback signals. While MORL is relatively recent, works in this field often rely on existing knowledge coming from older fields such as multi-objective optimization (MOO) and RL. Our initial contribution involves a comprehensive analysis of the relationships between RL, MOO, and MORL. This examination culminates in the development of a taxonomy for categorizing MORL algorithms, drawing on concepts derived from preceding fields. Building upon this foundational understanding, we proceed to investigate the feasibility of leveraging techniques from MOO and RL to enhance MORL methodologies. This exploration yields several contributions. Among these, we introduce the utilization of metaheuristics to address the exploration-exploitation dilemma in MORL. Additionally, we introduce a versatile framework rooted in the derived taxonomy, facilitating the creation of novel MORL algorithms based on techniques coming from MOO and RL. Furthermore, our efforts extend towards improving the scientific rigor and practical applicability of MORL in real-world scenarios. To this end, we introduce methods and a suite of open-source tools that have become the standard in MORL. Many real-world situations also involve collaboration among multiple agents to accomplish tasks efficiently. Therefore, the second part of this thesis transitions to settings involving multiple agents, leading to the nascent field of multi-objective multi-agent reinforcement learning (MOMARL). In this domain, as an initial contribution, we release a comprehensive set of open-source utilities aimed to accelerate and establish a robust foundation for research within this evolving domain. Furthermore, we perform an initial study exploring the transferability of knowledge and methodologies from both MORL and multi-agent RL to the MOMARL settings. Finally, we validate our approach in a real-world application. Specifically, we aim to automatically learn the coordination of multiple drones having different objectives, harnessing the MOMARL framework to orchestrate their actions effectively. This empirical validation serves as evidence of the viability and versatility of the proposed methodologies in addressing complex real-world challenges