Alayzing The Effects Of Modularity On Search Spaces

We are continuously challenged by ever increasing problem complexity and the need to develop algorithms that can solve complex problems and solve them within a reasonable amount of time. Modularity is thought to reduce problem complexity by decomposing large problems into smaller and less complex subproblems. In practice, introducing modularity into evolutionary algorithm representations appears to improve search performance; however, how and why modularity improves performance is not well understood. In this thesis, we seek to better understand the effects of modularity on search. In particular, what are the effects of module creation on the search space structure and how do these structural changes affect performance? We define a theoretical and empirical framework to study modularity in evolutionary algorithms. Using this framework, we provide evidence of the following. First, not all types of modularity have an effect on search. We can have highly modular spaces that in essence are equivalent to simpler non-modular spaces. This is the case, because these spaces achieve higher degree of modularity without changing the fundamental structure of the search space. Second, for the cases when modularity actually has an effect on the fundamental structure of the search space, if left without guidance, it would only crowd and complicate the space structure resulting in a harder space for most search algorithms. Finally, we have the case when modularity not only has an effect in the search space structure, but most importantly, module creation can be guided by problem domain knowledge. When this knowledge can be used to estimate the value of a module in terms of its contribution toward building the solution, then modularity is extremely effective. It is in this last case that creating high value modules or low value modules has a direct and decisive impact on performance. The results presented in this thesis help to better understand, in a principled way, the effects of modularity on search. Better understanding the effects of modularity on search is a step forward in the larger issue of evolutionary search applied to increasingly complex problems

Effective Task Transfer Through Indirect Encoding

An important goal for machine learning is to transfer knowledge between tasks. For example, learning to play RoboCup Keepaway should contribute to learning the full game of RoboCup soccer. Often approaches to task transfer focus on transforming the original representation to fit the new task. Such representational transformations are necessary because the target task often requires new state information that was not included in the original representation. In RoboCup Keepaway, changing from the 3 vs. 2 variant of the task to 4 vs. 3 adds state information for each of the new players. In contrast, this dissertation explores the idea that transfer is most effective if the representation is designed to be the same even across different tasks. To this end, (1) the bird’s eye view (BEV) representation is introduced, which can represent different tasks on the same two-dimensional map. Because the BEV represents state information associated with positions instead of objects, it can be scaled to more objects without manipulation. In this way, both the 3 vs. 2 and 4 vs. 3 Keepaway tasks can be represented on the same BEV, which is (2) demonstrated in this dissertation. Yet a challenge for such representation is that a raw two-dimensional map is highdimensional and unstructured. This dissertation demonstrates how this problem is addressed naturally by the Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) approach. HyperNEAT evolves an indirect encoding, which compresses the representation by exploiting its geometry. The dissertation then explores further exploiting the power of such encoding, beginning by (3) enhancing the configuration of the BEV with a focus on iii modularity. The need for further nonlinearity is then (4) investigated through the addition of hidden nodes. Furthermore, (5) the size of the BEV can be manipulated because it is indirectly encoded. Thus the resolution of the BEV, which is dictated by its size, is increased in precision and culminates in a HyperNEAT extension that is expressed at effectively infinite resolution. Additionally, scaling to higher resolutions through gradually increasing the size of the BEV is explored. Finally, (6) the ambitious problem of scaling from the Keepaway task to the Half-field Offense task is investigated with the BEV. Overall, this dissertation demonstrates that advanced representations in conjunction with indirect encoding can contribute to scaling learning techniques to more challenging tasks, such as the Half-field Offense RoboCup soccer domain

Evolutionary Algorithms Based on Effective Search Space Reduction for Financial Optimization Problems

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 8. 문병로.This thesis presents evolutionary algorithms incorporated with effective search space reduction for financial optimization problems. Typical evolutionary algorithms try to find optimal solutions in the original, or unrestricted search space. However, they can be unsuccessful if the optimal solutions are too complex to be discovered from scratch. This can be relieved by restricting the forms of meaningful solutions or providing the initial population with some promising solutions. To this end, we propose three evolution approaches including modular, grammatical, and seeded evolutions for financial optimization problems. We also adopt local optimizations for fine-tuning the solutions, resulting in hybrid evolutionary algorithms. First, the thesis proposes a modular evolution. In the modular evolution, the possible forms of solutions are statically restricted to certain combinations of module solutions, which reflect more domain knowledge. To preserve the module solutions, we devise modular genetic operators which work on modular search space. The modular genetic operators and statically defined modules help genetic programming focus on highly promising search space. Second, the thesis introduces a grammatical evolution. We restrict the possible forms of solutions in genetic programming by a context-free grammar. In the grammatical evolution, genetic programming works on more extended search space than modular one. Grammatically typed genetic operators are introduced for the grammatical evolution. Compared with the modular evolution, grammatical evolution requires less domain knowledge. Finally, the thesis presents a seeded evolution. Our seeded evolution provides the initial population with partially optimized solutions. The set of genes for the partial optimization is selected in terms of encoding complexity. The partially optimized solutions help genetic algorithm find more promising solutions efficiently. Since they are not too excessively optimized, genetic algorithm is still able to search better solutions. Extensive empirical results are provided using three real-world financial optimization problems: attractive technical pattern discovery, extended attractive technical pattern discovery, and large-scale stock selection. They show that our search space reductions are fairly effective for the problems. By combining the search space reductions with systematic evolutionary algorithm frameworks, we show that evolutionary algorithms can be exploited for realistic profitable trading.1. Introduction 1 1.1 Search Methods 3 1.2 Search Space Reduction 4 1.3 Main Contributions 5 1.4 Organization 7 2. Preliminaries 8 2.1 Evolutionary Algorithms 8 2.1.1 Genetic Algorithm 10 2.1.2 Genetic Programing 11 2.2 Evolutionary Algorithms in Finance 12 2.3 Search Space Reduction 12 2.3.1 Modular Evolution 12 2.3.2 Grammatical Evolution 13 2.3.3 Seeded Evolution 14 2.3.4 Summary 14 2.4 Terminology 15 2.4.1 Technical Pattern and Technical Trading Rule 15 2.4.2 Forecasting Model and Trading Model 16 2.4.3 Portfolio and Rebalancing 17 2.4.4 Data Snooping Bias 17 2.5 Financial Optimization Problems 19 2.5.1 Attractive Technical Pattern Discovery and Its Extension 19 2.5.2 Stock Selection 20 2.6 Issues 21 2.6.1 General Assumptions 21 2.6.2 Performance Measure 22 3. Modular Evolution 23 3.1 Modular Genetic Programming 24 3.2 Hybrid Genetic Programming 28 3.3 Attractive Technical Pattern Discovery 29 3.3.1 Introduction 29 3.3.2 Problem Formulation 31 3.3.3 Modular Search Space 33 3.3.4 Experimental Results 35 3.3.5 Summary 41 4. Grammatical Evolution 44 4.1 Grammatical Type System 45 4.2 Hybrid Genetic Programming 47 4.3 Extended Attractive Technical Pattern Discovery 51 4.3.1 Introduction 51 4.3.2 Problem Formulation 54 4.3.3 Experimental Results 56 4.3.4 Summary 73 5. Seeded Evolution 76 5.1 Heuristic Seeding 77 5.2 Hybrid Genetic Algorithm 78 5.3 Large-Scale Stock Selection 81 5.3.1 Introduction 81 5.3.2 Problem Formulation 83 5.3.3 Ranking with Partitions 85 5.3.4 Experimental Results 87 5.3.5 Summary 96 6. Conclusions 104Docto

Emergence through conflict : the Multi-Disciplinary Design System (MDDS)

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Architecture, 2009.Includes bibliographical references (p. 413-430).This dissertation proposes a framework and a group of systematic methodologies to construct a computational Multi-Disciplinary Design System (MDDS) that can support the design of complex systems within a variety of domains. The way in which the resulting design system is constructed, and the capabilities it brings to bare, are totally different from the methods used in traditional sequential design. The MDDS embraces diverse areas of research that include design science, systems theory, artificial intelligence, design synthesis and generative algorithms, mathematical modeling and disciplinary analyses, optimization theory, data management and model integration, and experimental design among many others. There are five phases to generate the MDDS. These phases involve decomposition, formulation, modeling, integration, and exploration. These phases are not carried out in a sequential manner, but rather in a continuous move back and forth between the different phases. The process of building the MDDS begins with a top-down decomposition of a design concept. The design, seen as an object, is decomposed into its components and aspects, while the design, seen as a process, is decomposed into developmental levels and design activities. Then based on the process decomposition, the architecture of the MDDS is formulated into hierarchical levels each of which comprises a group of design cycles that include design modules at different degrees of abstraction. Based on the design object decomposition, the design activities which include synthesis, analysis, evaluation and optimization are modeled within the design modules.(cont.) Subsequently through a bottom-up approach, the design modules are integrated into a data flow network. This network forms MDDS as an integrated system that acts as a holistic structured functional unit that explores the design space in search of satisfactory solutions. The MDDS emergent properties are not detectable through the properties and behaviors of its parts, and can only be enucleated through a holistic approach. The MDDS is an adaptable system that is continuously dependent on, and responsive to, the uncertainties of the design process. The evolving MDDS is thus characterized a multi-level, multi-module, multi-variable and multi-resolution system. Although the MDDS framework is intended to be domain-independent, several MDDS prototypes were developed within this dissertation to generate exploratory building designs.by Anas Alfaris.Ph.D

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways

A complex systems approach to education in Switzerland

The insights gained from the study of complex systems in biological, social, and engineered systems enables us not only to observe and understand, but also to actively design systems which will be capable of successfully coping with complex and dynamically changing situations. The methods and mindset required for this approach have been applied to educational systems with their diverse levels of scale and complexity. Based on the general case made by Yaneer Bar-Yam, this paper applies the complex systems approach to the educational system in Switzerland. It confirms that the complex systems approach is valid. Indeed, many recommendations made for the general case have already been implemented in the Swiss education system. To address existing problems and difficulties, further steps are recommended. This paper contributes to the further establishment complex systems approach by shedding light on an area which concerns us all, which is a frequent topic of discussion and dispute among politicians and the public, where billions of dollars have been spent without achieving the desired results, and where it is difficult to directly derive consequences from actions taken. The analysis of the education system's different levels, their complexity and scale will clarify how such a dynamic system should be approached, and how it can be guided towards the desired performance

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways

Energy: A continuing bibliography with indexes

This bibliography lists 1920 reports, articles, and other documents introduced into the NASA Scientific and Technical Information System from July 1, 1980 through September 30, 1980