상상 모델: 구성 패턴 생성 네트워크의 다양성 탐색을 통한 이미지 제작

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2019. 2. 문병로.Divergent Search methods are devised to resolve the problem falling into a trap of local optima, an arch-enemy of stochastic optimization algorithms. Novelty Search and Surprise Search, inter alia, use the concept of {\it behavior} and explore behavior space defined by it, maintaining evolutionary divergence and they have shown great performance in this respect. Moreover, coupling novelty and surprise concept was designed based on ideas that those two algorithms search behavioral space in a different way. The combination of two algorithms can be viewed as multiobjective optimization algorithm, and this approach enhanced the performance than using one divergent search method only. Since several divergent search methods have outperformed existing stochastic optimization algorithms in recent studies of robotics, it has been applied to many other domains, such as robot morphology, artificial life and generating images. Particularly, the Innovation Engines applied Novelty Search to image generating method so as to create novel and interesting images. In this paper, we propose Imagination Model that adopts Novelty-Surprise Search which is the combination of Novelty and Surprise Search instead of pure Novelty Search, as an extension of Innovation Engine. Evolutionary algorithms using Novelty Search, Surprise Search, Novelty-Surprise Search are compared via well-trained deep neural networks defining the behaviors of individuals in terms of creating interesting images. Results of experiments indicate that Novelty-Surprise Search outperforms Novelty Search and Surprise Search even in image domainit searches and explores vast behavioral space more extensively than each search algorithm on its own.다양성 검색 방법은 확률적 최적화 알고리즘의 주적인 지역 최적해의 함정에 빠지는 문제를 해결하기 위해 고안되었다. 그중에서도 참신함 탐색과 놀라움 탐색은 {\it 행동}이라는 개념과 그 개념이 정의하는 행동 공간을 탐색하며 진화적 다양성을 유지했고 이 점에 있어서 훌륭한 성능을 보여주었다. 그뿐만 아니라 두 다양성 탐색이 서로 다른 방식으로 행동 공간을 탐색하는 데에서 착안하여, 참신함과 놀라움을 결합하는 알고리즘이 설계되었다. 두 알고리즘의 조합은 다목적 최적화 알고리즘으로 간주할 수 있는데, 이 접근 방식은 둘 중 하나만의 다양성 탐색 방법을 사용할 때보다 성능이 개선됨을 다양한 연구에서 보여주었다. 이처럼 여러 다양성 탐색이 기존의 확률적 최적화 알고리즘을 뛰어 넘는 성능을 보였기 때문에, 로봇 형태학, 인공생명, 이미지 생성처럼 다양한 분야에 응용되어왔다. 특히, 혁신 엔진은 새로우면서도 흥미로운 이미지를 창조하기 위해 이미지 생성 방법에 참신함 탐색을 적용했다. 이에 더해 우리는 이 논문에서 상상 모델을 제안한다. 이 상상 모델은 혁신 엔진의 확장으로서 순수한 참신함 탐색 대신 참신함 탐색과 놀라움 탐색을 결합한 참신함-놀라움 탐색을 도입한다. 참신함 탐색, 놀라움 탐색 그리고 참신함-놀라움 탐색을 사용한 진화 연산을 이미지 생성에 관한 측면에서 비교하는 실험을 진행하며, 이들은 모두 심층 인공신경망을 통해 그들이 사용하는 행동이라는 개념이 정의된다. 실험 결과를 살펴보면, 참신함-놀라움 탐색은 단순히 참신함 탐색이나 놀라움 탐색 각각을 따로따로 사용하는 것보다 더 넓은 행동 공간을 더 광범위하게 탐색하는 모습을 보여주었다. 이로부터, 다른 분야뿐 아니라 이미지 생성 영역에서도 참신함-놀라움 탐색이 참신함 탐색과 놀라움 탐색 각각을 뛰어넘는 성능을 보인다는 것을 확인하였다.Abstract i Contents iii List of Figures v List of Tables vi Chapter 1 Introduction 1 Chapter 2 Background 4 2.1 CPPN-NEAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Novelty Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Surprise Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.4 Combining Novelty and Surprise Score . . . . . . . . . . . . . . . . . . . 7 2.5 Innovation Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Chapter 3 Methods 9 3.1 Image Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2 Behavioral Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3 Imagination Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Chapter 4 Experiments 13 4.1 Fitness Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2 Deep Neural Networks and Dataset . . . . . . . . . . . . . . . . . . . . . . 14 Chapter 5 Results 16 Chapter 6 Discussion 25 Chapter 7 Conclusion 27 Bibliography 29 요약 33Maste

Fusing novelty and surprise for evolving robot morphologies

Traditional evolutionary algorithms tend to converge to a single good solution, which can limit their chance of discovering more diverse and creative outcomes. Divergent search, on the other hand, aims to counter convergence to local optima by avoiding selection pressure towards the objective. Forms of divergent search such as novelty or surprise search have proven to be beneficial for both the efficiency and the variety of the solutions obtained in deceptive tasks. Importantly for this paper, early results in maze navigation have shown that combining novelty and surprise search yields an even more effective search strategy due to their orthogonal nature. Motivated by the largely unexplored potential of coupling novelty and surprise as a search strategy, in this paper we investigate how fusing the two can affect the evolution of soft robot morphologies. We test the capacity of the combined search strategy against objective, novelty, and surprise search, by comparing their efficiency and robustness, and the variety of robots they evolve. Our key results demonstrate that novelty-surprise search is generally more efficient and robust across eight different resolutions. Further, surprise search explores the space of robot morphologies more broadly than any other algorithm examined.peer-reviewe

Divergent Criticality – A Mechanism of Neural Function for Perception and Learning

The natural world presents opportunities to all organisms as they compete for the biological-value afforded to them through their ecological engagement. This presents two fundamental requirements for perceiving such opportunities: to be able to recognise value and learning how to access new value. Though many theoretical accounts of how we might achieve such selectionist ends have been explored – how ‘perception’ and ‘learning’ resonate with life’s challenges and opportunities, to date, no explanation has yet been able to naturalise such perception adequately in the Universal laws that govern our existence – not only for explaining the human experience of the world, but in exploring the true nature of our perception. This thesis explores our perceptions of engaging with the world and seeks to explain how the demands of our experiences resonate with the efficient functioning of our brain. It proposes, that in a world of challenge and opportunity, rather than the efficient functioning of our neural resources, it is, instead, the optimising of ‘learning’ that is selected for, as an evolutionary priority. Building on existing literature in the fields of Phenomenology, Free Energy and Neuroscience, this thesis considers perception and learning as synonymous with the cognitive constructs of an ‘attention’ tuned for learning optimisation, and explores the processes of learning in neural function. It addresses the philosophical issues of how an individual’s perception of subjective experiences, might provide some empirical objectivity in proposing a ‘Tolerance’ hypothesis. This is a relative definition able to coordinate a ‘perception of experience’ in terms of an learning-function, grounded in free-energy theory (the laws of physics) and the ecological dynamics of a spontaneous or ‘self- organising’ mechanism – Divergent Criticality. The methodology incorporated three studies: Pilot, Developmental and Exploratory. Over the three studies, Divergent Criticality was tested by developing a functional Affordance measure to address the Research Question – are perceptions as affective-cognitions made aware as reflecting the agential mediation of a self-regulating, optimal learning mechanism? Perception questionnaires of Situational Interest and Self-concept were used in Study One and Study Two to investigate their suitability in addressing the Research Question. Here, Factor Analysis and Structural Equation Modelling assessed the validity and reliability of these measures, developing robust questionnaires and a research design for testing Divergent Criticality. In Study Three, the Divergent Criticality hypothesis was found to be significant, supporting that a Divergent Criticality mechanism is in operation: When individuals are engaging with dynamic ecological challenges, perception is affective in accordance with Tolerance Optimisation, demonstrating that a Divergent Criticality mechanism is driving individuals to the limits of their Effectivity – an optimal learning state which is fundamental to life and naturalised in Universal laws

Learning action-oriented models through active inference

Converging theories suggest that organisms learn and exploit probabilistic models of their environment. However, it remains unclear how such models can be learned in practice. The open-ended complexity of natural environments means that it is generally infeasible for organisms to model their environment comprehensively. Alternatively, action-oriented models attempt to encode a parsimonious representation of adaptive agent-environment interactions. One approach to learning action-oriented models is to learn online in the presence of goal-directed behaviours. This constrains an agent to behaviourally relevant trajectories, reducing the diversity of the data a model need account for. Unfortunately, this approach can cause models to prematurely converge to sub-optimal solutions, through a process we refer to as a bad-bootstrap. Here, we exploit the normative framework of active inference to show that efficient action-oriented models can be learned by balancing goal-oriented and epistemic (information-seeking) behaviours in a principled manner. We illustrate our approach using a simple agent-based model of bacterial chemotaxis. We first demonstrate that learning via goal-directed behaviour indeed constrains models to behaviorally relevant aspects of the environment, but that this approach is prone to sub-optimal convergence. We then demonstrate that epistemic behaviours facilitate the construction of accurate and comprehensive models, but that these models are not tailored to any specific behavioural niche and are therefore less efficient in their use of data. Finally, we show that active inference agents learn models that are parsimonious, tailored to action, and which avoid bad bootstraps and sub-optimal convergence. Critically, our results indicate that models learned through active inference can support adaptive behaviour in spite of, and indeed because of, their departure from veridical representations of the environment. Our approach provides a principled method for learning adaptive models from limited interactions with an environment, highlighting a route to sample efficient learning algorithms

The active inference approach to ecological perception: general information dynamics for natural and artificial embodied cognition

The emerging neurocomputational vision of humans as embodied, ecologically embedded, social agents—who shape and are shaped by their environment—offers a golden opportunity to revisit and revise ideas about the physical and information-theoretic underpinnings of life, mind, and consciousness itself. In particular, the active inference framework (AIF) makes it possible to bridge connections from computational neuroscience and robotics/AI to ecological psychology and phenomenology, revealing common underpinnings and overcoming key limitations. AIF opposes the mechanistic to the reductive, while staying fully grounded in a naturalistic and information-theoretic foundation, using the principle of free energy minimization. The latter provides a theoretical basis for a unified treatment of particles, organisms, and interactive machines, spanning from the inorganic to organic, non-life to life, and natural to artificial agents. We provide a brief introduction to AIF, then explore its implications for evolutionary theory, ecological psychology, embodied phenomenology, and robotics/AI research. We conclude the paper by considering implications for machine consciousness

What is an emerging technology?

There is considerable and growing interest in the emergence of novel technologies, especially from the policy-making perspective. Yet as an area of study, emerging technologies lacks key foundational elements, namely a consensus on what classifies a technology as ’emergent’ and strong research designs that operationalize central theoretical concepts. The present paper aims to fill this gap by developing a definition of ’emerging technologies’ and linking this conceptual effort with the development of a framework for the operationalisation of technological emergence. The definition is developed by combining a basic understanding of the term and in particular the concept of ’emergence’ with a review of key innovation studies dealing with definitional issues of technological emergence. The resulting definition identifies five attributes that feature in the emergence of novel technologies. These are: (i) radical novelty, (ii) relatively fast growth, (iii) coherence, (iv) prominent impact, and (v) uncertainty and ambiguity. The framework for operationalising emerging technologies is then elaborated on the basis of the proposed attributes. To do so, we identify and review major empirical approaches (mainly in, although not limited to, the scientometric domain) for the detection and study of emerging technologies (these include indicators and trend analysis, citation analysis, co-word analysis, overlay mapping, and combinations thereof) and elaborate on how these can be used to operationalise the different attributes of emergence

Open-ended Search through Minimal Criterion Coevolution

Search processes guided by objectives are ubiquitous in machine learning. They iteratively reward artifacts based on their proximity to an optimization target, and terminate upon solution space convergence. Some recent studies take a different approach, capitalizing on the disconnect between mainstream methods in artificial intelligence and the field\u27s biological inspirations. Natural evolution has an unparalleled propensity for generating well-adapted artifacts, but these artifacts are decidedly non-convergent. This new class of non-objective algorithms induce a divergent search by rewarding solutions according to their novelty with respect to prior discoveries. While the diversity of resulting innovations exhibit marked parallels to natural evolution, the methods by which search is driven remain unnatural. In particular, nature has no need to characterize and enforce novelty; rather, it is guided by a single, simple constraint: survive long enough to reproduce. The key insight is that such a constraint, called the minimal criterion, can be harnessed in a coevolutionary context where two populations interact, finding novel ways to satisfy their reproductive constraint with respect to each other. Among the contributions of this dissertation, this approach, called minimal criterion coevolution (MCC), is the primary (1). MCC is initially demonstrated in a maze domain (2) where it evolves increasingly complex mazes and solutions. An enhancement to the initial domain (3) is then introduced, allowing mazes to expand unboundedly and validating MCC\u27s propensity for open-ended discovery. A more natural method of diversity preservation through resource limitation (4) is introduced and shown to maintain population diversity without comparing genetic distance. Finally, MCC is demonstrated in an evolutionary robotics domain (5) where it coevolves increasingly complex bodies with brain controllers to achieve principled locomotion. The overall benefit of these contributions is a novel, general, algorithmic framework for the continual production of open-ended dynamics without the need for a characterization of behavioral novelty