Unsupervised Feature Learning through Divergent Discriminative Feature Accumulation

Unlike unsupervised approaches such as autoencoders that learn to reconstruct their inputs, this paper introduces an alternative approach to unsupervised feature learning called divergent discriminative feature accumulation (DDFA) that instead continually accumulates features that make novel discriminations among the training set. Thus DDFA features are inherently discriminative from the start even though they are trained without knowledge of the ultimate classification problem. Interestingly, DDFA also continues to add new features indefinitely (so it does not depend on a hidden layer size), is not based on minimizing error, and is inherently divergent instead of convergent, thereby providing a unique direction of research for unsupervised feature learning. In this paper the quality of its learned features is demonstrated on the MNIST dataset, where its performance confirms that indeed DDFA is a viable technique for learning useful features.Comment: Corrected citation formattin

Machine learning for outlier detection in medical imaging

Outlier detection is an important problem with diverse practical applications. In medical imaging, there are many diagnostic tasks that can be framed as outlier detection. Since pathologies can manifest in so many different ways, the goal is typically to learn from normal, healthy data and identify any deviations. Unfortunately, many outliers in the medical domain can be subtle and specific, making them difficult to detect without labelled examples. This thesis analyzes some of the nuances of medical data and the value of labels in this context. It goes on to propose several strategies for unsupervised learning. More specifically, these methods are designed to learn discriminative features from data of a single class. One approach uses divergent search to continually find different ways to partition the data and thereby accumulates a repertoire of features. The other proposed methods are based on a self-supervised task that distorts normal data to form a contrasting class. A network can then be trained to localize the irregularities and estimate the degree of foreign interference. This basic technique is further enhanced using advanced image editing to create more natural irregularities. Lastly, the same self-supervised task is repurposed for few-shot learning to create a framework for adaptive outlier detection. These proposed methods are able to outperform conventional strategies across a range of datasets including brain MRI, abdominal CT, chest X-ray, and fetal ultrasound data. In particular, these methods excel at detecting more subtle irregularities. This complements existing methods and aims to maximize benefit to clinicians by detecting fine-grained anomalies that can otherwise require intense scrutiny. Note that all approaches to outlier detection must accept some assumptions; these will affect which types of outliers can be detected. As such, these methods aim for broad generalization within the most medically relevant categories. Ultimately, the hope is to support clinicians and to focus their attention and efforts on the data that warrants further analysis.Open Acces

상상 모델: 구성 패턴 생성 네트워크의 다양성 탐색을 통한 이미지 제작

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2019. 2. 문병로.Divergent Search methods are devised to resolve the problem falling into a trap of local optima, an arch-enemy of stochastic optimization algorithms. Novelty Search and Surprise Search, inter alia, use the concept of {\it behavior} and explore behavior space defined by it, maintaining evolutionary divergence and they have shown great performance in this respect. Moreover, coupling novelty and surprise concept was designed based on ideas that those two algorithms search behavioral space in a different way. The combination of two algorithms can be viewed as multiobjective optimization algorithm, and this approach enhanced the performance than using one divergent search method only. Since several divergent search methods have outperformed existing stochastic optimization algorithms in recent studies of robotics, it has been applied to many other domains, such as robot morphology, artificial life and generating images. Particularly, the Innovation Engines applied Novelty Search to image generating method so as to create novel and interesting images. In this paper, we propose Imagination Model that adopts Novelty-Surprise Search which is the combination of Novelty and Surprise Search instead of pure Novelty Search, as an extension of Innovation Engine. Evolutionary algorithms using Novelty Search, Surprise Search, Novelty-Surprise Search are compared via well-trained deep neural networks defining the behaviors of individuals in terms of creating interesting images. Results of experiments indicate that Novelty-Surprise Search outperforms Novelty Search and Surprise Search even in image domainit searches and explores vast behavioral space more extensively than each search algorithm on its own.다양성 검색 방법은 확률적 최적화 알고리즘의 주적인 지역 최적해의 함정에 빠지는 문제를 해결하기 위해 고안되었다. 그중에서도 참신함 탐색과 놀라움 탐색은 {\it 행동}이라는 개념과 그 개념이 정의하는 행동 공간을 탐색하며 진화적 다양성을 유지했고 이 점에 있어서 훌륭한 성능을 보여주었다. 그뿐만 아니라 두 다양성 탐색이 서로 다른 방식으로 행동 공간을 탐색하는 데에서 착안하여, 참신함과 놀라움을 결합하는 알고리즘이 설계되었다. 두 알고리즘의 조합은 다목적 최적화 알고리즘으로 간주할 수 있는데, 이 접근 방식은 둘 중 하나만의 다양성 탐색 방법을 사용할 때보다 성능이 개선됨을 다양한 연구에서 보여주었다. 이처럼 여러 다양성 탐색이 기존의 확률적 최적화 알고리즘을 뛰어 넘는 성능을 보였기 때문에, 로봇 형태학, 인공생명, 이미지 생성처럼 다양한 분야에 응용되어왔다. 특히, 혁신 엔진은 새로우면서도 흥미로운 이미지를 창조하기 위해 이미지 생성 방법에 참신함 탐색을 적용했다. 이에 더해 우리는 이 논문에서 상상 모델을 제안한다. 이 상상 모델은 혁신 엔진의 확장으로서 순수한 참신함 탐색 대신 참신함 탐색과 놀라움 탐색을 결합한 참신함-놀라움 탐색을 도입한다. 참신함 탐색, 놀라움 탐색 그리고 참신함-놀라움 탐색을 사용한 진화 연산을 이미지 생성에 관한 측면에서 비교하는 실험을 진행하며, 이들은 모두 심층 인공신경망을 통해 그들이 사용하는 행동이라는 개념이 정의된다. 실험 결과를 살펴보면, 참신함-놀라움 탐색은 단순히 참신함 탐색이나 놀라움 탐색 각각을 따로따로 사용하는 것보다 더 넓은 행동 공간을 더 광범위하게 탐색하는 모습을 보여주었다. 이로부터, 다른 분야뿐 아니라 이미지 생성 영역에서도 참신함-놀라움 탐색이 참신함 탐색과 놀라움 탐색 각각을 뛰어넘는 성능을 보인다는 것을 확인하였다.Abstract i Contents iii List of Figures v List of Tables vi Chapter 1 Introduction 1 Chapter 2 Background 4 2.1 CPPN-NEAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Novelty Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Surprise Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.4 Combining Novelty and Surprise Score . . . . . . . . . . . . . . . . . . . 7 2.5 Innovation Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Chapter 3 Methods 9 3.1 Image Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2 Behavioral Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3 Imagination Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Chapter 4 Experiments 13 4.1 Fitness Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2 Deep Neural Networks and Dataset . . . . . . . . . . . . . . . . . . . . . . 14 Chapter 5 Results 16 Chapter 6 Discussion 25 Chapter 7 Conclusion 27 Bibliography 29 요약 33Maste

Searching for surprise

Inspired by the notion of surprise for unconventional discovery in computational creativity, we introduce a general search algorithm we name surprise search. Surprise search is grounded in the divergent search paradigm and is fabricated within the principles of metaheuristic (evolutionary) search. The algorithm mimics the self-surprise cognitive process of creativity and equips computational creators with the ability to search for outcomes that deviate from the algorithm’s expected behavior. The predictive model of expected outcomes is based on historical trails of where the search has been and some local information about the search space. We showcase the basic steps of the algorithm via a problem solving (maze navigation) and a generative art task. What distinguishes surprise search from other forms of divergent search, such as the search for novelty, is its ability to diverge not from earlier and seen outcomes but rather from predicted and unseen points in the creative domain considered.This work has been supported in part by the FP7 Marie Curie CIG project AutoGameDesign (project no: 630665).peer-reviewe

Identifying divergent design thinking through the observable behavior of service design novices

© 2018, Springer Nature B.V. Design thinking holds the key to innovation processes, but is often difficult to detect because of its implicit nature. We undertook a study of novice designers engaged in team-based design exercises in order to explore the correlation between design thinking and designers’ physical (observable) behavior and to identify new, objective, design thinking identification methods. Our study addresses the topic by using data collection method of “think aloud” and data analysis method of “protocol analysis” along with the unconstrained concept generation environment. Collected data from the participants without service design experience were analyzed by open and selective coding. Through the research, we found correlations between physical activity and divergent thinking, and also identified physical behaviors that predict a designer’s transition to divergent thinking. We conclude that there are significant relations between designers’ design thinking and the behavioral features of their body and face. This approach opens possible new ways to undertake design process research and also design capability evaluation