Data-Efficient Machine Learning with Focus on Transfer Learning

Machine learning (ML) has attracted a significant amount of attention from the artifi- cial intelligence community. ML has shown state-of-art performance in various fields, such as signal processing, healthcare system, and natural language processing (NLP). However, most conventional ML algorithms suffer from three significant difficulties: 1) insufficient high-quality training data, 2) costly training process, and 3) domain dis- crepancy. Therefore, it is important to develop solutions for these problems, so the future of ML will be more sustainable. Recently, a new concept, data-efficient ma- chine learning (DEML), has been proposed to deal with the current bottlenecks of ML. Moreover, transfer learning (TL) has been considered as an effective solution to address the three shortcomings of conventional ML. Furthermore, TL is one of the most active areas in the DEML. Over the past ten years, significant progress has been made in TL. In this dissertation, I propose to address the three problems by developing a software- oriented framework and TL algorithms. Firstly, I introduce a DEML framework and a evaluation system. Moreover, I present two novel TL algorithms and applications on real-world problems. Furthermore, I will first present the first well-defined DEML framework and introduce how it can address the challenges in ML. After that, I will give an updated overview of the state-of-the-art and open challenges in the TL. I will then introduce two novel algorithms for two of the most challenging TL topics: distant domain TL and cross-modality TL (image-text). A detailed algorithm introduction and preliminary results on real-world applications (Covid-19 diagnosis and image clas- sification) will be presented. Then, I will discuss the current trends in TL algorithms and real-world applications. Lastly, I will present the conclusion and future research directions

Cross-layer design of multi-hop wireless networks

MULTI -hop wireless networks are usually defined as a collection of nodes equipped with radio transmitters, which not only have the capability to communicate each other in a multi-hop fashion, but also to route each others’ data packets. The distributed nature of such networks makes them suitable for a variety of applications where there are no assumed reliable central entities, or controllers, and may significantly improve the scalability issues of conventional single-hop wireless networks. This Ph.D. dissertation mainly investigates two aspects of the research issues related to the efficient multi-hop wireless networks design, namely: (a) network protocols and (b) network management, both in cross-layer design paradigms to ensure the notion of service quality, such as quality of service (QoS) in wireless mesh networks (WMNs) for backhaul applications and quality of information (QoI) in wireless sensor networks (WSNs) for sensing tasks. Throughout the presentation of this Ph.D. dissertation, different network settings are used as illustrative examples, however the proposed algorithms, methodologies, protocols, and models are not restricted in the considered networks, but rather have wide applicability. First, this dissertation proposes a cross-layer design framework integrating a distributed proportional-fair scheduler and a QoS routing algorithm, while using WMNs as an illustrative example. The proposed approach has significant performance gain compared with other network protocols. Second, this dissertation proposes a generic admission control methodology for any packet network, wired and wireless, by modeling the network as a black box, and using a generic mathematical 0. Abstract 3 function and Taylor expansion to capture the admission impact. Third, this dissertation further enhances the previous designs by proposing a negotiation process, to bridge the applications’ service quality demands and the resource management, while using WSNs as an illustrative example. This approach allows the negotiation among different service classes and WSN resource allocations to reach the optimal operational status. Finally, the guarantees of the service quality are extended to the environment of multiple, disconnected, mobile subnetworks, where the question of how to maintain communications using dynamically controlled, unmanned data ferries is investigated

Command & Control: Understanding, Denying and Detecting - A review of malware C2 techniques, detection and defences

In this survey, we first briefly review the current state of cyber attacks, highlighting significant recent changes in how and why such attacks are performed. We then investigate the mechanics of malware command and control (C2) establishment: we provide a comprehensive review of the techniques used by attackers to set up such a channel and to hide its presence from the attacked parties and the security tools they use. We then switch to the defensive side of the problem, and review approaches that have been proposed for the detection and disruption of C2 channels. We also map such techniques to widely-adopted security controls, emphasizing gaps or limitations (and success stories) in current best practices.Comment: Work commissioned by CPNI, available at c2report.org. 38 pages. Listing abstract compressed from version appearing in repor

Acoustic data optimisation for seabed mapping with visual and computational data mining

Oceans cover 70% of Earth’s surface but little is known about their waters. While the echosounders, often used for exploration of our oceans, have developed at a tremendous rate since the WWII, the methods used to analyse and interpret the data still remain the same. These methods are inefficient, time consuming, and often costly in dealing with the large data that modern echosounders produce. This PhD project will examine the complexity of the de facto seabed mapping technique by exploring and analysing acoustic data with a combination of data mining and visual analytic methods. First we test the redundancy issues in multibeam echosounder (MBES) data by using the component plane visualisation of a Self Organising Map (SOM). A total of 16 visual groups were identified among the 132 statistical data descriptors. The optimised MBES dataset had 35 attributes from 16 visual groups and represented a 73% reduction in data dimensionality. A combined Principal Component Analysis (PCA) + k-means was used to cluster both the datasets. The cluster results were visually compared as well as internally validated using four different internal validation methods. Next we tested two novel approaches in singlebeam echosounder (SBES) data processing and clustering – using visual exploration for outlier detection and direct clustering of time series echo returns. Visual exploration identified further outliers the automatic procedure was not able to find. The SBES data were then clustered directly. The internal validation indices suggested the optimal number of clusters to be three. This is consistent with the assumption that the SBES time series represented the subsurface classes of the seabed. Next the SBES data were joined with the corresponding MBES data based on identification of the closest locations between MBES and SBES. Two algorithms, PCA + k-means and fuzzy c-means were tested and results visualised. From visual comparison, the cluster boundary appeared to have better definitions when compared to the clustered MBES data only. The results seem to indicate that adding SBES did in fact improve the boundary definitions. Next the cluster results from the analysis chapters were validated against ground truth data using a confusion matrix and kappa coefficients. For MBES, the classes derived from optimised data yielded better accuracy compared to that of the original data. For SBES, direct clustering was able to provide a relatively reliable overview of the underlying classes in survey area. The combined MBES + SBES data provided by far the best accuracy for mapping with almost a 10% increase in overall accuracy compared to that of the original MBES data. The results proved to be promising in optimising the acoustic data and improving the quality of seabed mapping. Furthermore, these approaches have the potential of significant time and cost saving in the seabed mapping process. Finally some future directions are recommended for the findings of this research project with the consideration that this could contribute to further development of seabed mapping problems at mapping agencies worldwide

Video Summarization Using Deep Neural Networks: A Survey

Video summarization technologies aim to create a concise and complete synopsis by selecting the most informative parts of the video content. Several approaches have been developed over the last couple of decades and the current state of the art is represented by methods that rely on modern deep neural network architectures. This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization. After presenting the motivation behind the development of technologies for video summarization, we formulate the video summarization task and discuss the main characteristics of a typical deep-learning-based analysis pipeline. Then, we suggest a taxonomy of the existing algorithms and provide a systematic review of the relevant literature that shows the evolution of the deep-learning-based video summarization technologies and leads to suggestions for future developments. We then report on protocols for the objective evaluation of video summarization algorithms and we compare the performance of several deep-learning-based approaches. Based on the outcomes of these comparisons, as well as some documented considerations about the suitability of evaluation protocols, we indicate potential future research directions.Comment: Journal paper; Under revie

Distributed Knowledge Modeling and Integration of Model-Based Beliefs into the Clinical Decision-Making Process

Das Treffen komplexer medizinischer Entscheidungen wird durch die stetig steigende Menge an zu berücksichtigenden Informationen zunehmend komplexer. Dieser Umstand ist vor allem auf die Verfügbarkeit von immer präziseren diagnostischen Methoden zur Charakterisierung der Patienten zurückzuführen (z.B. genetische oder molekulare Faktoren). Hiermit einher geht die Entwicklung neuartiger Behandlungsstrategien und Wirkstoffe sowie die damit verbundenen Evidenzen aus klinischen Studien und Leitlinien. Dieser Umstand stellt die behandelnden Ärztinnen und Ärzte vor neuartige Herausforderungen im Hinblick auf die Berücksichtigung aller relevanten Faktoren im Kontext der klinischen Entscheidungsfindung. Moderne IT-Systeme können einen wesentlichen Beitrag leisten, um die klinischen Experten weitreichend zu unterstützen. Diese Assistenz reicht dabei von Anwendungen zur Vorverarbeitung von Daten für eine Reduktion der damit verbundenen Komplexität bis hin zur systemgestützten Evaluation aller notwendigen Patientendaten für eine therapeutischen Entscheidungsunterstützung. Möglich werden diese Funktionen durch die formale Abbildung von medizinischem Fachwissen in Form einer komplexen Wissensbasis, welche die kognitiven Prozesse im Entscheidungsprozess adaptiert. Entsprechend werden an den Prozess der IT-konformen Wissensabbildung erhöhte Anforderungen bezüglich der Validität und Signifikanz der enthaltenen Informationen gestellt. In den ersten beiden Kapiteln dieser Arbeit wurden zunächst wichtige methodische Grundlagen im Kontext der strukturierten Abbildung von Wissen sowie dessen Nutzung für die klinische Entscheidungsunterstützung erläutert. Hierbei wurden die inhaltlichen Kernthemen weiterhin im Rahmen eines State of the Art mit bestehenden Ansätzen abgeglichen, um den neuartigen Charakter der vorgestellten Lösungen herauszustellen. Als innovativer Kern wurde zunächst die Konzeption und Umsetzung eines neuartigen Ansatzes zur Fusion von fragmentierten Wissensbausteinen auf der formalen Grundlage von Bayes-Netzen vorgestellt. Hierfür wurde eine neuartige Datenstruktur unter Verwendung des JSON Graph Formats erarbeitet. Durch die Entwicklung von qualifizierten Methoden zum Umgang mit den formalen Kriterien eines Bayes-Netz wurden weiterhin Lösungen aufgezeigt, welche einen automatischen Fusionsprozess durch einen eigens hierfür entwickelten Algorithmus ermöglichen. Eine prototypische und funktionale Plattform zur strukturierten und assistierten Integration von Wissen sowie zur Erzeugung valider Bayes-Netze als Resultat der Fusion wurde unter Verwendung eines Blockchain Datenspeichers implementiert und in einer Nutzerstudie gemäß ISONORM 9241/110-S evaluiert. Aufbauend auf dieser technologischen Plattform wurden im Anschluss zwei eigenständige Entscheidungsunterstützungssysteme vorgestellt, welche relevante Anwendungsfälle im Kontext der HNO-Onkologie adressieren. Dies ist zum einen ein System zur personalisierten Bewertung von klinischen Laborwerten im Kontext einer Radiochemotherapie und zum anderen ein in Form eines Dashboard implementiertes Systems zur effektiveren Informationskommunikation innerhalb des Tumor Board. Beide Konzepte wurden hierbei zunächst im Rahmen einer initialen Nutzerstudie auf Relevanz geprüft, um eine nutzerzentrische Umsetzung zu gewährleisten. Aufgrund des zentralen Fokus dieser Arbeit auf den Bereich der klinischen Entscheidungsunterstützung, werden an zahlreichen Stellen sowohl kritische als auch optimistische Aspekte der damit verbundenen praktischen Lösungen diskutiert.:1 Introduction 1.1 Motivation and Clinical Setting 1.2 Objectives 1.3 Thesis Outline 2 State of the Art 2.1 Medical Knowledge Modeling 2.2 Knowledge Fusion 2.3 Clinical Decision Support Systems 2.4 Clinical Information Access 3 Fundamentals 3.1 Evidence-Based Medicine 3.1.1 Literature-Based Evidence 3.1.2 Practice-Based Evidence 3.1.3 Patient-Directed Evidence 3.2 Knowledge Representation Formats 3.2.1 Logic-Based Representation 3.2.2 Procedural Representation 3.2.3 Network or Graph-Based Representation 3.3 Knowledge-Based Clinical Decision Support 3.4 Conditional Probability and Bayesian Networks 3.5 Clinical Reasoning 3.5.1 Deterministic Reasoning 3.5.2 Probabilistic Reasoning 3.6 Knowledge Fusion of Bayesian Networks 4 Block-Based Collaborative Knowledge Modeling 4.1 Data Model 4.1.1 Belief Structure 4.1.2 Conditional Probabilities 4.1.3 Metadata 4.2 Constraint-Based Automatic Knowledge Fusion 4.2.1 Fusion of the Bayesian Network Structures 4.2.2 Fusion of the Conditional Probability Tables 4.3 Blockchain-Based Belief Storage and Retrieval 4.3.1 Blockchain Characteristics 4.3.2 Relevance for Belief Management 5 Selected CDS Applications for Clinical Practice 5.1 Distributed Knowledge Modeling Platform 5.1.1 Requirement Analysis 5.1.2 System Architecture 5.1.3 System Evaluation 5.1.4 Limitations of the Proposed Solution 5.2 Personalization of Laboratory Findings 5.2.1 Requirement Analysis 5.2.2 System Architecture 5.2.3 Limitations of the Proposed Solution 5.3 Dashboard for Collaborative Decision-Making in the Tumor Board 5.3.1 Requirement Analysis 5.3.2 System Architecture 5.3.3 Limitations of the Proposed Solution 6 Discussion 6.1 Goal Achievements 6.2 Contributions and Conclusion 7 Bibliograph

Landscape of standing variation for tandem duplications in Drosophila yakuba and Drosophila simulans

We have used whole genome paired-end Illumina sequence data to identify tandem duplications in 20 isofemale lines of D. yakuba, and 20 isofemale lines of D. simulans and performed genome wide validation with PacBio long molecule sequencing. We identify 1,415 tandem duplications that are segregating in D. yakuba as well as 975 duplications in D. simulans, indicating greater variation in D. yakuba. Additionally, we observe high rates of secondary deletions at duplicated sites, with 8% of duplicated sites in D. simulans and 17% of sites in D. yakuba modified with deletions. These secondary deletions are consistent with the action of the large loop mismatch repair system acting to remove polymorphic tandem duplication, resulting in rapid dynamics of gain and loss in duplicated alleles and a richer substrate of genetic novelty than has been previously reported. Most duplications are present in only single strains, suggesting deleterious impacts are common. D. simulans shows larger numbers of whole gene duplications in comparison to larger proportions of gene fragments in D. yakuba. D. simulans displays an excess of high frequency variants on the X chromosome, consistent with adaptive evolution through duplications on the D. simulans X or demographic forces driving duplicates to high frequency. We identify 78 chimeric genes in D. yakuba and 38 chimeric genes in D. simulans, as well as 143 cases of recruited non-coding sequence in D. yakuba and 96 in D. simulans, in agreement with rates of chimeric gene origination in D. melanogaster. Together, these results suggest that tandem duplications often result in complex variation beyond whole gene duplications that offers a rich substrate of standing variation that is likely to contribute both to detrimental phenotypes and disease, as well as to adaptive evolutionary change.Comment: Revised Version- Accepted at Molecular Biology and Evolutio

Data Science and Analytics in Industrial Maintenance: Selection, Evaluation, and Application of Data-Driven Methods

Data-driven maintenance bears the potential to realize various benefits based on multifaceted data assets generated in increasingly digitized industrial environments. By taking advantage of modern methods and technologies from the field of data science and analytics (DSA), it is possible, for example, to gain a better understanding of complex technical processes and to anticipate impending machine faults and failures at an early stage. However, successful implementation of DSA projects requires multidisciplinary expertise, which can rarely be covered by individual employees or single units within an organization. This expertise covers, for example, a solid understanding of the domain, analytical method and modeling skills, experience in dealing with different source systems and data structures, and the ability to transfer suitable solution approaches into information systems. Against this background, various approaches have emerged in recent years to make the implementation of DSA projects more accessible to broader user groups. These include structured procedure models, systematization and modeling frameworks, domain-specific benchmark studies to illustrate best practices, standardized DSA software solutions, and intelligent assistance systems. The present thesis ties in with previous efforts and provides further contributions for their continuation. More specifically, it aims to create supportive artifacts for the selection, evaluation, and application of data-driven methods in the field of industrial maintenance. For this purpose, the thesis covers four artifacts, which were developed in several publications. These artifacts include (i) a comprehensive systematization framework for the description of central properties of recurring data analysis problems in the field of industrial maintenance, (ii) a text-based assistance system that offers advice regarding the most suitable class of analysis methods based on natural language and domain-specific problem descriptions, (iii) a taxonomic evaluation framework for the systematic assessment of data-driven methods under varying conditions, and (iv) a novel solution approach for the development of prognostic decision models in cases of missing label information. Individual research objectives guide the construction of the artifacts as part of a systematic research design. The findings are presented in a structured manner by summarizing the results of the corresponding publications. Moreover, the connections between the developed artifacts as well as related work are discussed. Subsequently, a critical reflection is offered concerning the generalization and transferability of the achieved results. Thus, the thesis not only provides a contribution based on the proposed artifacts; it also paves the way for future opportunities, for which a detailed research agenda is outlined.:List of Figures List of Tables List of Abbreviations 1 Introduction 1.1 Motivation 1.2 Conceptual Background 1.3 Related Work 1.4 Research Design 1.5 Structure of the Thesis 2 Systematization of the Field 2.1 The Current State of Research 2.2 Systematization Framework 2.3 Exemplary Framework Application 3 Intelligent Assistance System for Automated Method Selection 3.1 Elicitation of Requirements 3.2 Design Principles and Design Features 3.3 Prototypical Instantiation and Evaluation 4 Taxonomic Framework for Method Evaluation 4.1 Survey of Prognostic Solutions 4.2 Taxonomic Evaluation Framework 4.3 Exemplary Framework Application 5 Method Application Under Industrial Conditions 5.1 Conceptualization of a Solution Approach 5.2 Prototypical Implementation and Evaluation 6 Discussion of the Results 6.1 Connections Between Developed Artifacts and Related Work 6.2 Generalization and Transferability of the Results 7 Concluding Remarks Bibliography Appendix I: Implementation Details Appendix II: List of Publications A Publication P1: Focus Area Systematization B Publication P2: Focus Area Method Selection C Publication P3: Focus Area Method Selection D Publication P4: Focus Area Method Evaluation E Publication P5: Focus Area Method ApplicationDatengetriebene Instandhaltung birgt das Potential, aus den in Industrieumgebungen vielfältig anfallenden Datensammlungen unterschiedliche Nutzeneffekte zu erzielen. Unter Verwendung von modernen Methoden und Technologien aus dem Bereich Data Science und Analytics (DSA) ist es beispielsweise möglich, das Verhalten komplexer technischer Prozesse besser nachzuvollziehen oder bevorstehende Maschinenausfälle und Fehler frühzeitig zu erkennen. Eine erfolgreiche Umsetzung von DSA-Projekten erfordert jedoch multidisziplinäres Expertenwissen, welches sich nur selten von einzelnen Personen bzw. Einheiten innerhalb einer Organisation abdecken lässt. Dies umfasst beispielsweise ein fundiertes Domänenverständnis, Kenntnisse über zahlreiche Analysemethoden, Erfahrungen im Umgang mit verschiedenen Quellsystemen und Datenstrukturen sowie die Fähigkeit, geeignete Lösungsansätze in Informationssysteme zu überführen. Vor diesem Hintergrund haben sich in den letzten Jahren verschiedene Ansätze herausgebildet, um die Durchführung von DSA-Projekten für breitere Anwendergruppen zugänglich zu machen. Dazu gehören strukturierte Vorgehensmodelle, Systematisierungs- und Modellierungsframeworks, domänenspezifische Benchmark-Studien zur Veranschaulichung von Best Practices, Standardlösungen für DSA-Software und intelligente Assistenzsysteme. An diese Arbeiten knüpft die vorliegende Dissertation an und liefert weitere Artefakte, um insbesondere die Selektion, Evaluation und Anwendung datengetriebener Methoden im Bereich der industriellen Instandhaltung zu unterstützen. Insgesamt erstreckt sich die Abhandlung auf vier Artefakte, die in einzelnen Publikationen erarbeitet wurden. Dies umfasst (i) ein umfangreiches Systematisierungsframework zur Beschreibung zentraler Ausprägungen wiederkehrender Datenanalyseprobleme im Bereich der industriellen Instandhaltung, (ii) ein textbasiertes Assistenzsystem, welches ausgehend von natürlichsprachlichen und domänenspezifischen Problembeschreibungen eine geeignete Klasse von Analysemethoden vorschlägt, (iii) ein taxonomisches Evaluationsframework zur systematischen Bewertung von datengetriebenen Methoden unter verschiedenen Rahmenbedingungen sowie (iv) einen neuartigen Lösungsansatz zur Entwicklung von prognostischen Entscheidungsmodellen im Fall von eingeschränkter Informationslage. Die Konstruktion der Artefakte wird durch einzelne Forschungsziele im Rahmen eines systematischen Forschungsdesigns angeleitet. Neben der Darstellung der einzelnen Forschungsbeiträge unter Bezugnahme auf die erzielten Ergebnisse der dazugehörigen Publikationen werden auch die Verbindungen zwischen den entwickelten Artefakten beleuchtet und Zusammenhänge zu angrenzenden Arbeiten hergestellt. Zudem erfolgt eine kritische Reflektion der Ergebnisse hinsichtlich ihrer Verallgemeinerung und Übertragung auf andere Rahmenbedingungen. Dadurch liefert die vorliegende Abhandlung nicht nur einen Beitrag anhand der erzeugten Artefakte, sondern ebnet auch den Weg für fortführende Forschungsarbeiten, wofür eine detaillierte Forschungsagenda erarbeitet wird.:List of Figures List of Tables List of Abbreviations 1 Introduction 1.1 Motivation 1.2 Conceptual Background 1.3 Related Work 1.4 Research Design 1.5 Structure of the Thesis 2 Systematization of the Field 2.1 The Current State of Research 2.2 Systematization Framework 2.3 Exemplary Framework Application 3 Intelligent Assistance System for Automated Method Selection 3.1 Elicitation of Requirements 3.2 Design Principles and Design Features 3.3 Prototypical Instantiation and Evaluation 4 Taxonomic Framework for Method Evaluation 4.1 Survey of Prognostic Solutions 4.2 Taxonomic Evaluation Framework 4.3 Exemplary Framework Application 5 Method Application Under Industrial Conditions 5.1 Conceptualization of a Solution Approach 5.2 Prototypical Implementation and Evaluation 6 Discussion of the Results 6.1 Connections Between Developed Artifacts and Related Work 6.2 Generalization and Transferability of the Results 7 Concluding Remarks Bibliography Appendix I: Implementation Details Appendix II: List of Publications A Publication P1: Focus Area Systematization B Publication P2: Focus Area Method Selection C Publication P3: Focus Area Method Selection D Publication P4: Focus Area Method Evaluation E Publication P5: Focus Area Method Applicatio

상상 모델: 구성 패턴 생성 네트워크의 다양성 탐색을 통한 이미지 제작

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2019. 2. 문병로.Divergent Search methods are devised to resolve the problem falling into a trap of local optima, an arch-enemy of stochastic optimization algorithms. Novelty Search and Surprise Search, inter alia, use the concept of {\it behavior} and explore behavior space defined by it, maintaining evolutionary divergence and they have shown great performance in this respect. Moreover, coupling novelty and surprise concept was designed based on ideas that those two algorithms search behavioral space in a different way. The combination of two algorithms can be viewed as multiobjective optimization algorithm, and this approach enhanced the performance than using one divergent search method only. Since several divergent search methods have outperformed existing stochastic optimization algorithms in recent studies of robotics, it has been applied to many other domains, such as robot morphology, artificial life and generating images. Particularly, the Innovation Engines applied Novelty Search to image generating method so as to create novel and interesting images. In this paper, we propose Imagination Model that adopts Novelty-Surprise Search which is the combination of Novelty and Surprise Search instead of pure Novelty Search, as an extension of Innovation Engine. Evolutionary algorithms using Novelty Search, Surprise Search, Novelty-Surprise Search are compared via well-trained deep neural networks defining the behaviors of individuals in terms of creating interesting images. Results of experiments indicate that Novelty-Surprise Search outperforms Novelty Search and Surprise Search even in image domainit searches and explores vast behavioral space more extensively than each search algorithm on its own.다양성 검색 방법은 확률적 최적화 알고리즘의 주적인 지역 최적해의 함정에 빠지는 문제를 해결하기 위해 고안되었다. 그중에서도 참신함 탐색과 놀라움 탐색은 {\it 행동}이라는 개념과 그 개념이 정의하는 행동 공간을 탐색하며 진화적 다양성을 유지했고 이 점에 있어서 훌륭한 성능을 보여주었다. 그뿐만 아니라 두 다양성 탐색이 서로 다른 방식으로 행동 공간을 탐색하는 데에서 착안하여, 참신함과 놀라움을 결합하는 알고리즘이 설계되었다. 두 알고리즘의 조합은 다목적 최적화 알고리즘으로 간주할 수 있는데, 이 접근 방식은 둘 중 하나만의 다양성 탐색 방법을 사용할 때보다 성능이 개선됨을 다양한 연구에서 보여주었다. 이처럼 여러 다양성 탐색이 기존의 확률적 최적화 알고리즘을 뛰어 넘는 성능을 보였기 때문에, 로봇 형태학, 인공생명, 이미지 생성처럼 다양한 분야에 응용되어왔다. 특히, 혁신 엔진은 새로우면서도 흥미로운 이미지를 창조하기 위해 이미지 생성 방법에 참신함 탐색을 적용했다. 이에 더해 우리는 이 논문에서 상상 모델을 제안한다. 이 상상 모델은 혁신 엔진의 확장으로서 순수한 참신함 탐색 대신 참신함 탐색과 놀라움 탐색을 결합한 참신함-놀라움 탐색을 도입한다. 참신함 탐색, 놀라움 탐색 그리고 참신함-놀라움 탐색을 사용한 진화 연산을 이미지 생성에 관한 측면에서 비교하는 실험을 진행하며, 이들은 모두 심층 인공신경망을 통해 그들이 사용하는 행동이라는 개념이 정의된다. 실험 결과를 살펴보면, 참신함-놀라움 탐색은 단순히 참신함 탐색이나 놀라움 탐색 각각을 따로따로 사용하는 것보다 더 넓은 행동 공간을 더 광범위하게 탐색하는 모습을 보여주었다. 이로부터, 다른 분야뿐 아니라 이미지 생성 영역에서도 참신함-놀라움 탐색이 참신함 탐색과 놀라움 탐색 각각을 뛰어넘는 성능을 보인다는 것을 확인하였다.Abstract i Contents iii List of Figures v List of Tables vi Chapter 1 Introduction 1 Chapter 2 Background 4 2.1 CPPN-NEAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Novelty Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Surprise Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.4 Combining Novelty and Surprise Score . . . . . . . . . . . . . . . . . . . 7 2.5 Innovation Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Chapter 3 Methods 9 3.1 Image Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2 Behavioral Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.3 Imagination Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Chapter 4 Experiments 13 4.1 Fitness Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2 Deep Neural Networks and Dataset . . . . . . . . . . . . . . . . . . . . . . 14 Chapter 5 Results 16 Chapter 6 Discussion 25 Chapter 7 Conclusion 27 Bibliography 29 요약 33Maste

Multi-Modality Human Action Recognition

Human action recognition is very useful in many applications in various areas, e.g. video surveillance, HCI (Human computer interaction), video retrieval, gaming and security. Recently, human action recognition becomes an active research topic in computer vision and pattern recognition. A number of action recognition approaches have been proposed. However, most of the approaches are designed on the RGB images sequences, where the action data was collected by RGB/intensity camera. Thus the recognition performance is usually related to various occlusion, background, and lighting conditions of the image sequences. If more information can be provided along with the image sequences, more data sources other than the RGB video can be utilized, human actions could be better represented and recognized by the designed computer vision system.;In this dissertation, the multi-modality human action recognition is studied. On one hand, we introduce the study of multi-spectral action recognition, which involves the information from different spectrum beyond visible, e.g. infrared and near infrared. Action recognition in individual spectra is explored and new methods are proposed. Then the cross-spectral action recognition is also investigated and novel approaches are proposed in our work. On the other hand, since the depth imaging technology has made a significant progress recently, where depth information can be captured simultaneously with the RGB videos. The depth-based human action recognition is also investigated. I first propose a method combining different type of depth data to recognize human actions. Then a thorough evaluation is conducted on spatiotemporal interest point (STIP) based features for depth-based action recognition. Finally, I advocate the study of fusing different features for depth-based action analysis. Moreover, human depression recognition is studied by combining facial appearance model as well as facial dynamic model