Reinforcement Learning for Impartial Games and Complex Combinatorial Optimisation Problems

AlphaZero-style reinforcement learning algorithms have succeeded in various games, from traditional board games to advanced video games. However, their integration into combinatorial games, particularly impartial games, presents inherent challenges. A pivotal aspect of these challenges stems from the parity function's role in determining the winning strategies and the fact that they do not exploit the underlying structures of games besides the game rules. Within the domain of combinatorial optimisation, the task of finding large Condorcet domains is remarkably complex, marked by numerous local optima and deep-rooted mathematical structures. Notably, the most phenomenal method hinged mainly on Fishburn's alternating scheme, an approach intricately tied to the parity function. We demonstrate the intrinsic complexity of Condorcet domain generation by showcasing that diverse AI learning paradigms, from deep reinforcement learning and genetic algorithms to local search algorithms, tend to get stuck in some of the numerous local optima. Thus, a genuinely novel approach is needed. The main contribution of the thesis is to present a novel heuristic approach inspired by AlphaZero-style reinforcement learning but using significant expert domain knowledge and heavily relying on various databases accessed during the search. The Condorcet Domain Library, a research software written in C++ for high execution speed and providing Python interfaces for fast prototyping, was initially developed to implement the algorithm. It has evolved into a flexible library with various functionalities, underpinning multiple impactful research projects. In collaboration with mathematicians, further computational ideas, combined with our new algorithms and the library, led to new results for Condorcet domains, including the construction of a set-alternating scheme leading to a set of large CDs that were hitherto unknown and the discovery of new CDs with distinct properties, some of which are intriguingly linked to multi-agent systems. Furthermore, one of the most prominent results is the discovery of new record-breaking Condorcet domains on the number of alternatives ranging from 9 to 24

The Performance Optimization of ASP Solving Based on Encoding Rewriting and Encoding Selection

Answer set programming (ASP) has long been used for modeling and solving hard search problems. These problems are modeled in ASP as encodings, a collection of rules that declaratively describe the logic of the problem without explicitly listing how to solve it. It is common that the same problem has several different but equivalent encodings in ASP. Experience shows that the performance of these ASP encodings may vary greatly from instance to instance when processed by current state-of-the-art ASP grounder/solver systems. In particular, it is rarely the case that one encoding outperforms all others. Moreover, running an ASP system on one encoding for a specific instance may “take forever,” while running it on another encoding for this instance may yield a solution in a fraction of a second. The selection of a ”good” encoding for each instance is crucial to the performance of ASP solving. In this dissertation, I propose methods to improve the performance of ASP solving that exploit these observations. First, I designed and implemented methods that, given an encoding for a problem, rewrite it in several ways into new different but equivalent encodings. Second, I designed and implemented a system that given a set of input encodings of a problem, a set of problem instances, and an ASP grounder/solver system, automatically generates equivalent encodings and builds for each selected encoding its performance model. The model predicts for any instance the execution time that the grounder/solver system takes to process the instance under the corresponding encoding. These performance models are then used to improve solving efficiency: whenever a new instance arrives, the system selects the encoding predicted to perform the best on the instance and invokes the grounder/solver. The system also supports a scheduled execution and an interleaved execution of encodings, which are complementary to machine learning techniques. Third, I implemented algorithms that generate hard structured instances for several combinatorial problems I selected for our experimental study of the efficacy of the methods I developed. Hard instances can serve as the benchmark for evaluating the hardness of specific problems and contribute as training data to the platform I created to help build encoding selection models. The process can also provide meaningful insights into finding hard instances of other combinatorial problems

단백질 상호작용 네트워크의 삼각형 기반 변 점수 산정법

학위논문 (석사)-- 서울대학교 대학원 사범대학 수학교육과, 2017. 8. 김서령.Motivation: Uncovering the mystery of evolutionary mechanism of protein interaction networks has been actively conducted in order to understand interactions of proteins that induce biological processes in organisms. There have been many attempts to solve the mystery by proposing evolutionary models of protein interaction networks. Topological properties of protein interaction networks are mentioned several times and given an important role in these attempts since a validation of suggested models is made through topological properties of known protein interaction networks. While one group of researchers have made efforts to generate current protein interaction networks from some hypothetical infant state of protein interaction networks through suggested evolutionary models, another group of researchers have made efforts to estimate the phylogenetic age of proteins from evolutionary relationships. Recently, these efforts gave rise to the database of phylogenetic age of proteins and this allows many researchers to estimate ages of proteins in their interest easily. Recent studies on Mendelian diseases and cancer suggested that proteins associated with specific diseases populate certain category of the phylogenetic age of proteins. The fact that the topological properties of the protein interaction network have played important roles in the evolution of protein interaction networks tells us that topological properties of protein interaction network and properties of proteins, which is related to the evolution of the protein interaction network, is closely related in some level. As one can see from closeness in terms, the evolutionary model of protein interaction networks and phylogenetic age of proteins are closely related and thus topological properties of protein interactions, which is important in studies of the evolutionary models, can be used to estimate the phylogenetic age of proteins. Besides, the research results on the relationship between diseases and phylogenetic age of proteins motivate us to predict proteins associated to diseases by utilizing topological properties of protein interaction networks. Results: We construct a weighted human protein interaction network from a human protein interaction network which is provided via BioGRID database. The weight of an edge is defined as the number of triangles which contains this edge in the protein interaction network and thus we call this weight as the triangle score. We make comparison between the edge scores of a human protein interaction network given by STRING database and the triangle score. In an attempt to find relationship between the triangle score and properties of proteins that is related to the evolution of protein interaction networks, we make comparison between the triangle score and bit score, which is a measurement of protein sequence similarity. Moreover, we attempt to sieve out self-interacting proteins from the whole human proteins based on the triangle score. In an effort to predict the phylogenetic age of proteins based on the triangle score, firstly, we extract proteins that are incident on an edge that has a high triangle score from the weighted protein interaction network which we constructed with the triangle score. After the extraction, we make inquiries to the ProteinHistorian database to get phylogenetic ages of extracted proteins. Finally, we show that there is a relationship between triangle score and phylogenetic age by comparing the ratio of proteins with each phylogenetic age to whole human proteins and the ratio of extracted proteins with each phylogenetic age to whole extracted proteins. Based on the triangle score, we also attempt to predict disease associated proteins for several diseases. The fact that the topological properties of the protein interaction network have played important roles in the evolution of protein interaction networks tells us that topological properties of protein interaction network and properties of proteins, which is related to the evolution of the protein interaction network, is closely related in some level. As one can see from closeness in terms, the evolutionary model of protein interaction networks and phylogenetic age of proteins are closely related and topological properties of protein interactions, which is important in studies of the evolutionary models, can be used to estimate the phylogenetic age of proteins. Besides, the research results on the relationship between diseases and phylogenetic age of proteins motivate us to predict proteins associated to diseases by utilizing topological properties of protein interaction networks. Results: We construct a weighted human protein interaction network from a human protein interaction network which is provided via BioGrid database. The weight of an edge is defined as the number of triangles which contains this edge in the protein interaction network and thus we call this weight as the triangle score. We make comparison between the edge scores of a human protein interaction network given by STRING database and the triangle score. In an attempt to find relationship between the triangle score and properties of proteins that is related to the evolution of protein interaction networks, we make comparison between the triangle score and bit score, which is a measurement of protein sequence similarity. Moreover, we attempt to sieve out self-interacting proteins from the whole human proteins based on the triangle score. In an effort to predict the phylogenetic age of proteins based on the triangle score, firstly, we extract proteins that are incident on an edge that has a high triangle score from the weighted protein interaction network which we constructed with the triangle score. After the extraction, we make inquiries to the ProteinHistorian database to get phylogenetic ages of extracted proteins. Finally, we show that there is a relationship between triangle score and phylogenetic age by comparing the ratio of proteins with each phylogenetic age to whole human proteins and the ratio of extracted proteins with each phylogenetic age to whole extracted proteins. Based on the triangle score, we also attempt to predict disease associated proteins for several diseases.제 1 장 Introduction 1 제 2 장 Materials and Methods 11 제 3 장 Results 40 제 4 장 Conclusions 55 Bibliography 59 국문초록 63Maste

prototypical implementations

In this technical report, we present prototypical implementations of innovative tools and methods developed according to the working plan outlined in Technical Report TR-B-09-05 [23]. We present an ontology modularization and integration framework and the SVoNt server, the server-side end of an SVN- based versioning system for ontologies in the Corporate Ontology Engineering pillar. For the Corporate Semantic Collaboration pillar, we present the prototypical implementation of a light-weight ontology editor for non-experts and an ontology based expert finder system. For the Corporate Semantic Search pillar, we present a prototype for algorithmic extraction of relations in folksonomies, a tool for trend detection using a semantic analyzer, a tool for automatic classification of web documents using Hidden Markov models, a personalized semantic recommender for multimedia content, and a semantic search assistant developed in co-operation with the Museumsportal Berlin. The prototypes complete the next milestone on the path to an integral Cor- porate Semantic Web architecture based on the three pillars Corporate Ontol- ogy Engineering, Corporate Semantic Collaboration, and Corporate Semantic Search, as envisioned in [23]

Semantic social network analysis

Mestrado em Engenharia InformáticaOver the last few years, online social networks have become part of society, affecting the way people interact, and share and spread ideas. Their large and still increasing popularity led to the emergence of multiple huge social datasets. Although a vast set of social network analysis methods and algorithms have already been proposed and scrutinized by the research community, most of them aren’t prepared for a direct application over the rich and heterogeneous information contained in the online social network datasets. With the emergence of the Semantic Web, the up until now closed datasets are evolving to become one semantically enriched and distributed social dataset shared by all online social network applications. In such an environment, new methods and tools that fill the gap between the new social and semantic web technologies, and well established and accepted social network analysis methods are required. In that sense, the main goal of this thesis work, as part of the Toursplan project, is to adapt the current Toursplan platform so it fits into the Social and Semantic Web environments, while also providing mechanisms to perform social network analysis over the Toursplan semantically enriched social dataset.Nos últimos anos as redes sociais online têm sido progressivamente adoptadas pela sociedade, influenciando a maneira como as pessoas interagem, partilham e distribuem ideias. A sua crescente popularidade levou ao aparecimento de múltiplas bases de dados com enormes quantidades de dados relativos a interacções sociais. Apesar de já existir uma vasta quantidade de métodos e algoritmos para análise de redes sociais propostos e escrutinados pela comunidade científica, a maior parte não se encontra preparada para a sua aplicação directa sobre a informação, rica e heterogénea, contida nas bases de dados das redes sociais online. Com o aparecimento da Web Semântica, os dados sociais até agora enclausurados e protegidos por cada uma das entidades responsáveis pelos mesmos, estão a convergir para formar uma enorme massa de informação distribuida e semanticamente enriquecida, partilhada por todas as aplicações com funcionalidades sociais. Num ambiente como este, novos métodos e ferramentas são necessários para que exista uma ponte entre as novas tecnologias emergentes devido à Web Social e Semântica, e os já bem aceites e estabelecidos métodos e algoritmos para análise de redes sociais. Sendo assim, o principal objectivo desta tese como parte do projecto Toursplan, é não só a adaptação da actual plataforma Toursplan de forma a que esta se possa encaixar na Web Social e Semântica, mas também o desenvolvimento de mecanismos que permitam analisar a semanticamente enriquecida base de dados Toursplan. Do trabalho desenvolvido resultaram: • A análise do estado da arte das actuais redes sociais online, e de métodos e algoritmos de análise de redes sociais; • A ontologia Toursplan, que descreve o domínio de conhecimento da plataforma Toursplan, incluindo o perfil dos utilizadores, pontos de interesse turísticos e planeamento de viagens; • A ontologia SocioNet, que descreve o domínio de conhecimento relativo a métodos e algoritmos de análise de redes sociais, proporcionando um modelo para a persistência de dados resultantes da execução de múltiplos algoritmos previamente analisados e descritos; • A implementação de uma base de triplos para a plataforma Toursplan, e a migração de toda a informação encontrada na base de dados relacional para a base de triplos; • A implementação de uma camada de acesso a dados, com base na framework Jena para a Web Semântica, que permite o acesso à base de triplos através das ontologias Toursplan e SocioNet; • A implementação de um protótipo que corresponde à nova aplicação Web da rede social Toursplan; • A especificação e implementação de uma fase de normalização de dados provenientes de bases de triplos (ou grafos de triplos) com base em múltiplos padrões de modelação, usando a camada de acesso a dados anterior e a ontologia SocioNet; • A implementação de alguns dos algoritmos de análise de redes sociais previamente analisados com base na camada SocioNet, a sua execução sobre informação normalizada, e a análise dos resultados obtidos

Linked data as medium for distributed Multi-Agent Systems

The conceptual design and discussion of multi-agents systems (MAS) typically focuses on agents and their models, and the elements and effects in the environment which they perceive. This view, however, leaves out potential pitfalls in the later implementation of the system that may stem from limitations in data models, interfaces, or protocols by which agents and environments exchange information. By today, the research community agrees that for this, that the environment should be understood as well as abstraction layer by which agents access, interpret, and modify elements within the environment. This, however, blurs the the line of the environment being the sum of interactive elements and phenomena perceivable by agents, and the underlying technology by which this information and interactions are offered to agents. This thesis proposes as remedy to consider as third component of multi agent systems, besides agents and environments, the digital medium by which the environment is provided to agents. "Medium" then refers to exactly this technological component via which environment data is published interactively towards the agents, and via which agents perceive, interpret, and finally, modify the underlying environment data. Furthermore, this thesis will detail how MAS may use capabilities of a properly chosen medium to achieve coordinating system behaviors. A suitable candidate technology for digital agent media comes from the Semantic Web in form of Linked Data. In addition to conceptual discussions about the notions of digital agent media, this thesis will provide in detail a specification of a Linked Data agent medium, and detail on means to implement MAS around Linked Data media technologies.Sowohl der konzeptuelle Entwurf von, als auch die wissenschaftliche Diskussion über Multi-Agenten-Systeme (MAS) konzentrieren sich für gewöhnlich auf die Agenten selbst, die Agentenmodelle, sowie die Elemente und Effekte, die sie in ihrer Umgebung wahrnehmen. Diese Betrachtung lässt jedoch mögliche Probleme in einer späteren Implementierung aus, die von Einschränkungen in Datenmodellen, Schnittstellen, oder Protokollen herrühren können, über die Agenten und ihre Umgebung Informationen miteinander austauschen. Heutzutage ist sich die Forschungsgemeinschaft einig, dass die Umgebung als solche als Abstraktionsschicht verstanden werden sollte, über die Agenten Umgebungseffekte und -elemente wahrnehmen, interpretieren, und mit ihnen interagieren. Diese Betrachtungsweise verschleiert jedoch die Trennung zwischen der Umgebung als die Sammlung interaktiver Elemente und wahrnehmbarer Phänomene auf der einen Seite, und der zugrundeliegenden Technologie, über die diese Information den Agenten bereitgestellt wird, auf der anderen. Diese Dissertation schlägt als Lösung vor, zusätzlich zu Agenten undUmgebung ein digitales Medium, über das Agenten die Umgebung bereitgestellt wird, als drittes Element von Multi-Agenten-Systemen zu betrachten. Der Begriff "Medium" bezieht sich dann genau auf diese technologische Komponente, über die Umgebungsinformationen Agenten interaktiv bereitgestellt werden, und über die Agenten die zugrundeliegenden Daten wahrnehmen, interpretieren, und letztendlich modifizieren. Desweiteren wird diese Dissertation aufzeigen, wie die Eigenschaften eines sorgfältig gewählten Mediums ausgenutzt werden können, um ein koordiniertes Systemverhalten zu erreichen. Ein geeigneter Kandidat für ein digitales Agentenmedium findet sich im Ökosystem des „Semantic Web”, in Form von „Linked Data”, wörtlich („verknüpfte Daten”). Zusätzlich zu einer konzeptionellen Diskussion über die Natur digitaler Agenten- Media, spezifiziert diese Dissertation „Linked Data” als Agentenmedium detailliert aus, und beschreibt im Detail die Mittel, wie sich MAS um Linked Data Technologien herum implementieren lassen