A machine learning approach to constructing Ramsey graphs leads to the Trahtenbrot-Zykov problem.

Attempts at approaching the well-known and difficult problem of constructing Ramsey graphs via machine learning lead to another difficult problem posed by Zykov in 1963 (now commonly referred to as the Trahtenbrot-Zykov problem): For which graphs F does there exist some graph G such that the neighborhood of every vertex in G induces a subgraph isomorphic to F? Chapter 1 provides a brief introduction to graph theory. Chapter 2 introduces Ramsey theory for graphs. Chapter 3 details a reinforcement learning implementation for Ramsey graph construction. The implementation is based on board game software, specifically the AlphaZero program and its success learning to play games from scratch. The chapter ends with a description of how computing challenges naturally shifted the project towards the Trahtenbrot-Zykov problem. Chapter 3 also includes recommendations for continuing the project and attempting to overcome these challenges. Chapter 4 defines the Trahtenbrot-Zykov problem and outlines its history, including proofs of results omitted from their original papers. This chapter also contains a program for constructing graphs with all neighborhood-induced subgraphs isomorphic to a given graph F. The end of Chapter 4 presents constructions from the program when F is a Ramsey graph. Constructing such graphs is a non-trivial task, as Bulitko proved in 1973 that the Trahtenbrot-Zykov problem is undecidable. Chapter 5 is a translation from Russian to English of this famous result, a proof not previously available in English. Chapter 6 introduces Cayley graphs and their relationship to the Trahtenbrot-Zykov problem. The chapter ends with constructions of Cayley graphs Γ in which the neighborhood of every vertex of Γ induces a subgraph isomorphic to a given Ramsey graph, which leads to a conjecture regarding the unique extremal Ramsey(4, 4) graph

SmartSwarm - A Multi-Agent Reinforcement Learning based Particle Swarm Optimization Algorithm

Particle Swarm Optimization is a renowned continuous optimization method that utilizes Swarm Intelligence to find solutions to complex non-linear optimization problems efficiently. Since its proposal, many developments have been put forward to improve its capabilities by enhancing the stochastic and tunable component of the algorithm. This thesis introduces SmartSwarm, a variant of Particle Swarm Optimization that utilizes Multi-Agent Reinforcement Learning to control the velocity of a swarm of particles. This framework has the capability of incorporating domain-specific information in the optimization process, as well as adapting a self-taught velocity function. We show how this framework has the ability to discover a velocity function to maximize the performance of the algorithm.Masteroppgave i informatikkINF399MAMN-PROGMAMN-IN

Artificial Intelligence-Based Drug Design and Discovery

The drug discovery process from hit-to-lead has been a challenging task that requires simultaneously optimizing numerous factors from maximizing compound activity, efficacy to minimizing toxicity and adverse reactions. Recently, the advance of artificial intelligence technique enables drugs to be efficiently purposed in silico prior to chemical synthesis and experimental evaluation. In this chapter, we present fundamental concepts of artificial intelligence and their application in drug design and discovery. The emphasis will be on machine learning and deep learning, which demonstrated extensive utility in many branches of computer-aided drug discovery including de novo drug design, QSAR (Quantitative Structure–Activity Relationship) analysis, drug repurposing and chemical space visualization. We will demonstrate how artificial intelligence techniques can be leveraged for developing chemoinformatics pipelines and presented with real-world case studies and practical applications in drug design and discovery. Finally, we will discuss limitations and future direction to guide this rapidly evolving field

Design with shapes grammars and reinforcement learning.

Política de acceso abierto tomada de: https://v2.sherpa.ac.uk/id/publication/1733Shape grammars are a powerful and appealing formalism for automatic shape generation in computer-based design systems. This paper presents a proposal complementing the generative power of shape grammars with reinforcement learning techniques. We use simple (naive) shape grammars capable of generating a large variety of different designs. In order to generate those designs that comply with given design requirements, the grammar is subject to a process of machine learning using reinforcement learning techniques. Based on this method, we have developed a system for architectural design, aimed at generating two-dimensional layout schemes of single-family housing units. Using relatively simple grammar rules, we learn to generate schemes that satisfy a set of requirements stated in a design guideline. Obtained results are presented and discussed.La publicación recoge los resultados del proyecto “Nuevas Técnicas Inteligentes de Decisión Aplicada al Proyecto Arquitectónico” (TIN2009-14179, Gobierno de España

Improving the generalizability and robustness of large-scale traffic signal control

A number of deep reinforcement-learning (RL) approaches propose to control traffic signals. In this work, we study the robustness of such methods along two axes. First, sensor failures and GPS occlusions create missing-data challenges and we show that recent methods remain brittle in the face of these missing data. Second, we provide a more systematic study of the generalization ability of RL methods to new networks with different traffic regimes. Again, we identify the limitations of recent approaches. We then propose using a combination of distributional and vanilla reinforcement learning through a policy ensemble. Building upon the state-of-the-art previous model which uses a decentralized approach for large-scale traffic signal control with graph convolutional networks (GCNs), we first learn models using a distributional reinforcement learning (DisRL) approach. In particular, we use implicit quantile networks (IQN) to model the state-action return distribution with quantile regression. For traffic signal control problems, an ensemble of standard RL and DisRL yields superior performance across different scenarios, including different levels of missing sensor data and traffic flow patterns. Furthermore, the learning scheme of the resulting model can improve zero-shot transferability to different road network structures, including both synthetic networks and real-world networks (e.g., Luxembourg, Manhattan). We conduct extensive experiments to compare our approach to multi-agent reinforcement learning and traditional transportation approaches. Results show that the proposed method improves robustness and generalizability in the face of missing data, varying road networks, and traffic flows

Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis

Each year, expert-level performance is attained in increasingly-complex multiagent domains, notable examples including Go, Poker, and StarCraft II. This rapid progression is accompanied by a commensurate need to better understand how such agents attain this performance, to enable their safe deployment, identify limitations, and reveal potential means of improving them. In this paper we take a step back from performance-focused multiagent learning, and instead turn our attention towards agent behavior analysis. We introduce a model-agnostic method for discovery of behavior clusters in multiagent domains, using variational inference to learn a hierarchy of behaviors at the joint and local agent levels. Our framework makes no assumption about agents' underlying learning algorithms, does not require access to their latent states or policies, and is trained using only offline observational data. We illustrate the effectiveness of our method for enabling the coupled understanding of behaviors at the joint and local agent level, detection of behavior changepoints throughout training, discovery of core behavioral concepts, demonstrate the approach's scalability to a high-dimensional multiagent MuJoCo control domain, and also illustrate that the approach can disentangle previously-trained policies in OpenAI's hide-and-seek domain

Supporting And Developing Self-Regulatory Behaviours In Early Childhood In Young Children With High Levels Of Impulsive Behaviour

Deficits in self-regulatory skills underlie or contribute to a range of adverse developmental problems and disorders, including ADHD (Barkley, 1997), eating disorders (Attie & Brooks-Gunn, 1995) and risk-taking behaviour (Cantor & Sanderson 1998; Eisenberg et al., 2005). Self-regulation has been recognised as an important factor in aiding academic achievements of school-age children. There is less knowledge of the subject in early childhood, yet the development of self-regulatory has been described as an important milestone in early childhood development (Shonkoff & Phillips, 2000). This research describes the implementation of an intervention programme in kindergartens that aimed to help young children with highly impulsive behaviour, develop self-regulatory behaviors. The children were identified by the Achenbach Child behaviour check list (1.5-5) and by the kindergarten teachers. This Research was based on mixed methods. The quantitave data reveled a number of children with highly impulsive behaviour and difficulties in self-regulation. The qualitative data gave a deeper interpretation to these children’s behaviour and the difficulties involved. After the implementation of the program the kindergarten teachers reported on an increase in the children’s self–regulatory skills.By understanding and supporting the processes involved in the development of self-regulation skills, it might be possible for significant adults in young children's lives to have a substantial effect in aiding young children, who are highly impulsive. This was the rationale for the present research

Imbalanced data classification and its application in cyber security

Cyber security, also known as information technology security or simply as information security, aims to protect government organizations, companies and individuals by defending their computers, servers, electronic systems, networks, and data from malicious attacks. With the advancement of client-side on the fly web content generation techniques, it becomes easier for attackers to modify the content of a website dynamically and gain access to valuable information. The impact of cybercrime to the global economy is now more than ever, and it is growing day by day. Among various types of cybercrimes, financial attacks are widely spread and the financial sector is among most targeted. Both corporations and individuals are losing a huge amount of money each year. The majority portion of financial attacks is carried out by banking malware and web-based attacks. The end users are not always skilled enough to differentiate between injected content and actual contents of a webpage. Designing a real-time security system for ensuring a safe browsing experience is a challenging task. Some of the existing solutions are designed for client side and all the users have to install it in their system, which is very difficult to implement. In addition, various platforms and tools are used by organizations and individuals, therefore, different solutions are needed to be designed. The existing server-side solution often focuses on sanitizing and filtering the inputs. It will fail to detect obfuscated and hidden scripts. This is a realtime security system and any significant delay will hamper user experience. Therefore, finding the most optimized and efficient solution is very important. To ensure an easy installation and integration capabilities of any solution with the existing system is also a critical factor to consider. If the solution is efficient but difficult to integrate, then it may not be a feasible solution for practical use. Unsupervised and supervised data classification techniques have been widely applied to design algorithms for solving cyber security problems. The performance of these algorithms varies depending on types of cyber security problems and size of datasets. To date, existing algorithms do not achieve high accuracy in detecting malware activities. Datasets in cyber security and, especially those from financial sectors, are predominantly imbalanced datasets as the number of malware activities is significantly less than the number of normal activities. This means that classifiers for imbalanced datasets can be used to develop supervised data classification algorithms to detect malware activities. Development of classifiers for imbalanced data sets has been subject of research over the last decade. Most of these classifiers are based on oversampling and undersampling techniques and are not efficient in many situations as such techniques are applied globally. In this thesis, we develop two new algorithms for solving supervised data classification problems in imbalanced datasets and then apply them to solve malware detection problems. The first algorithm is designed using the piecewise linear classifiers by formulating this problem as an optimization problem and by applying the penalty function method. More specifically, we add more penalty to the objective function for misclassified points from minority classes. The second method is based on the combination of the supervised and unsupervised (clustering) algorithms. Such an approach allows one to identify areas in the input space where minority classes are located and to apply local oversampling or undersampling. This approach leads to the design of more efficient and accurate classifiers. The proposed algorithms are tested using real-world datasets. Results clearly demonstrate superiority of newly introduced algorithms. Then we apply these algorithms to design classifiers to detect malwares.Doctor of Philosoph