Generalised agent for solving higher board states of tic tac toe using Reinforcement Learning

Tic Tac Toe is amongst the most well-known games. It has already been shown that it is a biased game, giving more chances to win for the first player leaving only a draw or a loss as possibilities for the opponent, assuming both the players play optimally. Thus on average majority of the games played result in a draw. The majority of the latest research on how to solve a tic tac toe board state employs strategies such as Genetic Algorithms, Neural Networks, Co-Evolution, and Evolutionary Programming. But these approaches deal with a trivial board state of 3X3 and very little research has been done for a generalized algorithm to solve 4X4,5X5,6X6 and many higher states. Even though an algorithm exists which is Min-Max but it takes a lot of time in coming up with an ideal move due to its recursive nature of implementation. A Sample has been created on this link \url{https://bk-tic-tac-toe.herokuapp.com/} to prove this fact. This is the main problem that this study is aimed at solving i.e providing a generalized algorithm(Approximate method, Learning-Based) for higher board states of tic tac toe to make precise moves in a short period. Also, the code changes needed to accommodate higher board states will be nominal. The idea is to pose the tic tac toe game as a well-posed learning problem. The study and its results are promising, giving a high win to draw ratio with each epoch of training. This study could also be encouraging for other researchers to apply the same algorithm to other similar board games like Minesweeper, Chess, and GO for finding efficient strategies and comparing the results.Comment: 29 pages, 20 figures, 2022 Seventh International Conference on Parallel, Distributed and Grid Computing(PDGC

Automatic Game Parameter Tuning using General Video Game Agents

Automatic Game Design is a subfield of Game Artificial Intelligence that aims to study the usage of AI algorithms for assisting in game design tasks. This dissertation presents a research work in this field, focusing on applying an evolutionary algorithm to video game parameterization. The task we are interested in is player experience. N-Tuple Bandit Evolutionary Algorithm (NTBEA) is an evolutionary algorithm that was recently proposed and successfully applied in game parameterization in a simple domain, which is the first experiment included in this project. To further investigating its ability in evolving game parameters, We applied NTBEA to evolve parameter sets for three General Video Game AI (GVGAI) games, because GVGAI has variety supplies of video games in different types and the framework has already been prepared for parameterization. 9 positive increasing functions were picked as target functions as representations of the player expected score trends. Our initial assumption was that the evolved games should provide the game environments that allow players to obtain score in the same trend as one of these functions. The experiment results confirm this for some functions, and prove that the NTBEA is very much capable of evolving GVGAI games to satisfy this task

Learning From Geometry In Learning For Tactical And Strategic Decision Domains

Artificial neural networks (ANNs) are an abstraction of the low-level architecture of biological brains that are often applied in general problem solving and function approximation. Neuroevolution (NE), i.e. the evolution of ANNs, has proven effective at solving problems in a variety of domains. Information from the domain is input to the ANN, which outputs its desired actions. This dissertation presents a new NE algorithm called Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT), based on a novel indirect encoding of ANNs. The key insight in HyperNEAT is to make the algorithm aware of the geometry in which the ANNs are embedded and thereby exploit such domain geometry to evolve ANNs more effectively. The dissertation focuses on applying HyperNEAT to tactical and strategic decision domains. These domains involve simultaneously considering short-term tactics while also balancing long-term strategies. Board games such as checkers and Go are canonical examples of such domains; however, they also include real-time strategy games and military scenarios. The dissertation details three proposed extensions to HyperNEAT designed to work in tactical and strategic decision domains. The first is an action selector ANN architecture that allows the ANN to indicate its judgements on every possible action all at once. The second technique is called substrate extrapolation. It allows learning basic concepts at a low resolution, and then increasing the resolution to learn more advanced concepts. The iii final extension is geometric game-tree pruning, whereby HyperNEAT can endow the ANN the ability to focus on specific areas of a domain (such as a checkers board) that deserve more inspection. The culminating contribution is to demonstrate the ability of HyperNEAT with these extensions to play Go, a most challenging game for artificial intelligence, by combining HyperNEAT with UC

The dark side of the board: advances in chess Kriegspiel

While imperfect information games are an excellent model of real-world problems and tasks, they are often difficult for computer programs to play at a high level of proficiency, especially if they involve major uncertainty and a very large state space. Kriegspiel, a variant of chess making it similar to a wargame, is a perfect example: while the game was studied for decades from a game-theoretical viewpoint, it was only very recently that the first practical algorithms for playing it began to appear. This thesis presents, documents and tests a multi-sided effort towards making a strong Kriegspiel player, using heuristic searching, retrograde analysis and Monte Carlo tree search algorithms to achieve increasingly higher levels of play. The resulting program is currently the strongest computer player in the world and plays at an above-average human level

Machine learning at the nanoscale

Although scanning probe microscopy (SPM) techniques have allowed researchers to interact with the nanoscale for decades now, little improvement has been made to the incredibly manual, time consuming process of setting up, running, and analysing the results of these experiments, often arising due to the constantly varying shape of the probe apex. Unlike traditional computing methods, machine learning methods (with neural networks in particular) are considerably more capable of automating subjective tasks such as these, and we are only just beginning to explore the potential applications of this technology in SPM. In this thesis we explore a number of areas where machine learning could potentially massively change the way we go about SPM experimentation. We begin by discussing the history, theory, and experimental concepts of scanning tunnelling microscopy (STM), atomic force microscopy (AFM), and normal-incidence-x-ray standing wave (NIXSW). We then explore the makeup of a neural network and demonstrate how they can be applied to a variety of use-cases in SPM, including classification and policy prediction. Moving to the experimental chapters, we first discuss how we can successfully distinguish between STM tip states of the H:Si(100), Au(111) and Cu(111) surfaces. We also show that by adapting this network to work in real time, we improve performance while requiring on the order of 100x less data. We next discuss our attempts to combine these networks with expert examples to intelligently maintain tip apex sharpness during experimentation, envisioning an end-to-end automatic experiment. Because one of the main difficulties in applying machine learning is the frequent need to manually label data, we then show how we can use Monte Carlo simulations of self-organised AFM nanostructures to automatically label training data for a network, and then combine it with classical statistics and preprocessing to find specific structures in a mixed, messy dataset of real, experimental AFM images. As part of this, we also build a network to denoise experimental images. Finally, we present NIXSW results from an investigation into the temperature dependence of H20@C60, discussing the potential to use unsupervised clustering techniques to distinguish between noisy human-indistinguishable spectra to overcome limitations in data collection

Towards Real-World Federated Learning: Empirical Studies in the Domain of Embedded Systems

Context: Artificial intelligence (AI) has led a new phase of technical revolution and industrial development around the world since the twenty-first century, revolutionizing the way of production. Artificial intelligence (AI), an emerging information technology, is thriving, and AI application technologies are gaining traction, particularly in professional services such as healthcare, education, finance, security, etc. More machine learning technologies have begun to be thoroughly applied to the production stage as big data and cloud computing capabilities have improved. With the increased focus on Machine Learning applications and the rapid growth of distributed edge devices in the industry, we believe that utilizing a large number of edge devices will become increasingly important. The introduction of Federated Learning changes the situation in which data must be centrally uploaded to the cloud for processing and maximizes the use of edge devices\u27 computing and storage capabilities. With local data processing, the learning approach eliminates the need to upload large amounts of local data and reduces data transfer latency. Because Federated Learning does not require centralized data for model training, it is better suited to edge learning scenarios with limited data and privacy concerns. Objective: The purpose of this research is to identify the characteristics and problems of the Federated Learning methods, our new algorithms and frameworks that can assist companies in making the transition to Federated Learning, and empirically validate the proposed approaches. Method: To achieve these objectives, we adopted an empirical research approach with design science being our primary research method. We conducted a literature review, case studies, including semi-structured interviews and simulation experiments in close collaboration with software-intensive companies in the embedded systems domain. Results: We present four major findings in this paper. First, we present a state-of-the-art review of the empirical results reported in the existing Federated Learning literature. We then categorize those Federated Learning implementations into different application domains, identify their challenges, and propose six open research questions based on the problems identified in the literature. Second, we conduct a case study to explain why companies anticipate Federated Learning as a potential solution to the challenges they encountered when implementing machine learning components. We summarize the services that a comprehensive Federated Learning system must enable in industrial settings. Furthermore, we identify the primary barriers that companies must overcome in order to embrace and transition to Federated Learning. Based on our empirical findings, we propose five requirements for companies implementing reliable Federated Learning systems. Third, we develop and evaluate four architecture alternatives for a Federated Learning system, including centralized, hierarchical, regional, and decentralized architectures. We investigate the trade-o between communication latency, model evolution time, and model classification performance, which is critical for applying our findings to real-world industrial systems. Fourth, we introduce techniques and asynchronous frameworks for end-to-end on-device Federated Learning. The method is validated using a steering wheel angle prediction case. The local models of each edge vehicle can be continuously trained and shared with other vehicles to improve their local model prediction accuracy. Furthermore, we combine the asynchronous Federated Learning approach with Deep Neural Decision Forests and validate our method using important industry use cases in the automotive domain. Our findings show that Federated Learning can improve model training speed while lowering communication overhead without sacrificing accuracy, demonstrating that this technique has significant benefits to a wide range of real-world embedded systems. Future Work: In the future, we plan to test our approach in other use cases and look into more sophisticated neural networks integrated with our approach. In order to improve model training performance on resource-constrained edge devices in real-world embedded systems, we intend to design more appropriate aggregation methods and protocols. Furthermore, we intend to use the Federated Learning and Reinforcement Learning methods to assist the edge in evolving themselves autonomously and fully utilizing the computation capabilities of the edge devices

Adolescent Development in Context: Social, Psychological, and Neurological Foundations

This project was funded by KU Libraries’ Parent’s Campaign with support from the David Shulenburger Office of Scholarly Communication & Copyright and the Open Educational Resources Working Group in the University of Kansas Libraries.Increasingly, there is a tendency to characterize the teenage years as a time of general moral degeneration and deviance. This is unfortunate because the teenage years represent a key developmental period of the typical human lifespan, and from an evolutionary point of view, the actual characteristics that define adolescence represent critical learning opportunities. The increased sensitivity to social influences, identity formation, and social-emotional skills are just a few of such opportunities that require appropriate environments and contexts for optimal, healthy outcomes. Research in the field of adolescent development has not been immune to the negative stereotypes surrounding adolescence, and it is common to see researchers, either implicitly or explicitly, refer to adolescence as a high-risk, anomalous developmental stage that must be controlled, managed, or simply endured until adult-level abilities emerge spontaneously as a result of having survived an intrinsically tumultuous developmental time. More enlightened views of adolescence recognize that all biological adaptations have a cause and a purpose, and that the purpose of adolescence can be discerned from understanding the complex evolutionary history of humans as a group-based, family-based, highly social, sometimes competitive, abstract-thinking species. Understanding the biological foundations of adolescence is meaningless if one does not also consider how biology and environment interact. In humans, these interactions are highly complex and involve not only immediate physical realities, but also social, cultural, and historical realities that create complex contexts and webs of interactions. Therefore, this textbook seeks to reconcile the biological and neurological foundations of human development with the psychological and sociological mechanisms that formed and continue to influence human developmental trajectories. To this end, we have divided the textbook into three main sections. The first, Foundations of Adolescent Development, introduces the historical science of studying adolescence and the biological foundations of puberty. The second section, Contexts of Adolescent Development, considers the primary contextual factors that influence developmental outcomes during adolescence. These include work and employment, peers, in-school and out-of-school contexts, leisure time, and the family. The final section, Milestones of Adolescent Development, addresses the primary psychological milestones that represent healthy adjustment to adult roles and responsibilities in society. The domains of these milestones include cognition and decision-making; identity, meaning, and purpose, moral development, and sexuality. From an educational point of view, the objective of this textbook is to provide a resource that is capable of fostering advanced conceptual change and learning in the field of adolescent development in order to go beyond stereotypical portrayals of adolescence as a pathological condition. Organized in a manner designed to scaffold increasingly complex ideas, the textbook redefines adolescence a sensitive period of development characterized by phylogenetically derived experience-expectant states and complex interactions of biological, psychological, and social factors. The textbook draws from the latest advances in neuroscience and psychology to construct a practical framework for use in a wide range of academic and professional contexts, and it presents historical as well as contemporary research to accomplish a radical redefining of an often misunderstood and maligned developmental period

An evaluation of the challenges of Multilingualism in Data Warehouse development

In this paper we discuss Business Intelligence and define what is meant by support for Multilingualism in a Business Intelligence reporting context. We identify support for Multilingualism as a challenging issue which has implications for data warehouse design and reporting performance. Data warehouses are a core component of most Business Intelligence systems and the star schema is the approach most widely used to develop data warehouses and dimensional Data Marts. We discuss the way in which Multilingualism can be supported in the Star Schema and identify that current approaches have serious limitations which include data redundancy and data manipulation, performance and maintenance issues. We propose a new approach to enable the optimal application of multilingualism in Business Intelligence. The proposed approach was found to produce satisfactory results when used in a proof-of-concept environment. Future work will include testing the approach in an enterprise environmen

Proceedings of the 1st International Conference on Live Coding

Open Access peer reviewed papers on live coding published at the 1st International Conference on Live Coding (ICLC) in Leeds