A hyperheuristic methodology to generate adaptive strategies for games

Hyperheuristics have been successfully applied in solving a variety of computational search problems. In this study, we investigate a hyper-heuristic methodology to generate adaptive strategies for games. Based on a set of low-level heuristics (or strategies), a hyper-heuristic game player can generate strategies which adapt to both the behaviour of the co-players and the game dynamics. By using a simple heuristic selection mechanism, a number of existing heuristics for specialised games can be integrated into an automated game player. As examples, we develop hyperheuristic game players for three games: iterated prisoner's dilemma, repeated Goofspiel and the competitive traveling salesmen problem. The results demonstrate that a hyperheuristic game player outperforms the low-level heuristics, when used individually in game playing and it can generate adaptive strategies even if the low-level heuristics are deterministic. This methodology provides an efficient way to develop new strategies for games based on existing strategies

DRLMA: An Intelligent Move Acceptance for Combinatorial Optimization Problems based on Deep Reinforcement Learning

Numerous heuristic solution methods have been developed to tackle combinatorial opti- mization problems, often customized for specific problem domains and use-cases where they exhibit remarkable performance. However, their effectiveness diminishes significantly when applied to problem domains for which they were not originally designed, showcasing poor generalization capabilities. In contrast, metaheuristics are higher-level heuristics so- lution methods that aim to be applicable to a wide range of different problems. Perturba- tive metaheuristics operate by traversing the solution space through iterative application of modifications induced by low-level heuristics. This process continues until a specified stopping criteria is met, enabling the method to efficiently explore and refine solutions. A central aspect of these search-based methods is the move acceptance scheme, which determines whether or not the suggested modification is to be applied. The Simulated Annealing acceptance criteria, for instance, occasionally accepts uphill moves, or worse so- lutions, in order to explore the space of solutions and help the search escape local optima. In this thesis we propose Deep Reinforcement Learning Move Acceptance (DRLMA), a general move acceptance framework that leverages Deep Reinforcement Learning into the acceptance decision. A Deep RL agent is trained using problem-independent search information, enabling it to learn high-level acceptance strategies regardless of the specific combinatorial optimization problem at hand. We show that by replacing the Simulated Annealing acceptance criteria with DRLMA in two different heuristic selection frame- works, namely Adaptive Large Neighborhood Search (ALNS) and Deep Reinforcement Learning Hyperheuristic (DRLH), we are generally able to improve the performance of the respective search methods, the degree of improvement ranging from only slightly in the worst cases to considerably in the best cases.Masteroppgave i informatikkINF399MAMN-INFMAMN-PRO

Enhancing the Performance of Search Heuristics. Variable Fitness Functions and other Methods to Enhance Heuristics for Dynamic Workforce Scheduling.

Scheduling large real world problems is a complex process and finding high quality solutions is not a trivial task. In cooperation with Trimble MRM Ltd., who provide scheduling solutions for many large companies, a problem is identified and modelled. It is a general model which encapsulates several important scheduling, routing and resource allocation problems in literature. Many of the state-of-the-art heuristics for solve scheduling problems and indeed other problems require specialised heuristics tailored for the problem they are to solve. While these provide good solutions a lot of expert time is needed to study the problem, and implement solutions. This research investigates methods to enhance existing search based methods. We study hyperheuristic techniques as a general search based heuristic. Hyperheuristics raise the generality of the solution method by using a set of tools (low level heuristics) to work on the solution. These tools are problem specific and usually make small changes to the problem. It is the task of the hyperheuristic to determine which tool to use and when. Low level heuristics using exact/heuristic hybrid method are used in this thesis along with a new Tabu based hyperheuristic which decreases the amount of CPU time required to produce good quality solutions. We also develop and investigate the Variable Fitness Function approach, which provides a new way of enhancing most search-based heuristics in terms of solution quality. If a fitness function is pushing hard in a certain direction, a heuristic may ultimately fail because it cannot escape local minima. The Variable Fitness Function allows the fitness function to change over the search and use objective measures not used in the fitness calculation. The Variable Fitness Function and its ability to generalise are extensively tested in this thesis. The two aims of the thesis are achieved and the methods are analysed in depth. General conclusions and areas of future work are also identified

General Game Heuristic Prediction Based on Ludeme Descriptions

This paper investigates the performance of different general-game-playing heuristics for games in the Ludii general game system. Based on these results, we train several regression learning models to predict the performance of these heuristics based on each game's description file. We also provide a condensed analysis of the games available in Ludii, and the different ludemes that define them.Comment: 4 pages, 1 figure, 2 table

Solving Competitive Traveling Salesman Problem Using Gray Wolf Optimization Algorithm

In this paper a Gray Wolf Optimization (GWO) algorithm is presented to solve the Competitive Traveling Salesman Problem (CTSP). In CTSP, there are numbers of non-cooperative salesmen their goal is visiting a larger possible number of cities with lowest cost and most gained benefit. Each salesman will get a benefit when he visits unvisited city before all other salesmen. Two approaches have been used in this paper, the first one called static approach, it is mean evenly divides the cities among salesmen. The second approach is called parallel at which all cities are available to all salesmen and each salesman tries to visit as much as possible of the unvisited cities. The algorithms are executed for 1000 times and the results prove that the GWO is very efficient giving an indication of the superiority of GWO in solving CTSP

General Board Game Concepts

Many games often share common ideas or aspects between them, such as their rules, controls, or playing area. However, in the context of General Game Playing (GGP) for board games, this area remains under-explored. We propose to formalise the notion of "game concept", inspired by terms generally used by game players and designers. Through the Ludii General Game System, we describe concepts for several levels of abstraction, such as the game itself, the moves played, or the states reached. This new GGP feature associated with the ludeme representation of games opens many new lines of research. The creation of a hyper-agent selector, the transfer of AI learning between games, or explaining AI techniques using game terms, can all be facilitated by the use of game concepts. Other applications which can benefit from game concepts are also discussed, such as the generation of plausible reconstructed rules for incomplete ancient games, or the implementation of a board game recommender system

Hybrid optimizer for expeditious modeling of virtual urban environments

Tese de mestrado. Engenharia Informática. Faculdade de Engenharia. Universidade do Porto. 200

Automation and Control

Advances in automation and control today cover many areas of technology where human input is minimized. This book discusses numerous types and applications of automation and control. Chapters address topics such as building information modeling (BIM)–based automated code compliance checking (ACCC), control algorithms useful for military operations and video games, rescue competitions using unmanned aerial-ground robots, and stochastic control systems

Is Evolutionary Computation evolving fast enough?

Evolutionary Computation (EC) has been an active research area for over 60 years, yet its commercial/home uptake has not been as prolific as we might have expected. By way of comparison, technologies such as 3D printing, which was introduced about 35 years ago, has seen much wider uptake, to the extent that it is now available to home users and is routinely used in manufacturing. Other technologies, such as immersive reality and artificial intelligence have also seen commercial uptake and acceptance by the general public. In this paper we provide a brief history of EC, recognizing the significant contributions that have been made by its pioneers. We focus on two methodologies (Genetic Programming and Hyper-heuristics), which have been proposed as being suitable for automated software development, and question why they are not used more widely by those outside of the academic community. We suggest that different research strands need to be brought together into one framework before wider uptake is possible. We hope that this position paper will serve as a catalyst for automated software development that is used on a daily basis by both companies and home users

A Heuristically Generated Metric Approach to the Solution of Chase Problem

In this work, heuristic, hyper-heuristic, and metaheuristic approaches are reviewed. Distance metrics are also examined to solve the “puzzle problems by searching” in AI. A viewpoint is brought by introducing the so-called Heuristically Generated Angular Metric Approach (HAMA) through the explanation of the metrics world. Distance metrics are applied to “cat and mouse” problem where cat and mouse makes smart moves relative to each other and therefore makes more appropriate decisions. The design is built around Fuzzy logic control to determine route finding between the pursuer and prey. As the puzzle size increases, the effect of HAMA can be distinguished more clearly in terms of computation time towards a solution. Hence, mouse will gain more time in perceiving the incoming danger, thus increasing the percentage of evading the danger. ‘Caught and escape percentages vs. number of cats’ for three distance metrics have been created and the results evaluated comparatively. Given three termination criteria, it is never inconsistent to define two different objective functions: either the cat travels the distance to catch the mouse, or the mouse increases the percentage of escape from the cat