Monte-Carlo tree search with heuristic knowledge: A novel way in solving capturing and life and death problems in Go

Monte-Carlo (MC) tree search is a new research field. Its effectiveness in searching large state spaces, such as the Go game tree, is well recognized in the computer Go community. Go domain- specific heuristics and techniques as well as domain-independent heuristics and techniques are sys- tematically investigated in the context of the MC tree search in this dissertation. The search extensions based on these heuristics and techniques can significantly improve the effectiveness and efficiency of the MC tree search. Two major areas of investigation are addressed in this dissertation research: I. The identification and use of the effective heuristic knowledge in guiding the MC simulations, II. The extension of the MC tree search algorithm with heuristics. Go, the most challenging board game to the machine, serves as the test bed. The effectiveness of the MC tree search extensions is demonstrated through the performances of Go tactic problem solvers using these techniques. The main contributions of this dissertation include: 1. A heuristics based Monte-Carlo tactic tree search framework is proposed to extend the standard Monte-Carlo tree search. 2. (Go) Knowledge based heuristics are systematically investigated to improve the Monte-Carlo tactic tree search. 3. Pattern learning is demonstrated as effective in improving the Monte-Carlo tactic tree search. 4. Domain knowledge independent tree search enhancements are shown as effective in improving the Monte-Carlo tactic tree search performances. 5. A strong Go Tactic solver based on proposed algorithms outperforms traditional game tree search algorithms. The techniques developed in this dissertation research can benefit other game domains and ap- plication fields

Constructing constrained-version of magic squares using selection hyper-heuristics

A square matrix of distinct numbers in which every row, column and both diagonals have the same total is referred to as a magic square. Constructing a magic square of a given order is considered a difficult computational problem, particularly when additional constraints are imposed. Hyper-heuristics are emerging high-level search methodologies that explore the space of heuristics for solving a given problem. In this study, we present a range of effective selection hyper-heuristics mixing perturbative low-level heuristics for constructing the constrained version of magic squares. The results show that selection hyper-heuristics, even the non-learning ones deliver an outstanding performance, beating the best-known heuristic solution on average

Heterogeneous Heuristic Optimisation and Scheduling for First-Order Theorem Proving

Good heuristics are essential for successful proof search in first-order automated theorem proving. As a result, state-of-the-art theorem provers offer a range of options for tuning the proof search process to specific problems. However, the vast configuration space makes it exceedingly challenging to construct effective heuristics. In this paper we present a new approach called HOS-ML, for automatically discovering new heuristics and mapping problems into optimised local schedules comprising of these heuristics. Our approach is based on interleaving Bayesian hyper-parameter optimisation for discovering promising heuristics and dynamic clustering to make optimisation efficient on heterogeneous problems. HOS-ML also use constraint programming to devise locally optimal schedules and machine learning for mapping unseen problems into such schedules. We evaluated HOS-ML on the theorem prover iProver and demonstrated that it can discover new heuristics that considerably improve performance and can solve problems that have not been solved previously by any other system.<br/

Decompositions of Grammar Constraints

A wide range of constraints can be compactly specified using automata or formal languages. In a sequence of recent papers, we have shown that an effective means to reason with such specifications is to decompose them into primitive constraints. We can then, for instance, use state of the art SAT solvers and profit from their advanced features like fast unit propagation, clause learning, and conflict-based search heuristics. This approach holds promise for solving combinatorial problems in scheduling, rostering, and configuration, as well as problems in more diverse areas like bioinformatics, software testing and natural language processing. In addition, decomposition may be an effective method to propagate other global constraints.Comment: Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligenc

Automated design of population-based algorithms: a case study in vehicle routing

Metaheuristics have been extensively studied to solve constraint combinatorial optimisation problems such as vehicle routing problems. Most existing algorithms require considerable human effort and different kinds of expertise in algorithm design. These manually designed algorithms are discarded after solving the specific instances. It is highly desirable to automate the design of search algorithms, thus to solve problem instances effectively with less human intervention. This thesis develops a novel general search framework to formulate in a unified way a range of population-based algorithms. Within this framework, generic algorithmic components such as selection heuristics on the population and evolution operators are defined, and can be composed using machine learning to generate effective search algorithms automatically. This unified framework aims to serve as the basis to analyse algorithmic components, generating effective search algorithms for complex combinatorial optimisation problems. Three key research issues within the general search framework are identified: automated design of evolution operators, of selection heuristics, and of both. To accurately describe the search space of algorithm design as a new task for machine learning, this thesis identifies new key features, namely search-dependent and instance-dependent features. These features are identified to assist effective algorithm design. With these features, a set of state-of-the-art reinforcement learning techniques, such as deep Q-network based and proximal policy optimisation based models and maximum entropy mechanisms have been developed to intelligently select and combine appropriate evolution operators and selection heuristics during different stages of the optimisation process. The effectiveness and generality of these algorithms automatically designed within the proposed general search framework are validated comprehensively across different capacitated vehicle routing problem with time windows benchmark instances. This thesis contributes to making a key step towards automated algorithm design with a general framework supporting fundamental analysis by effective machine learning

Effective learning hyper-heuristics for the course timetabling problem

Course timetabling is an important and recurring administrative activity in most educational institutions. This article combines a general modeling methodology with effective learning hyper-heuristics to solve this problem. The proposed hyper-heuristics are based on an iterated local search procedure that autonomously combines a set of move operators. Two types of learning for operator selection are contrasted: a static (offline) approach, with a clear distinction between training and execution phases; and a dynamic approach that learns on the fly. The resulting algorithms are tested over the set of real-world instances collected by the first and second International Timetabling competitions. The dynamic scheme statistically outperforms the static counterpart, and produces competitive results when compared to the state-of-the-art, even producing a new best-known solution. Importantly, our study illustrates that algorithms with increased autonomy and generality can outperform human designed problem-specific algorithms