2,934 research outputs found
Making Change in 2048
The 2048 game involves tiles labeled with powers of two that can be merged to form bigger powers of two; variants of the same puzzle involve similar merges of other tile values. We analyze the maximum score achievable in these games by proving a min-max theorem equating this maximum score (in an abstract generalized variation of 2048 that allows all the moves of the original game) with the minimum value that causes a greedy change-making algorithm to use a given number of coins. A widely-followed strategy in 2048 maintains tiles that represent the move number in binary notation, and a similar strategy in the Fibonacci number variant of the game (987) maintains the Zeckendorf representation of the move number as a sum of the fewest possible Fibonacci numbers; our analysis shows that the ability to follow these strategies is intimately connected with the fact that greedy change-making is optimal for binary and Fibonacci coinage. For variants of 2048 using tile values for which greedy change-making is suboptimal, it is the greedy strategy, not the optimal representation as sums of tile values, that controls the length of the game. In particular, the game will always terminate whenever the sequence of allowable tile values has arbitrarily large gaps between consecutive values
Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search
Neural Architecture Search (NAS) has shown great success in automating the
design of neural networks, but the prohibitive amount of computations behind
current NAS methods requires further investigations in improving the sample
efficiency and the network evaluation cost to get better results in a shorter
time. In this paper, we present a novel scalable Monte Carlo Tree Search (MCTS)
based NAS agent, named AlphaX, to tackle these two aspects. AlphaX improves the
search efficiency by adaptively balancing the exploration and exploitation at
the state level, and by a Meta-Deep Neural Network (DNN) to predict network
accuracies for biasing the search toward a promising region. To amortize the
network evaluation cost, AlphaX accelerates MCTS rollouts with a distributed
design and reduces the number of epochs in evaluating a network by transfer
learning, which is guided with the tree structure in MCTS. In 12 GPU days and
1000 samples, AlphaX found an architecture that reaches 97.84\% top-1 accuracy
on CIFAR-10, and 75.5\% top-1 accuracy on ImageNet, exceeding SOTA NAS methods
in both the accuracy and sampling efficiency. Particularly, we also evaluate
AlphaX on NASBench-101, a large scale NAS dataset; AlphaX is 3x and 2.8x more
sample efficient than Random Search and Regularized Evolution in finding the
global optimum. Finally, we show the searched architecture improves a variety
of vision applications from Neural Style Transfer, to Image Captioning and
Object Detection.Comment: To appear in the Thirty-Fourth AAAI conference on Artificial
Intelligence (AAAI-2020
Learning to Search in Reinforcement Learning
In this thesis, we investigate the use of search based algorithms with deep neural
networks to tackle a wide range of problems ranging from board games to video
games and beyond. Drawing inspiration from AlphaGo, the first computer program
to achieve superhuman performance in the game of Go, we developed a new algorithm AlphaZero. AlphaZero is a general reinforcement learning algorithm that
combines deep neural networks with a Monte Carlo Tree search for planning and
learning. Starting completely from scratch, without any prior human knowledge
beyond the basic rules of the game, AlphaZero managed to achieve superhuman
performance in Go, chess and shogi. Subsequently, building upon the success of AlphaZero, we investigated ways to extend our methods to problems in which the rules
are not known or cannot be hand-coded. This line of work led to the development
of MuZero, a model-based reinforcement learning agent that builds a deterministic
internal model of the world and uses it to construct plans in its imagination. We
applied our method to Go, chess, shogi and the classic Atari suite of video-games,
achieving superhuman performance. MuZero is the first RL algorithm to master
a variety of both canonical challenges for high performance planning and visually complex problems using the same principles. Finally, we describe Stochastic
MuZero, a general agent that extends the applicability of MuZero to highly stochastic environments. We show that our method achieves superhuman performance in
stochastic domains such as backgammon and the classic game of 2048 while matching the performance of MuZero in deterministic ones like Go
- …