Bandit Algorithms for Tree Search

Bandit based methods for tree search have recently gained popularity when applied to huge trees, e.g. in the game of go (Gelly et al., 2006). The UCT algorithm (Kocsis and Szepesvari, 2006), a tree search method based on Upper Confidence Bounds (UCB) (Auer et al., 2002), is believed to adapt locally to the effective smoothness of the tree. However, we show that UCT is too ``optimistic'' in some cases, leading to a regret O(exp(exp(D))) where D is the depth of the tree. We propose alternative bandit algorithms for tree search. First, a modification of UCT using a confidence sequence that scales exponentially with the horizon depth is proven to have a regret O(2^D \sqrt{n}), but does not adapt to possible smoothness in the tree. We then analyze Flat-UCB performed on the leaves and provide a finite regret bound with high probability. Then, we introduce a UCB-based Bandit Algorithm for Smooth Trees which takes into account actual smoothness of the rewards for performing efficient ``cuts'' of sub-optimal branches with high confidence. Finally, we present an incremental tree search version which applies when the full tree is too big (possibly infinite) to be entirely represented and show that with high probability, essentially only the optimal branches is indefinitely developed. We illustrate these methods on a global optimization problem of a Lipschitz function, given noisy data

On quantum vs. classical probability

Quantum theory shares with classical probability theory many important properties. I show that this common core regards at least the following six areas, and I provide details on each of these: the logic of propositions, symmetry, probabilities, composition of systems, state preparation and reductionism. The essential distinction between classical and quantum theory, on the other hand, is shown to be joint decidability versus smoothness; for the latter in particular I supply ample explanation and motivation. Finally, I argue that beyond quantum theory there are no other generalisations of classical probability theory that are relevant to physics.Comment: Major revision: key results unchanged, but derivation and discussion completely rewritten; 33 pages, no figure

Minimax estimation of smooth optimal transport maps

Brenier's theorem is a cornerstone of optimal transport that guarantees the existence of an optimal transport map $T$ between two probability distributions $P$ and $Q$ over $\mathbb{R}^d$ under certain regularity conditions. The main goal of this work is to establish the minimax estimation rates for such a transport map from data sampled from $P$ and $Q$ under additional smoothness assumptions on $T$. To achieve this goal, we develop an estimator based on the minimization of an empirical version of the semi-dual optimal transport problem, restricted to truncated wavelet expansions. This estimator is shown to achieve near minimax optimality using new stability arguments for the semi-dual and a complementary minimax lower bound. Furthermore, we provide numerical experiments on synthetic data supporting our theoretical findings and highlighting the practical benefits of smoothness regularization. These are the first minimax estimation rates for transport maps in general dimension.Comment: 53 pages, 6 figure

Random rewards, fractional Brownian local times and stable self-similar processes

We describe a new class of self-similar symmetric $\alpha$-stable processes with stationary increments arising as a large time scale limit in a situation where many users are earning random rewards or incurring random costs. The resulting models are different from the ones studied earlier both in their memory properties and smoothness of the sample paths.Comment: Published at http://dx.doi.org/10.1214/105051606000000277 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org