Efficient-UCBV: An Almost Optimal Algorithm using Variance Estimates

We propose a novel variant of the UCB algorithm (referred to as Efficient-UCB-Variance (EUCBV)) for minimizing cumulative regret in the stochastic multi-armed bandit (MAB) setting. EUCBV incorporates the arm elimination strategy proposed in UCB-Improved \citep{auer2010ucb}, while taking into account the variance estimates to compute the arms' confidence bounds, similar to UCBV \citep{audibert2009exploration}. Through a theoretical analysis we establish that EUCBV incurs a \emph{gap-dependent} regret bound of {\scriptsize $O\left( \dfrac{K\sigma^2_{\max} \log (T\Delta^2 /K)}{\Delta}\right)$} after $T$ trials, where $\Delta$ is the minimal gap between optimal and sub-optimal arms; the above bound is an improvement over that of existing state-of-the-art UCB algorithms (such as UCB1, UCB-Improved, UCBV, MOSS). Further, EUCBV incurs a \emph{gap-independent} regret bound of {\scriptsize $O\left(\sqrt{KT}\right)$} which is an improvement over that of UCB1, UCBV and UCB-Improved, while being comparable with that of MOSS and OCUCB. Through an extensive numerical study we show that EUCBV significantly outperforms the popular UCB variants (like MOSS, OCUCB, etc.) as well as Thompson sampling and Bayes-UCB algorithms

Exactly solvable Kitaev model in three dimensions

We introduce a spin-1/2 model in three dimensions which is a generalization of the well-known Kitaev model on a honeycomb lattice. Following Kitaev, we solve the model exactly by mapping it to a theory of non-interacting fermions in the background of a static Z_2 gauge field. The phase diagram consists of a gapped phase and a gapless one, similar to the two-dimensional case. Interestingly, unlike in the two-dimensional model, in the gapless phase the gap vanishes on a contour in the k space. Furthermore, we show that the flux excitations of the gauge field, due to some local constraints, form loop like structures; such loops exist on a lattice formed by the plaquettes in the original lattice and is topologically equivalent to the pyrochlore lattice. Finally, we derive a low-energy effective Hamiltonian that can be used to study the properties of the excitations in the gapped phase.Comment: 9 pages, 7 figures; published version; a new section and more references adde