Solving Imperfect Information Games Using Decomposition

Decomposition, i.e. independently analyzing possible subgames, has proven to be an essential principle for effective decision-making in perfect information games. However, in imperfect information games, decomposition has proven to be problematic. To date, all proposed techniques for decomposition in imperfect information games have abandoned theoretical guarantees. This work presents the first technique for decomposing an imperfect information game into subgames that can be solved independently, while retaining optimality guarantees on the full-game solution. We can use this technique to construct theoretically justified algorithms that make better use of information available at run-time, overcome memory or disk limitations at run-time, or make a time/space trade-off to overcome memory or disk limitations while solving a game. In particular, we present an algorithm for subgame solving which guarantees performance in the whole game, in contrast to existing methods which may have unbounded error. In addition, we present an offline game solving algorithm, CFR-D, which can produce a Nash equilibrium for a game that is larger than available storage.Comment: 7 pages by 2 columns, 5 figures; April 21 2014 - expand explanations and theor

Preliminary results of the heavy-light meson spectrum using chirally improved light quarks

Using a ``wall'' of quark point sources, we invert the chirally improved Dirac operator to create an ``incoherent'' collection of quark propagators which originate from all spatial points of the source time slice. The lowest-order NRQCD approximation is used to create heavy-quark propagators from the same wall source. However, since the numerical cost involved in computing such heavy-quark propagators is low, we are able to use a number of source gauge paths to establish coherence between the heavy and light quarks at several spatial separations. The resulting collection of heavy-light meson correlators is analyzed to extract the corresponding mass spectrum.Comment: 3 pages, 1 figure, Lattice2004(spectrum), minor corrections adde

Domain decomposition improvement of quark propagator estimation

Applying domain decomposition to the lattice Dirac operator and the associated quark propagator, we arrive at expressions which, with the proper insertion of random sources therein, can provide improvement to the estimation of the propagator. Schemes are presented for both open and closed (or loop) propagators. In the end, our technique for improving open contributions is similar to the ``maximal variance reduction'' approach of Michael and Peisa, but contains the advantage, especially for improved actions, of dealing directly with the Dirac operator. Using these improved open propagators for the Chirally Improved operator, we present preliminary results for the static-light meson spectrum. The improvement of closed propagators is modest: on some configurations there are signs of significant noise reduction of disconnected correlators; on others, the improvement amounts to a smoothening of the same correlators.Comment: 19 pages, 8 figures, version to appear in Computer Physics Communication

Isolating the Roper Resonance in Lattice QCD

We present results for the first positive parity excited state of the nucleon, namely, the Roper resonance ($N^{{{1/2}}^{+}}$=1440 MeV) from a variational analysis technique. The analysis is performed for pion masses as low as 224 MeV in quenched QCD with the FLIC fermion action. A wide variety of smeared-smeared correlation functions are used to construct correlation matrices. This is done in order to find a suitable basis of operators for the variational analysis such that eigenstates of the QCD Hamiltonian may be isolated. A lower lying Roper state is observed that approaches the physical Roper state. To the best of our knowledge, the first time this state has been identified at light quark masses using a variational approach.Comment: 7pp, 4 figures; minor typos corrected and one Ref. adde

No-Regret Learning in Extensive-Form Games with Imperfect Recall

Counterfactual Regret Minimization (CFR) is an efficient no-regret learning algorithm for decision problems modeled as extensive games. CFR's regret bounds depend on the requirement of perfect recall: players always remember information that was revealed to them and the order in which it was revealed. In games without perfect recall, however, CFR's guarantees do not apply. In this paper, we present the first regret bound for CFR when applied to a general class of games with imperfect recall. In addition, we show that CFR applied to any abstraction belonging to our general class results in a regret bound not just for the abstract game, but for the full game as well. We verify our theory and show how imperfect recall can be used to trade a small increase in regret for a significant reduction in memory in three domains: die-roll poker, phantom tic-tac-toe, and Bluff.Comment: 21 pages, 4 figures, expanded version of article to appear in Proceedings of the Twenty-Ninth International Conference on Machine Learnin

Variance Reduction in Monte Carlo Counterfactual Regret Minimization (VR-MCCFR) for Extensive Form Games using Baselines

Learning strategies for imperfect information games from samples of interaction is a challenging problem. A common method for this setting, Monte Carlo Counterfactual Regret Minimization (MCCFR), can have slow long-term convergence rates due to high variance. In this paper, we introduce a variance reduction technique (VR-MCCFR) that applies to any sampling variant of MCCFR. Using this technique, per-iteration estimated values and updates are reformulated as a function of sampled values and state-action baselines, similar to their use in policy gradient reinforcement learning. The new formulation allows estimates to be bootstrapped from other estimates within the same episode, propagating the benefits of baselines along the sampled trajectory; the estimates remain unbiased even when bootstrapping from other estimates. Finally, we show that given a perfect baseline, the variance of the value estimates can be reduced to zero. Experimental evaluation shows that VR-MCCFR brings an order of magnitude speedup, while the empirical variance decreases by three orders of magnitude. The decreased variance allows for the first time CFR+ to be used with sampling, increasing the speedup to two orders of magnitude

Masses of excited baryons from chirally improved quenched lattice QCD

Whereas ground state spectroscopy for quenched QCD is well understood, it is still a challenge to obtain results for excited hadron states. In our study we present results from a new approach for determining spatially optimized operators for lattice spectroscopy of excited hadrons. In order to be able to approach physical quark masses we work with the chirally improved Dirac operator, i.e., approximate Ginsparg-Wilson fermions. Since these are computationally expensive we restrict ourselves to a few quark sources. We use Jacobi smeared quark sources with different widths and combine them to construct hadron operators with different spatial wave functions. This allows us to identify the Roper state and other excited baryons, also in the strange sector.Comment: Contribution to BARYONS 2004, Palaiseau, France, October 25 - 29, 2004; 4 pages, 1 figure, Style espcrc

Excited hadrons from improved interpolating fields

The calculation of quark propagators for Ginsparg-Wilson-type Dirac operators is costly and thus limited to a few different sources. We present a new approach for determining spatially optimized operators for lattice spectroscopy of excited hadrons. Jacobi smeared quark sources with different widths are combined to construct hadron operators with different spatial wave functions. We study the Roper state and excited rho and pion mesons.Comment: Lattice2004(spectrum), 3 pages, 1 figure, (LaTeX style file espcrc2.sty and AMS style files