Assessing the Potential of Classical Q-learning in General Game Playing

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee $\&$ Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex)\footnote{source code: https://github.com/wh1992v/ggp-rl}, to allow comparison to Banerjee et al.. We find that Q-learning converges to a high win rate in GGP. For the $\epsilon$-greedy strategy, we propose a first enhancement, the dynamic $\epsilon$ algorithm. In addition, inspired by (Gelly $\&$ Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.Comment: arXiv admin note: substantial text overlap with arXiv:1802.0594

Formation and decay of electron-hole droplets in diamond

We study the formation and decay of electron-hole droplets in diamonds at both low and high temperatures under different excitations by master equations. The calculation reveals that at low temperature the kinetics of the system behaves as in direct-gap semiconductors, whereas at high temperature it shows metastability as in traditional indirect-gap semiconductors. Our results at low temperature are consistent with the experimental findings by Nagai {\em et al.} [Phys. Rev. B {\bf 68}, 081202 (R) (2003)]. The kinetics of the e-h system in diamonds at high temperature under both low and high excitations is also predicted.Comment: 7 pages, 8 figures, revised with some modifications in physics discussion, to be published in PR

Alternate wet/dry irrigation in rice cultivation: a practical way to save water and control malaria and Japanese encephalitis?

Water management / Water scarcity / Water use efficiency / Water conservation / Irrigated farming / Waterborne diseases / Rice / Malaria / Disease vectors / Productivity / Flood irrigation / Environmental control / Climate / China / East Africa / India / Indonesia / Japan / Philippines / Portugal / USA

Magnetic Properties of the Second Mott Lobe in Pairing Hamiltonians

We explore the Mott insulating state of single-band bosonic pairing Hamiltonians using analytical approaches and large scale density matrix renormalization group calculations. We focus on the second Mott lobe which exhibits a magnetic quantum phase transition in the Ising universality class. We use this feature to discuss the behavior of a range of physical observables within the framework of the 1D quantum Ising model and the strongly anisotropic Heisenberg model. This includes the properties of local expectation values and correlation functions both at and away from criticality. Depending on the microscopic interactions it is possible to achieve either antiferromagnetic or ferromagnetic exchange interactions and we highlight the possibility of observing the E8 mass spectrum for the critical Ising model in a longitudinal magnetic field.Comment: 14 pages, 15 figure

The Starburst in the Central Kiloparsec of Markarian 231

We present VLBA observations at 0.33 and 0.61 GHz, and VLA observations between 5 and 22 GHz, of subkiloparsec scale radio emission from Mrk 231. In addition to jet components clearly associated with the AGN, we also find a smooth extended component of size 100 - 1000 pc most probably related to the purported massive star forming disk in Mrk 231. The diffuse radio emission from the disk is found to have a steep spectrum at high frequencies, characteristic of optically thin synchrotron emission. The required relativistic particle density in the disk can be produced by a star formation rate of 220 Msolar/yr in the central kiloparsec. At low frequencies the disk is absorbed, most likely by ionized gas with an emission measure of 8 x 10^5 pc cm-6. We have also identified 4 candidate radio supernovae that, if confirmed, represent direct evidence for ongoing star formation in the central kiloparsec.Comment: in press at ApJ for v. 519 July 1999, 14 page LaTeX document includes 6 postscript figure

Research and education in management of large-scale technical programs

A research effort is reported which was conducted by NASA in conjunction with Drexel University, and which was aimed at an improved understanding of large scale systems technology and management

Description of recent large-qq neutron inclusive scattering data from liquid 4^4He

We report dynamical calculations for large-$q$ structure functions of liquid $^4$He at $T$=1.6 and 2.3 K and compare those with recent MARI data. We extend those calculations far beyond the experimental range q\le 29\Ain in order to study the approach of the response to its asymptotic limit for a system with interactions having a strong short-range repulsion. We find only small deviations from theoretical $1/q$ behavior, valid for smooth $V$. We repeat an extraction by Glyde et al of cumulant coefficients from data. We argue that fits determine the single atom momentum distribution, but express doubt as to the extraction of meaningful Final State Interaction parameters.Comment: 37 pages, 13 postscript fig

Competition Between Antiferromagnetic Order and Spin-Liquid Behavior in the Two-Dimensional Periodic Anderson Model at Half-Filling

We study the two-dimensional periodic Anderson model at half-filling using quantum Monte Carlo (QMC) techniques. The ground state undergoes a magnetic order-disorder transition as a function of the effective exchange coupling between the conduction and localized bands. Low-lying spin and charge excitations are determined using the maximum entropy method to analytically continue the QMC data. At finite temperature we find a competition between the Kondo effect and antiferromagnetic order which develops in the localized band through Ruderman-Kittel-Kasuya-Yosida interactions.Comment: Revtex 3.0, 10 pages + 5 figures, UCSBTH-94-2

An inquiry-based learning approach to teaching information retrieval

The study of information retrieval (IR) has increased in interest and importance with the explosive growth of online information in recent years. Learning about IR within formal courses of study enables users of search engines to use them more knowledgeably and effectively, while providing the starting point for the explorations of new researchers into novel search technologies. Although IR can be taught in a traditional manner of formal classroom instruction with students being led through the details of the subject and expected to reproduce this in assessment, the nature of IR as a topic makes it an ideal subject for inquiry-based learning approaches to teaching. In an inquiry-based learning approach students are introduced to the principles of a subject and then encouraged to develop their understanding by solving structured or open problems. Working through solutions in subsequent class discussions enables students to appreciate the availability of alternative solutions as proposed by their classmates. Following this approach students not only learn the details of IR techniques, but significantly, naturally learn to apply them in solution of problems. In doing this they not only gain an appreciation of alternative solutions to a problem, but also how to assess their relative strengths and weaknesses. Developing confidence and skills in problem solving enables student assessment to be structured around solution of problems. Thus students can be assessed on the basis of their understanding and ability to apply techniques, rather simply their skill at reciting facts. This has the additional benefit of encouraging general problem solving skills which can be of benefit in other subjects. This approach to teaching IR was successfully implemented in an undergraduate module where students were assessed in a written examination exploring their knowledge and understanding of the principles of IR and their ability to apply them to solving problems, and a written assignment based on developing an individual research proposal