Learning Policies from Self-Play with Policy Gradients and MCTS Value Estimates

In recent years, state-of-the-art game-playing agents often involve policies that are trained in self-playing processes where Monte Carlo tree search (MCTS) algorithms and trained policies iteratively improve each other. The strongest results have been obtained when policies are trained to mimic the search behaviour of MCTS by minimising a cross-entropy loss. Because MCTS, by design, includes an element of exploration, policies trained in this manner are also likely to exhibit a similar extent of exploration. In this paper, we are interested in learning policies for a project with future goals including the extraction of interpretable strategies, rather than state-of-the-art game-playing performance. For these goals, we argue that such an extent of exploration is undesirable, and we propose a novel objective function for training policies that are not exploratory. We derive a policy gradient expression for maximising this objective function, which can be estimated using MCTS value estimates, rather than MCTS visit counts. We empirically evaluate various properties of resulting policies, in a variety of board games.Comment: Accepted at the IEEE Conference on Games (CoG) 201

Ludii -- The Ludemic General Game System

While current General Game Playing (GGP) systems facilitate useful research in Artificial Intelligence (AI) for game-playing, they are often somewhat specialised and computationally inefficient. In this paper, we describe the "ludemic" general game system Ludii, which has the potential to provide an efficient tool for AI researchers as well as game designers, historians, educators and practitioners in related fields. Ludii defines games as structures of ludemes -- high-level, easily understandable game concepts -- which allows for concise and human-understandable game descriptions. We formally describe Ludii and outline its main benefits: generality, extensibility, understandability and efficiency. Experimentally, Ludii outperforms one of the most efficient Game Description Language (GDL) reasoners, based on a propositional network, in all games available in the Tiltyard GGP repository. Moreover, Ludii is also competitive in terms of performance with the more recently proposed Regular Boardgames (RBG) system, and has various advantages in qualitative aspects such as generality.Comment: Accepted at ECAI 202

Preconception nutrition: building advocacy and social movements to stimulate action

Action to improve preconception nutrition is a collective, societal responsibility. We believe that the Developmental Origins of Health and Disease (DOHaD) society is ideally placed to facilitate the development of a global agenda for preconception nutrition which recognises the societal importance of nutrition for young women and men, and supports them in optimising their nutritional status for the benefit of the next generation. In this paper, we outline four key actions that can be taken by the members of DOHaD's international society located across 67 countries, and nine regional societies, to demonstrate this leadership role. The recommended actions to place preconception nutrition at the top of national and regional agendas include (i) continuing to build the scientific evidence, (ii) monitoring of progress made by governments and commercial companies, (iii) developing advocacy coalitions that unite individuals and organisations around common policy options and (iv) working with partners to develop an emotive and empowering preconception nutrition awareness campaign. Collectively, these actions hold the potential to develop into a preconception nutrition social movement to invoke high-level government support and across-sector policy action, while raising public demand for action and engaging corporate actors

Density Matrix Renormalization Group Study of the Disorder Line in the Quantum ANNNI Model

We apply Density Matrix Renormalization Group methods to study the phase diagram of the quantum ANNNI model in the region of low frustration where the ferromagnetic coupling is larger than the next-nearest-neighbor antiferromagnetic one. By Finite Size Scaling on lattices with up to 80 sites we locate precisely the transition line from the ferromagnetic phase to a paramagnetic phase without spatial modulation. We then measure and analyze the spin-spin correlation function in order to determine the disorder transition line where a modulation appears. We give strong numerical support to the conjecture that the Peschel-Emery one-dimensional line actually coincides with the disorder line. We also show that the critical exponent governing the vanishing of the modulation parameter at the disorder transition is $\beta_q = 1/2$.Comment: 4 pages, 5 eps figure

Frustrated quantum Heisenberg ferrimagnetic chains

We study the ground-state properties of weakly frustrated Heisenberg ferrimagnetic chains with nearest and next-nearest neighbor antiferromagnetic exchange interactions and two types of alternating sublattice spins S_1 > S_2, using 1/S spin-wave expansions, density-matrix renormalization group, and exact- diagonalization techniques. It is argued that the zero-point spin fluctuations completely destroy the classical commensurate- incommensurate continuous transition. Instead, the long-range ferrimagnetic state disappears through a discontinuous transition to a singlet state at a larger value of the frustration parameter. In the ferrimagnetic phase we find a disorder point marking the onset of incommensurate real-space short-range spin-spin correlations.Comment: 16 pages (LaTex 2.09), 6 eps figure

Non-equilibrium Relaxation Study of Ferromagnetic Transition in Double-Exchange Systems

Ferromagnetic transition in double-exchange systems is studied by non-equilibrium relaxation technique combined with Monte Carlo calculations. Critical temperature and critical exponents are estimated from relaxation of the magnetic moment. The results are consistent with the previous Monte Carlo results in thermal equilibrium. The exponents estimated by these independent techniques suggest that the universality class of this transition is the same as that of short-range interaction models but is different from the mean-field one.Comment: 3 pages including 1 figure, submitted to J. Phys. Soc. Jp

A Model for the Analysis of Caries Occurrence in Primary Molar Tooth Surfaces

Recently methods of caries quantification in the primary dentition have moved away from summary ‘whole mouth’ measures at the individual level to methods based on generalised linear modelling (GLM) approaches or survival analysis approaches. However, GLM approaches based on logistic transformation fail to take into account the time-dependent process of tooth/surface survival to caries. There may also be practical difficulties associated with casting parametric survival-based approaches in a complex multilevel hierarchy and the selection of an optimal survival distribution, while non-parametric survival methods are not generally suitable for the assessment of supplementary information recorded on study participants. In the current investigation, a hybrid semi-parametric approach comprising elements of survival-based and GLM methodologies suitable for modelling of caries occurrence within fixed time periods is assessed, using an illustrative multilevel data set of caries occurrence in primary molars from a cohort study, with clustering of data assumed to occur at surface and tooth levels. Inferences of parameter significance were found to be consistent with previous parametric survival-based analyses of the same data set, with gender, socio-economic status, fluoridation status, tooth location, surface type and fluoridation status-surface type interaction significantly associated with caries occurrence. The appropriateness of the hierarchical structure facilitated by the hybrid approach was also confirmed. Hence the hybrid approach is proposed as a more appropriate alternative to primary caries modelling than non-parametric survival methods or other GLM-based models, and as a practical alternative to more rigorous survival-based methods unlikely to be fully accessible to most researchers

Comparing the effectiveness of polymer debriding devices using a porcine wound biofilm model

Objective: Debridement to remove necrotic and/or infected tissue and promote active healing remains a cornerstone of contemporary chronic wound management. While there has been a recent shift toward less invasive polymer-based debriding devices, their efficacy requires rigorous evaluation.Approach: This study was designed to directly compare monofilament debriding devices to traditional gauze using a wounded porcine skin biofilm model with standardized application parameters. Biofilm removal was determined using a surface viability assay, bacterial counts, histological assessment, and scanning electron microscopy (SEM).Results: Quantitative analysis revealed that monofilament debriding devices outperformed the standard gauze, resulting in up to 100-fold greater reduction in bacterial counts. Interestingly, histological and morphological analyses suggested that debridement not only removed bacteria, but also differentially disrupted the bacterially-derived extracellular polymeric substance. Finally, SEM of post-debridement monofilaments showed structural changes in attached bacteria, implying a negative impact on viability.Innovation: This is the first study to combine controlled and defined debridement application with a biologically relevant ex vivo biofilm model to directly compare monofilament debriding devices.Conclusion: These data support the use of monofilament debriding devices for the removal of established wound biofilms and suggest variable efficacy towards biofilms composed of different species of bacteria