Assessing the Potential of Classical Q-learning in General Game Playing

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee $\&$ Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex)\footnote{source code: https://github.com/wh1992v/ggp-rl}, to allow comparison to Banerjee et al.. We find that Q-learning converges to a high win rate in GGP. For the $\epsilon$-greedy strategy, we propose a first enhancement, the dynamic $\epsilon$ algorithm. In addition, inspired by (Gelly $\&$ Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.Comment: arXiv admin note: substantial text overlap with arXiv:1802.0594

Jugular venous reflux and white matter abnormalities in Alzheimer's disease: a pilot study.

To determine whether jugular venous reflux (JVR) is associated with cerebral white matter changes (WMCs) in individuals with Alzheimer's disease (AD), we studied 12 AD patients 24 mild cognitive impairment (MCI) patients, and 17 elderly age- and gender-matched controls. Duplex ultrasonography and 1.5T MRI scanning was applied to quantify cerebral WMCs [T2 white matter (WM) lesion and dirty-appearing-white-matter (DAWM)]. Subjects with severe JVR had more frequently hypertension (p = 0.044), more severe WMC, including increased total (p = 0.047) and periventricular DAWM volumes (p = 0.008), and a trend for increased cerebrospinal fluid volumes (p = 0.067) compared with the other groups. A significantly decreased (65.8%) periventricular DAWM volume (p = 0.01) in the JVR-positive AD individuals compared with their JVR-negative counterparts was detected. There was a trend for increased periventricular and subcortical T2 WMC lesion volumes in the JVR-positive AD individuals compared with their JVR-negative counterparts (p = 0.073). This phenomenon was not observed in either the control or MCI groups. In multiple regression analysis, the increased periventricular WMC lesion volume and decreased DAWM volume resulted in 85.7% sensitivity and 80% specificity for distinguishing between JVR-positive and JVR-negative AD patients. These JVR-WMC association patterns were not seen in the control and MCI groups. Therefore, this pilot study suggests that there may be an association between JVR and WMCs in AD patients, implying that cerebral venous outflow impairment might play a role in the dynamics of WMCs formation in AD patients, particularly in the periventricular regions. Further longitudinal studies are needed to confirm and validate our findings

MapRepeat: an approach for effective assembly of repetitive regions in prokaryotic genomes

The newest technologies for DNA sequencing have led to the determination of the primary structure of the genomes of organisms, mainly prokaryotes, with high efficiency and at lower costs. However, the presence of regions with repetitive sequences, in addition to the short reads produced by the Next-Generation Sequencing (NGS) platforms, created a lot of difficulty in reconstructing the original genome in silico. Thus, even today, genome assembly continues to be one of the major challenges in bioinformatics specifically when repetitive sequences are considered. In this paper, we present an approach to assemble repetitive regions in prokaryotic genomes. Our methodology enables (i) the identification of these regions through visual tools, (ii) the characterization of sequences on the extremities of gaps and (iii) the extraction of consensus sequences based on mapping of raw data to a reference genome. We also present a case study on the assembly of regions that encode ribosomal RNAs (rRNA) in the genome of Corynebacterium ulceransFRC11, in order to show the efficiency of the strategies presented here. The proposed methods and tools will help in finishing genome assemblies, besides reducing the running time and associated costs. Availability All scripts are available at http://github.com/dcbmariano/maprepea

Bilateral synchronous occurrence of three different histological types of renal tumor: a case report

This is an Open Access article distributed under the terms of the Creative Commons Attribution Licens

Phylogenetic relationships within Chrysogorgia (Alcyonacea: Octocorallia), a morphologically diverse genus of octocoral, revealed using a target enrichment approach

The octocoral genus Chrysogorgia (Duchassaing and Michelotti, 1864) contains 81 nominal species that are ecologically important components of benthic communities. Taxonomic examination of a large set of samples revealed many provisional new species, exhibiting a wide range of morphological variation. We established nine, distinct morphological groups of Chrysogorgia s.l. that were hypothesized to represent distinct genera. Here, we applied a recently developed universal target enrichment bait method for octocoral exons and ultraconserved elements (UCEs) on 96 specimens varying in morphology, collection ages and DNA quality and quantity to determine whether there was genetic support for these morphologically defined groups. Following Illumina sequencing and SPAdes assembly we recovered 1,682 of 1,700 targeted exon loci and 1,333 of 1,340 targeted UCE loci. Locus recovery per sample was highly variable and significantly correlated with time since specimen collection (2-60 years) and DNA quantity and quality. Phylogenetically informative sites in UCE and exon loci were ∼35% for 50% and 75% taxon-occupancy matrices. Maximum likelihood analyses recovered highly resolved trees with topologies supporting the recognition of 11 candidate genera, corresponding with morphological groups assigned a priori, nine of which are novel. Our results also demonstrate that this target-enrichment approach can be successfully applied to degraded museum specimens of up to 60 years old. This study shows that an integrative approach consisting of molecular and morphological methods will be essential to a proper revision of Chrysogorgia taxonomy and to understand regional diversity of these ecologically important corals

High-yield methods for accurate two-alternative visual psychophysics in head-fixed mice

Research in neuroscience relies increasingly on the mouse, a mammalian species that affords unparalleled genetic tractability and brain atlases. Here we introduce high-yield methods for probing mouse visual decisions. Mice are head-fixed, which facilitates repeatable visual stimulation, eye tracking, and brain access. They turn a steering wheel to make two-alternative choices, forced or unforced. Learning is rapid thanks to intuitive coupling of stimuli to wheel position. The mouse decisions deliver high-quality psychometric curves for detection and discrimination, and conform to the predictions of a simple probabilistic observer model. The task is readily paired with two-photon imaging of cortical activity. Optogenetic inactivation reveals that the task requires the visual cortex. Mice are motivated to perform the task by fluid reward or optogenetic stimulation of dopaminergic neurons. This stimulation elicits larger number of trials and faster learning. These methods provide a platform to accurately probe mouse vision and its neural basis

Enhancement of the high-field critical current density of superconducting MgB2 by proton irradiation

A relatively high critical temperature, Tc, approaching 40 K, places the recently-discovered superconductor magnesium diboride (MgB2) intermediate between the families of low- and copper-oxide-based high-temperature superconductors (HTS). Supercurrent flow in MgB2 is unhindered by grain boundaries, unlike the HTS materials. Thus, long polycrystalline MgB2 conductors may be easier to fabricate, and so could fill a potentially important niche of applications in the 20 to 30 K temperature range. However, one disadvantage of MgB2 is that in bulk material the critical current density, Jc, appears to drop more rapidly with increasing magnetic field than it does in the HTS phases. The magnitude and field dependence of Jc are related to the presence of structural defects that can "pin" the quantised magnetic vortices that permeate the material, and prevent them from moving under the action of the Lorentz force. Vortex studies suggest that it is the paucity of suitable defects in MgB2 that causes the rapid decay of Jc with field. Here we show that modest levels of atomic disorder, induced by proton irradiation, enhance the pinning, and so increase Jc significantly at high fields. We anticipate that chemical doping or mechanical processing should be capable of generating similar levels of disorder, and so achieve technologically-attractive performance in MgB2 by economically-viable routes.Comment: 5 pages, 4 figures, to be published in Nature (in press