Assessing the Potential of Classical Q-learning in General Game Playing

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee $\&$ Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex)\footnote{source code: https://github.com/wh1992v/ggp-rl}, to allow comparison to Banerjee et al.. We find that Q-learning converges to a high win rate in GGP. For the $\epsilon$-greedy strategy, we propose a first enhancement, the dynamic $\epsilon$ algorithm. In addition, inspired by (Gelly $\&$ Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.Comment: arXiv admin note: substantial text overlap with arXiv:1802.0594

Self-gravitating elastic bodies

Extended objects in GR are often modelled using distributional solutions of the Einstein equations with point-like sources, or as the limit of infinitesimally small "test" objects. In this note, I will consider models of finite self-gravitating extended objects, which make it possible to give a rigorous treatment of the initial value problem for (finite) extended objects.Comment: 16 pages. Based on a talk given at the 2013 WE-Heraeus seminar on "Equations of motion in relativistic gravity

Elevated Paracellular Glucose Flux across Cystic Fibrosis Airway Epithelial Monolayers Is an Important Factor for Pseudomonas aeruginosa Growth.

People with cystic fibrosis (CF) who develop related diabetes (CFRD) have accelerated pulmonary decline, increased infection with antibiotic-resistant Pseudomonas aeruginosa and increased pulmonary exacerbations. We have previously shown that glucose concentrations are elevated in airway surface liquid (ASL) of people with CF, particularly in those with CFRD. We therefore explored the hypotheses that glucose homeostasis is altered in CF airway epithelia and that elevation of glucose flux into ASL drives increased bacterial growth, with an effect over and above other cystic fibrosis transmembrane conductance regulator (CFTR)-related ASL abnormalities. The aim of this study was to compare the mechanisms governing airway glucose homeostasis in CF and non-CF primary human bronchial epithelial (HBE) monolayers, under normal conditions and in the presence of Ps. aeruginosa filtrate. HBE-bacterial co-cultures were performed in the presence of 5 mM or 15 mM basolateral glucose to investigate how changes in blood glucose, such as those seen in CFRD, affects luminal Ps. aeruginosa growth. Calu-3 cell monolayers were used to evaluate the potential importance of glucose on Ps. aeruginosa growth, in comparison to other hallmarks of the CF ASL, namely mucus hyperviscosity and impaired CFTR-dependent fluid secretions. We show that elevation of basolateral glucose promotes the apical growth of Ps. aeruginosa on CF airway epithelial monolayers more than non-CF monolayers. Ps. aeruginosa secretions elicited more glucose flux across CF airway epithelial monolayers compared to non-CF monolayers which we propose increases glucose availability in ASL for bacterial growth. In addition, elevating basolateral glucose increased Ps. aeruginosa growth over and above any CFTR-dependent effects and the presence or absence of mucus in Calu-3 airway epithelia-bacteria co-cultures. Together these studies highlight the importance of glucose as an additional factor in promoting Ps. aeruginosa growth and respiratory infection in CF disease

Variable strength of forest stand attributes and weather conditions on the questing activity of Ixodes ricinus ticks over years in managed forests

Given the ever-increasing human impact through land use and climate change on the environment, we crucially need to achieve a better understanding of those factors that influence the questing activity of ixodid ticks, a major disease-transmitting vector in temperate forests. We investigated variation in the relative questing nymph densities of Ixodes ricinus in differently managed forest types for three years (2008–2010) in SW Germany by drag sampling. We used a hierarchical Bayesian modeling approach to examine the relative effects of habitat and weather and to consider possible nested structures of habitat and climate forces. The questing activity of nymphs was considerably larger in young forest successional stages of thicket compared with pole wood and timber stages. Questing nymph density increased markedly with milder winter temperatures. Generally, the relative strength of the various environmental forces on questing nymph density differed across years. In particular, winter temperature had a negative effect on tick activity across sites in 2008 in contrast to the overall effect of temperature across years. Our results suggest that forest management practices have important impacts on questing nymph density. Variable weather conditions, however, might override the effects of forest management practices on the fluctuations and dynamics of tick populations and activity over years, in particular, the preceding winter temperatures. Therefore, robust predictions and the detection of possible interactions and nested structures of habitat and climate forces can only be quantified through the collection of long-term data. Such data are particularly important with regard to future scenarios of forest management and climate warming

BCAA catabolism in brown fat controls energy homeostasis through SLC25A44.

Branched-chain amino acid (BCAA; valine, leucine and isoleucine) supplementation is often beneficial to energy expenditure; however, increased circulating levels of BCAA are linked to obesity and diabetes. The mechanisms of this paradox remain unclear. Here we report that, on cold exposure, brown adipose tissue (BAT) actively utilizes BCAA in the mitochondria for thermogenesis and promotes systemic BCAA clearance in mice and humans. In turn, a BAT-specific defect in BCAA catabolism attenuates systemic BCAA clearance, BAT fuel oxidation and thermogenesis, leading to diet-induced obesity and glucose intolerance. Mechanistically, active BCAA catabolism in BAT is mediated by SLC25A44, which transports BCAAs into mitochondria. Our results suggest that BAT serves as a key metabolic filter that controls BCAA clearance via SLC25A44, thereby contributing to the improvement of metabolic health

Protein disulfide-isomerase interacts with a substrate protein at all stages along its folding pathway

In contrast to molecular chaperones that couple protein folding to ATP hydrolysis, protein disulfide-isomerase (PDI) catalyzes protein folding coupled to formation of disulfide bonds (oxidative folding). However, we do not know how PDI distinguishes folded, partly-folded and unfolded protein substrates. As a model intermediate in an oxidative folding pathway, we prepared a two-disulfide mutant of basic pancreatic trypsin inhibitor (BPTI) and showed by NMR that it is partly-folded and highly dynamic. NMR studies show that it binds to PDI at the same site that binds peptide ligands, with rapid binding and dissociation kinetics; surface plasmon resonance shows its interaction with PDI has a Kd of ca. 10−5 M. For comparison, we characterized the interactions of PDI with native BPTI and fully-unfolded BPTI. Interestingly, PDI does bind native BPTI, but binding is quantitatively weaker than with partly-folded and unfolded BPTI. Hence PDI recognizes and binds substrates via permanently or transiently unfolded regions. This is the first study of PDI's interaction with a partly-folded protein, and the first to analyze this folding catalyst's changing interactions with substrates along an oxidative folding pathway. We have identified key features that make PDI an effective catalyst of oxidative protein folding – differential affinity, rapid ligand exchange and conformational flexibility

A randomised controlled trial and cost-effectiveness evaluation of "booster" interventions to sustain increases in physical activity in middle-aged adults in deprived urban neighbourhoods

Background: Systematic reviews have identified a range of brief interventions which increase physical activity in previously sedentary people. There is an absence of evidence about whether follow up beyond three months can maintain long term physical activity. This study assesses whether it is worth providing motivational interviews, three months after giving initial advice, to those who have become more active. Methods/Design: Study candidates (n = 1500) will initially be given an interactive DVD and receive two telephone follow ups at monthly intervals checking on receipt and use of the DVD. Only those that have increased their physical activity after three months (n = 600) will be randomised into the study. These participants will receive either a "mini booster" (n = 200), "full booster" (n = 200) or no booster (n = 200). The "mini booster" consists of two telephone calls one month apart to discuss physical activity and maintenance strategies. The "full booster" consists of a face-to-face meeting with the facilitator at the same intervals. The purpose of these booster sessions is to help the individual maintain their increase in physical activity. Differences in physical activity, quality of life and costs associated with the booster interventions, will be measured three and nine months from randomisation. The research will be conducted in 20 of the most deprived neighbourhoods in Sheffield, which have large, ethnically diverse populations, high levels of economic deprivation, low levels of physical activity, poorer health and shorter life expectancy. Participants will be recruited through general practices and community groups, as well as by postal invitation, to ensure the participation of minority ethnic groups and those with lower levels of literacy. Sheffield City Council and Primary Care Trust fund a range of facilities and activities to promote physical activity and variations in access to these between neighbourhoods will make it possible to examine whether the effectiveness of the intervention is modified by access to community facilities. A one-year integrated feasibility study will confirm that recruitment targets are achievable based on a 10% sample.Discussion: The choice of study population, study interventions, brief intervention preceding the study, and outcome measure are discussed

Adaptive Immunity against Leishmania Nucleoside Hydrolase Maps Its C-Terminal Domain as the Target of the CD4+ T Cell–Driven Protective Response

Nucleoside hydrolases (NHs) show homology among parasite protozoa, fungi and bacteria. They are vital protagonists in the establishment of early infection and, therefore, are excellent candidates for the pathogen recognition by adaptive immune responses. Immune protection against NHs would prevent disease at the early infection of several pathogens. We have identified the domain of the NH of L. donovani (NH36) responsible for its immunogenicity and protective efficacy against murine visceral leishmaniasis (VL). Using recombinant generated peptides covering the whole NH36 sequence and saponin we demonstrate that protection against L. chagasi is related to its C-terminal domain (amino-acids 199–314) and is mediated mainly by a CD4+ T cell driven response with a lower contribution of CD8+ T cells. Immunization with this peptide exceeds in 36.73±12.33% the protective response induced by the cognate NH36 protein. Increases in IgM, IgG2a, IgG1 and IgG2b antibodies, CD4+ T cell proportions, IFN-γ secretion, ratios of IFN-γ/IL-10 producing CD4+ and CD8+ T cells and percents of antibody binding inhibition by synthetic predicted epitopes were detected in F3 vaccinated mice. The increases in DTH and in ratios of TNFα/IL-10 CD4+ producing cells were however the strong correlates of protection which was confirmed by in vivo depletion with monoclonal antibodies, algorithm predicted CD4 and CD8 epitopes and a pronounced decrease in parasite load (90.5–88.23%; p = 0.011) that was long-lasting. No decrease in parasite load was detected after vaccination with the N-domain of NH36, in spite of the induction of IFN-γ/IL-10 expression by CD4+ T cells after challenge. Both peptides reduced the size of footpad lesions, but only the C-domain reduced the parasite load of mice challenged with L. amazonensis. The identification of the target of the immune response to NH36 represents a basis for the rationale development of a bivalent vaccine against leishmaniasis and for multivalent vaccines against NHs-dependent pathogens

Short Telomeres in Hatchling Snakes: Erythrocyte Telomere Dynamics and Longevity in Tropical Pythons

Telomere length (TL) has been found to be associated with life span in birds and humans. However, other studies have demonstrated that TL does not affect survival among old humans. Furthermore, replicative senescence has been shown to be induced by changes in the protected status of the telomeres rather than the loss of TL. In the present study we explore whether age- and sex-specific telomere dynamics affect life span in a long-lived snake, the water python (Liasis fuscus)

The Surgical Infection Society revised guidelines on the management of intra-abdominal infection

Background: Previous evidence-based guidelines on the management of intra-abdominal infection (IAI) were published by the Surgical Infection Society (SIS) in 1992, 2002, and 2010. At the time the most recent guideline was released, the plan was to update the guideline every five years to ensure the timeliness and appropriateness of the recommendations. Methods: Based on the previous guidelines, the task force outlined a number of topics related to the treatment of patients with IAI and then developed key questions on these various topics. All questions were approached using general and specific literature searches, focusing on articles and other information published since 2008. These publications and additional materials published before 2008 were reviewed by the task force as a whole or by individual subgroups as to relevance to individual questions. Recommendations were developed by a process of iterative consensus, with all task force members voting to accept or reject each recommendation. Grading was based on the GRADE (Grades of Recommendation Assessment, Development, and Evaluation) system; the quality of the evidence was graded as high, moderate, or weak, and the strength of the recommendation was graded as strong or weak. Review of the document was performed by members of the SIS who were not on the task force. After responses were made to all critiques, the document was approved as an official guideline of the SIS by the Executive Council. Results: This guideline summarizes the current recommendations developed by the task force on the treatment of patients who have IAI. Evidence-based recommendations have been made regarding risk assessment in individual patients; source control; the timing, selection, and duration of antimicrobial therapy; and suggested approaches to patients who fail initial therapy. Additional recommendations related to the treatment of pediatric patients with IAI have been included. Summary: The current recommendations of the SIS regarding the treatment of patients with IAI are provided in this guideline