Sampled Policy Gradient for Learning to Play the Game Agar.io

In this paper, a new offline actor-critic learning algorithm is introduced: Sampled Policy Gradient (SPG). SPG samples in the action space to calculate an approximated policy gradient by using the critic to evaluate the samples. This sampling allows SPG to search the action-Q-value space more globally than deterministic policy gradient (DPG), enabling it to theoretically avoid more local optima. SPG is compared to Q-learning and the actor-critic algorithms CACLA and DPG in a pellet collection task and a self play environment in the game Agar.io. The online game Agar.io has become massively popular on the internet due to intuitive game design and the ability to instantly compete against players around the world. From the point of view of artificial intelligence this game is also very intriguing: The game has a continuous input and action space and allows to have diverse agents with complex strategies compete against each other. The experimental results show that Q-Learning and CACLA outperform a pre-programmed greedy bot in the pellet collection task, but all algorithms fail to outperform this bot in a fighting scenario. The SPG algorithm is analyzed to have great extendability through offline exploration and it matches DPG in performance even in its basic form without extensive sampling

Hierarchical reinforcement learning for real-time strategy games

Real-Time Strategy (RTS) games can be abstracted to resource allocation applicable in many fields and industries. We consider a simplified custom RTS game focused on mid-level combat using reinforcement learning (RL) algorithms. There are a number of contributions to game playing with RL in this paper. First, we combine hierarchical RL with a multi-layer perceptron (MLP) that receives higher-order inputs for increased learning speed and performance. Second, we compare Q-learning against Monte Carlo learning as reinforcement learning algorithms. Third, because the teams in the RTS game are multi-agent systems, we examine two different methods for assigning rewards to agents. Experiments are performed against two different fixed opponents. The results show that the combination of Q-learning and individual rewards yields the highest win-rate against the different opponents, and is able to defeat the opponent within 26 training games

Non local damage model Boundary and evolving boundary effects

International audienceThe present contribution aims at providing a closer insight on boundary effects in non local damage modelling. From micromechanics, we show that on a boundary interaction stress components normal to the surface should vanish. These interaction stresses are at the origin of non locality and therefore the material response of points located on the boundary should be partially local. Then, we discuss a tentative modification of the classical non local damage model aimed at accounting for this effect due to existing boundaries and also boundaries that arise from crack propagation. One-dimensional computations show that the profiles of damage are quite different compared to those obtained with the original formulation. The region in which damage is equal to 1 is small. The modified model performs better at complete failure, with a consistent description of discontinuity of the displacement field after failure

Ethics and social responsibility in practice: interpreters and translators engaging with and beyond the professions

Interpreting and translation are unregulated activities in most countries, yet interpreters and translators perform challenging work in sensitive domains, such as the law, medicine and social work. Other professionals working in these sectors must complete formal ethics training to qualify, then subscribe to Codes of Practice or Ethics. When they face ethical challenges in their work, they can access ongoing support. They must undertake regular refresher training in ethics. Interpreters and translators rarely have access to this sort of ethical infrastructure. This places the onus on interpreters and translators to reflect on ethical aspects of their practice, for reasons related to both professional performance and social responsibility. This contribution presents original UK-based research with one type of professional ‘clients’ who rely on interpreters and translators, social workers and social work students prior to their first work experience placement. Findings suggest that insufficient attention has been paid to such professional clients and that ethical aspects of professional communication can be compromised as a result. By framing ethics training and ongoing support in terms of social responsibility, we point to some ways in which the different professional groups might communicate and work more effectively with one another and with service users

Identification of Residues in the Cysteine-rich Domain of Raf-1 That Control Ras Binding and Raf-1 Activity

We have identified mutations in Raf-1 that increase binding to Ras. The mutations were identified making use of three mutant forms of Ras that have reduced Raf-1 binding (Winkler, D. G., Johnson, J. C., Cooper, J. A., and Vojtek, A. B. (1997) J. Biol. Chem. 272, 24402-24409). One mutation in Raf-1, N64L, suppresses the Ras mutant R41Q but not other Ras mutants, suggesting that this mutation structurally complements the Ras R41Q mutation. Missense substitutions of residues 143 and 144 in the Raf-1 cysteine-rich domain were isolated multiple times. These Raf-1 mutants, R143Q, R143W, and K144E, were general suppressors of three different Ras mutants and had increased interaction with non-mutant Ras. Each was slightly activated relative to wild-type Raf-1 in a transformation assay. In addition, two mutants, R143W and K144E, were active when tested for induction of germinal vesicle breakdown in Xenopus oocytes. Interestingly, all three cysteine-rich domain mutations reduced the ability of the Raf-1 N-terminal regulatory region to inhibit Xenopus oocyte germinal vesicle breakdown induced by the C-terminal catalytic region of Raf-1. We propose that a direct or indirect regulatory interaction between the N- and C-terminal regions of Raf-1 is reduced by the R143W, R143Q, and K144E mutations, thereby increasing access to the Ras-binding regions of Raf-1 and increasing Raf-1 activity

Ras Interaction with Two Distinct Binding Domains in Raf-1 5 Be Required for Ras Transformation

Although Raf-1 is a critical Ras effector target, how Ras mediates Raf-1 activation remains unresolved. Raf-1 residues 55-131 define a Ras-binding domain essential for Raf-1 activation. Therefore, our identification of a second Ras-binding site in the Raf-1 cysteine-rich domain (residues 139-184) was unexpected and suggested a more complex role for Ras in Raf-1 activation. Both Ras recognition domains preferentially associate with Ras-GTP. Therefore, mutations that impair Ras activity by perturbing regions that distinguish Ras-GDP from Ras-GTP (switch I and II) may disrupt interactions with either Raf-1-binding domain. We observed that mutations of Ras that impaired Ras transformation by perturbing its switch I (T35A and E37G) or switch II (G60A and Y64W) domain preferentially diminished binding to Raf-1-(55-131) or the Raf-1 cysteine-rich domain, respectively. Thus, these Ras-binding domains recognize distinct Ras-GTP determinants, and both may be essential for Ras transforming activity. Finally, since Ha-Ras T35A and E37G mutations prevent Ras interaction with full-length Raf-1, we suggest that Raf-Cys is a cryptic binding site that is unmasked upon Ras interaction with Raf-1-(55-131)

Leadership and the Australian Greens

This paper examines the inherent tension between a Green political party’s genesis and official ideology and the conventional forms and practices of party leadership enacted in the vast bulk of other parties, regardless of their place on the ideological spectrum. A rich picture is painted of this ongoing struggle through a case study of the Australian Greens with vivid descriptions presented on organisational leadership issues by Australian state and federal Green members of parliaments. What emerges from the data is the Australian Green MPs’ conundrum in retaining an egalitarian and participatory democracy ethos while seeking to expand their existing frame of leadership to being both more pragmatic and oriented towards active involvement in government

Maternal hormone levels among populations at high and low risk of testicular germ cell cancer

Ethnic differences in maternal oestrogen levels have been suggested as explaining the significantly higher risk of testicular germ cell tumours (TGCT) of white men than black men in the United States. We therefore examined levels of maternal oestrogens, as well as testosterone and alphafetoprotein (AFP), in 150 black and 150 white mothers in the Collaborative Perinatal Project. Serum levels of estradiol (total, free and bioavailable), estriol, testosterone (total, free and bioavailable), sex hormone binding globulin (SHBG), and AFP were examined during first and third trimesters. We found that the black mothers, rather than the white mothers, had significantly higher estradiol levels in first trimester (P=0.05). Black mothers also had significantly higher levels of all testosterone (P<0.001) and AFP (P<0.001) in both trimesters. In addition, the ratios of sex hormones (estradiol/testosterone) were significantly lower among black mothers. These findings provide little support to the oestrogen hypothesis, but are consistent with higher levels of testosterones and/or AFP being associated with reduced risk of TGCT; alternatively, lower oestrogen/androgen ratios may be associated with reduced risk

Genetic Disruption of Both Tryptophan Hydroxylase Genes Dramatically Reduces Serotonin and Affects Behavior in Models Sensitive to Antidepressants

The neurotransmitter serotonin (5-HT) plays an important role in both the peripheral and central nervous systems. The biosynthesis of serotonin is regulated by two rate-limiting enzymes, tryptophan hydroxylase-1 and -2 (TPH1 and TPH2). We used a gene-targeting approach to generate mice with selective and complete elimination of the two known TPH isoforms. This resulted in dramatically reduced central 5-HT levels in Tph2 knockout (TPH2KO) and Tph1/Tph2 double knockout (DKO) mice; and substantially reduced peripheral 5-HT levels in DKO, but not TPH2KO mice. Therefore, differential expression of the two isoforms of TPH was reflected in corresponding depletion of 5-HT content in the brain and periphery. Surprisingly, despite the prominent and evolutionarily ancient role that 5-HT plays in both vertebrate and invertebrate physiology, none of these mutations resulted in an overt phenotype. TPH2KO and DKO mice were viable and normal in appearance. Behavioral alterations in assays with predictive validity for antidepressants were among the very few phenotypes uncovered. These behavioral changes were subtle in the TPH2KO mice; they were enhanced in the DKO mice. Herein, we confirm findings from prior descriptions of TPH1 knockout mice and present the first reported phenotypic evaluations of Tph2 and Tph1/Tph2 knockout mice. The behavioral effects observed in the TPH2 KO and DKO mice strongly confirm the role of 5-HT and its synthetic enzymes in the etiology and treatment of affective disorders