SMIX(λ\lambda): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learning

Learning a stable and generalizable centralized value function (CVF) is a crucial but challenging task in multi-agent reinforcement learning (MARL), as it has to deal with the issue that the joint action space increases exponentially with the number of agents in such scenarios. This paper proposes an approach, named SMIX(${\lambda}$), to address the issue using an efficient off-policy centralized training method within a flexible learner search space. As importance sampling for such off-policy training is both computationally costly and numerically unstable, we proposed to use the ${\lambda}$-return as a proxy to compute the TD error. With this new loss function objective, we adopt a modified QMIX network structure as the base to train our model. By further connecting it with the ${Q(\lambda)}$ approach from an unified expectation correction viewpoint, we show that the proposed SMIX(${\lambda}$) is equivalent to ${Q(\lambda)}$ and hence shares its convergence properties, while without being suffered from the aforementioned curse of dimensionality problem inherent in MARL. Experiments on the StarCraft Multi-Agent Challenge (SMAC) benchmark demonstrate that our approach not only outperforms several state-of-the-art MARL methods by a large margin, but also can be used as a general tool to improve the overall performance of other CTDE-type algorithms by enhancing their CVFs

Effects of the Interception of Litterfall by the Understory on Carbon Cycling in Eucalyptus Plantations of South China

For the purposes of forest restoration, carbon (C) fixation, and economic improvement, eucalyptus (Eucalyptus urophylla) has been widely planted in South China. The understory of eucalyptus plantations is often occupied by a dense community of the fern Dicranopteris dichotoma, which intercepts tree canopy leaf litter before it reaches the ground. To understand the effects of this interception of litterfall on C cycling in eucalyptus plantations, we quantified the mass of intercepted litter and the influences of litterfall interception on litter decomposition and soil respiration. The total mass of E. urophylla litterfall collected on the understory was similar to that collected by the traditional litter trap method. All of the eucalyptus litterfall is intercepted by the D. dichotoma canopy. Of the litterfall that was intercepted by D. dichotoma, 20–40% and 60–80% was intercepted by the top (50–100 cm) and bottom (0–50 cm) of the understory canopy, respectively. Intercepted litterfall decomposed faster at the bottom of understory canopy (at the base of the plants) than at the top, and decomposition was slower on the soil surface in the absence of understory than on any location in the understory canopy. Soil respiration was highest when both the understory and litter were present and was lowest when both the understory and litter were absent. These results indicate that litterfall interception changed carbon flow between aboveground and belowground through litter decomposition and soil respiration, which changed carbon cycling in eucalyptus plantations. The effects of the understory on litter decomposition and soil respiration should be considered in ecosystem carbon models

Three dimensional spider-web-like superconducting filamentary paths in KxFe2−ySe2K_xFe_{2-y}Se_2 single crystals

Since the discovery of high temperature superconductivity in F-doped LaFeAsO, many new iron based superconductors with different structures have been fabricated2. The observation of superconductivity at about 32 K in KxFe2-ySe2 with the iso-structure of the FeAs-based 122 superconductors was a surprise and immediately stimulated the interests because the band structure calculation8 predicted the absence of the hole pocket which was supposed to be necessary for the theoretical picture of S+- pairing. Soon later, it was found that the material may separate into the insulating antiferromagnetic K2Fe4Se5 phase and the superconducting phase. It remains unresolved that how these two phases coexist and what is the parent phase for superconductivity. In this study we use different quenching processes to produce the target samples with distinct microstructures, and apply multiple measuring techniques to reveal a close relationship between the microstructures and the global appearance of superconductivity. In addition, we clearly illustrate three dimensional spider-web-like superconducting filamentary paths, and for the first time propose that the superconducting phase may originate from a state with one vacancy in every eight Fe-sites with the root8*root10 parallelogram structure.Comment: 22 pages, 7 figure

Identification of novel amplification gene targets in mouse and human breast cancer at a syntenic cluster mapping to mouse identification of novel amplification gene targets in mouse and human breast cancer at a syntenic cluster mapping to mouse ch8a1 and human ch13q34

Serial analysis of gene expression from aggressive mammary tumors derived from transplantable p53 null mouse mammary outgrowth lines revealed significant up-regulation of Tfdp1 (transcription factor Dp1), Lamp1 (lysosomal membrane glycoprotein 1) and Gas6 (growth arrest specific 6) transcripts. All of these genes belong to the same linkage cluster, mapping to mouse chromosome band 8A1. BAC-array comparative genomic hybridization and fluorescence in situ hybridization analyses revealed genomic amplification at mouse region ch8A1.1. The minimal region of amplification contained genes Cul4a, Lamp1, Tfdp1, and Gas6, highly overexpressed in the p53 null mammary outgrowth lines at preneoplastic stages, and in all its derived tumors. The same amplification was also observed in spontaneous p53 null mammary tumors. Interestingly, this region is homologous to human chromosome 13q34, and some of the same genes were previously observed amplified in human carcinomas. Thus, we further investigated the occurrence and frequency of gene amplification affecting genes mapping to ch13q34 in human breast cancer. TFDP1 showed the highest frequency of amplification affecting 31% of 74 breast carcinomas analyzed. Statistically significant positive correlation was observed for the amplification of CUL4A, LAMP1, TFDP1, and GAS6 genes (P < 0.001). Meta-analysis of publicly available gene expression data sets showed a strong association between the high expression of TFDP1 and decreased overall survival (P = 0.00004), relapse-free survival (P = 0.0119), and metastasis-free interval (P = 0.0064). In conclusion, our findings suggest that CUL4A, LAMP1, TFDP1, and GAS6 are targets for overexpression and amplification in breast cancers. Therefore, overexpression of these genes and, in particular, TFDP1 might be of relevance in the development and/or progression in a significant subset of human breastFil: Abba, Martín Carlos. University of Texas; Estados Unidos. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Fabris, Victoria Teresa. University of Texas; Estados Unidos. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; ArgentinaFil: Hu, Yuhui. University of Texas; Estados UnidosFil: Kittrell, Frances S.. Baylor College of Medicine; Estados Unidos. University of Texas; Estados UnidosFil: Cai, Wei Wen. University of Texas; Estados Unidos. Baylor College of Medicine; Estados UnidosFil: Donehower, Lawrence A.. University of Texas; Estados UnidosFil: Sahin, Aysegui. University of Texas; Estados UnidosFil: Medina, Daniel. University of Texas; Estados Unidos. Baylor College of Medicine; Estados UnidosFil: Aldaz, Claudio Marcelo. University of Texas; Estados Unido

Chemical probing of thiotetronate bio-assembly

Chemical ‘chain termination’ probes were utilised for the investigation of thiotetronate antibiotic biosynthesis in the filamentous bacteria Lentzea sp. and Streptomyces thiolactonus NRRL 15439. The use of these tools led to the capture of biosynthetic intermediates involved in the thiotetronate polyketide backbone assembly, providing first insights into substrate specificity and in vivo intermediate processing by unusual iterative synthases

A Single Residue Mutation in the Gαq Subunit of the G Protein Complex Causes Blindness in Drosophila.

Heterotrimeric G proteins play central roles in many signaling pathways, including the phototransduction cascade in animals. However, the degree of involvement of the G protein subunit Gαq is not clear since animals with previously reported strong loss-of-function mutations remain responsive to light stimuli. We recovered a new allele of Gαq in Drosophila that abolishes light response in a conventional electroretinogram assay, and reduces sensitivity in whole-cell recordings of dissociated cells by at least five orders of magnitude. In addition, mutant eyes demonstrate a rapid rate of degeneration in the presence of light. Our new allele is likely the strongest hypomorph described to date. Interestingly, the mutant protein is produced in the eyes but carries a single amino acid change of a conserved hydrophobic residue that has been assigned to the interface of interaction between Gαq and its downstream effector, PLC. Our study has thus uncovered possibly the first point mutation that specifically affects this interaction in vivo