Learning to Communicate Using Counterfactual Reasoning

This paper introduces a new approach for multi-agent communication learning called multi-agent counterfactual communication (MACC) learning. Many real-world problems are currently tackled using multi-agent techniques. However, in many of these tasks the agents do not observe the full state of the environment but only a limited observation. This absence of knowledge about the full state makes completing the objectives significantly more complex or even impossible. The key to this problem lies in sharing observation information between agents or learning how to communicate the essential data. In this paper we present a novel multi-agent communication learning approach called MACC. It addresses the partial observability problem of the agents. MACC lets the agent learn the action policy and the communication policy simultaneously. We focus on decentralized Markov Decision Processes (Dec-MDP), where the agents have joint observability. This means that the full state of the environment can be determined using the observations of all agents. MACC uses counterfactual reasoning to train both the action and the communication policy. This allows the agents to anticipate on how other agents will react to certain messages and on how the environment will react to certain actions, allowing them to learn more effective policies. MACC uses actor-critic with a centralized critic and decentralized actors. The critic is used to calculate an advantage for both the action and communication policy. We demonstrate our method by applying it on the Simple Reference Particle environment of OpenAI and a MNIST game. Our results are compared with a communication and non-communication baseline. These experiments demonstrate that MACC is able to train agents for each of these problems with effective communication policies.Comment: Submitted to NeurIPS2020. Contains 10 pages with 9 figures and 4 appendice

Analysis of 41 plant genomes supports a wave of successful genome duplications in association with the Cretaceous-Paleogene boundary

Ancient whole-genome duplications (WGDs), also referred to as paleopolyploidizations, have been reported in most evolutionary lineages. Their attributed role remains a major topic of discussion, ranging from an evolutionary dead end to a road toward evolutionary success, with evidence supporting both fates. Previously, based on dating WGDs in a limited number of plant species, we found a clustering of angiosperm paleopolyploidizations around the Cretaceous Paleogene (K-Pg) extinction event about 66 million years ago. Here we revisit this finding, which has proven controversial, by combining genome sequence information for many more plant lineages and using more sophisticated analyses. We include 38 full genome sequences and three transcriptome assemblies in a Bayesian evolutionary analysis framework that incorporates uncorrelated relaxed clock methods and fossil uncertainty. In accordance with earlier findings, we demonstrate a strongly nonrandom pattern of genome duplications over time with many WGDs clustering around the K-Pg boundary. We interpret these results in the context of recent studies on invasive polyploid plant species, and suggest that polyploid establishment is promoted during times of environmental stress. We argue that considering the evolutionary potential of polyploids in light of the environmental and ecological conditions present around the time of polyploidization could mitigate the stark contrast in the proposed evolutionary fates of polyploids

Strategy for the identification of micro-organisms producing food and feed products : bacteria producing food enzymes as study case

Recent European regulations require safety assessments of food enzymes (FE) before their commercialization. FE are mainly produced by micro-organisms, whose viable strains nor associated DNA can be present in the final products. Currently, no strategy targeting such impurities exists in enforcement laboratories. Therefore, a generic strategy of first line screening was developed to detect and identify, through PCR amplification and sequencing of the 16S-rRNA gene, the potential presence of FE producing bacteria in FE preparations. First, the specificity was verified using all microbial species reported to produce FE. Second, an in-house database, with 16S reference sequences from bacteria producing FE, was constructed for their fast identification through blast analysis. Third, the sensitivity was assessed on a spiked FE preparation. Finally, the applicability was verified using commercial FE preparations. Using straightforward PCR amplifications, Sanger sequencing and blast analysis, the proposed strategy was demonstrated to be convenient for implementation in enforcement laboratories

Expansive evolution of the TREHALOSE-6-PHOSPHATE PHOSPHATASE gene family in Arabidopsis

Trehalose is a nonreducing sugar used as a reserve carbohydrate and stress protectant in a variety of organisms. While higher plants typically do not accumulate high levels of trehalose, they encode large families of putative trehalose biosynthesis genes. Trehalose biosynthesis in plants involves a two-step reaction in which trehalose-6-phosphate (T6P) is synthesized from UDPglucose and glucose-6-phosphate (catalyzed by T6P synthase [TPS]), and subsequently dephosphorylated to produce the disaccharide trehalose (catalyzed by T6P phosphatase [TPP]). In Arabidopsis (Arabidopsis thaliana), 11 genes encode proteins with both TPS- and TPP-like domains but only one of these (AtTPS1) appears to be an active (TPS) enzyme. In addition, plants contain a large family of smaller proteins with a conserved TPP domain. Here, we present an in-depth analysis of the 10 TPP genes and gene products in Arabidopsis (TPPA-TPPJ). Collinearity analysis revealed that all of these genes originate from whole-genome duplication events. Heterologous expression in yeast (Saccharomyces cerevisiae) showed that all encode active TPP enzymes with an essential role for some conserved residues in the catalytic domain. These results suggest that the TPP genes function in the regulation of T6P levels, with T6P emerging as a novel key regulator of growth and development in higher plants. Extensive gene expression analyses using a complete set of promoter-beta-glucuronidase/green fluorescent protein reporter lines further uncovered cell- and tissue-specific expression patterns, conferring spatiotemporal control of trehalose metabolism. Consistently, phenotypic characterization of knockdown and overexpression lines of a single TPP, AtTPPG, points to unique properties of individual TPPs in Arabidopsis, and underlines the intimate connection between trehalose metabolism and abscisic acid signaling

Horsetails are ancient polyploids : evidence from Equisetum giganteum

Horsetails represent an enigmatic clade within the land plants. Despite consisting only of one genus (Equisetum) that contains 15 species, they are thought to represent the oldest extant genus within the vascular plants dating back possibly as far as the Triassic. Horsetails have retained several ancient features and are also characterized by a particularly high chromosome count (n = 108). Whole-genome duplications (WGDs) have been uncovered in many angiosperm clades and have been associated with the success of angiosperms, both in terms of species richness and biomass dominance, but remain understudied in nonangiosperm clades. Here, we report unambiguous evidence of an ancient WGD in the fern linage, based on sequencing and de novo assembly of an expressed gene catalog (transcriptome) from the giant horsetail (Equisetum giganteum). We demonstrate that horsetails underwent an independent paleopolyploidy during the Late Cretaceous prior to the diversification of the genus but did not experience any recent polyploidizations that could account for their high chromosome number. We also discuss the specific retention of genes following the WGD and how this may be linked to their long-term survival

Tangled up in two : a burst of genome duplications at the end of the Cretaceous and the consequences for plant evolution

Genome sequencing has demonstrated that besides frequent small-scale duplications, large-scale duplication events such as whole genome duplications (WGDs) are found on many branches of the evolutionary tree of life. Especially in the plant lineage, there is evidence for recurrent WGDs, and the ancestor of all angiospermswas in fact most likely a polyploid species. The number of WGDs found in sequenced plant genomes allows us to investigate questions about the roles of WGDs that were hitherto impossible to address. An intriguing observation is that many plant WGDs seem associated with periods of increased environmental stress and/or fluctuations, a trend that is evident for both present-day polyploids and palaeopolyploids formed around the Cretaceous–Palaeogene (K–Pg) extinction at 66 Ma. Here, we revisit the WGDs in plants that mark the K–Pg boundary, and discuss some specific examples of biological innovations and/or diversifications that may be linked to these WGDs. We review evidence for the processes that could have contributed to increased polyploid establishment at the K–Pg boundary, and discuss the implications on subsequent plant evolution in the Cenozoic.http://rstb.royalsocietypublishing.orghb201

Revisiting ancestral polyploidy in plants

Whole-genome duplications (WGDs) or polyploidy events have been studied extensively in plants. In a now widely cited paper, Jiao et al. presented evidence for two ancient, ancestral plant WGDs predating the origin of flowering and seed plants, respectively. This finding was based primarily on a bimodal age distribution of gene duplication events obtained from molecular dating of almost 800 phylogenetic gene trees. We reanalyzed the phylogenomic data of Jiao et al. and found that the strong bimodality of the age distribution may be the result of technical and methodological issues and may hence not be a "true" signal of two WGD events. By using a state-of-the-art molecular dating algorithm, we demonstrate that the reported bimodal age distribution is not robust and should be interpreted with caution. Thus, there exists little evidence for two ancient WGDs in plants from phylogenomic dating

Development of an NGS-based workflow for improved monitoring of circulating plasmids in support of risk assessment of antimicrobial resistance gene dissemination

Antimicrobial resistance (AMR) is one of the most prominent public health threats. AMR genes localized on plasmids can be easily transferred between bacterial isolates by horizontal gene transfer, thereby contributing to the spread of AMR. Next-generation sequencing (NGS) technologies are ideal for the detection of AMR genes; however, reliable reconstruction of plasmids is still a challenge due to large repetitive regions. This study proposes a workflow to reconstruct plasmids with NGS data in view of AMR gene localization, i.e., chromosomal or on a plasmid. Whole-genome and plasmid DNA extraction methods were compared, as were assemblies consisting of short reads (Illumina MiSeq), long reads (Oxford Nanopore Technologies) and a combination of both (hybrid). Furthermore, the added value of conjugation of a plasmid to a known host was evaluated. As a case study, an isolate harboring a large, low-copymcr-1-carrying plasmid (>200 kb) was used. Hybrid assemblies of NGS data obtained from whole-genome DNA extractions of the original isolates resulted in the most complete reconstruction of plasmids. The optimal workflow was successfully applied to multidrug-resistantSalmonellaKentucky isolates, where the transfer of an ESBL-gene-containing fragment from a plasmid to the chromosome was detected. This study highlights a strategy including wet and dry lab parameters that allows accurate plasmid reconstruction, which will contribute to an improved monitoring of circulating plasmids and the assessment of their risk of transfer