Efficient Deep Reinforcement Learning via Adaptive Policy Transfer

Transfer Learning (TL) has shown great potential to accelerate Reinforcement Learning (RL) by leveraging prior knowledge from past learned policies of relevant tasks. Existing transfer approaches either explicitly computes the similarity between tasks or select appropriate source policies to provide guided explorations for the target task. However, how to directly optimize the target policy by alternatively utilizing knowledge from appropriate source policies without explicitly measuring the similarity is currently missing. In this paper, we propose a novel Policy Transfer Framework (PTF) to accelerate RL by taking advantage of this idea. Our framework learns when and which source policy is the best to reuse for the target policy and when to terminate it by modeling multi-policy transfer as the option learning problem. PTF can be easily combined with existing deep RL approaches. Experimental results show it significantly accelerates the learning process and surpasses state-of-the-art policy transfer methods in terms of learning efficiency and final performance in both discrete and continuous action spaces.Comment: Accepted by IJCAI'202

RIdeogram : Drawing SVG graphics to visualize and map genome-wide data on the idiograms

Background. Owing to the rapid advances in DNA sequencing technologies, whole genome from more and more species are becoming available at increasing pace. For whole-genome analysis, idiograms provide a very popular, intuitive and effective way to map and visualize the genome-wide information, such asGCcontent, gene and repeat density, DNA methylation distribution, genomic synteny, etc. However, most available software programs and web servers are available only for a few model species, such as human, mouse and fly, or have limited application scenarios. As more and more non-model species are sequenced with chromosome-level assembly being available, tools that can generate idiograms for a broad range of species and be capable of visualizing more data types are needed to help better understanding fundamental genome characteristics. Results. The R package RIdeogram allows users to build high-quality idiograms of any species of interest. It can map continuous and discrete genome-wide data on the idiograms and visualize them in a heat map and track labels, respectively. Conclusion. The visualization of genome-wide data mapping and comparison allow users to quickly establish a clear impression of the chromosomal distribution pattern, thus making RIdeogram a useful tool for any researchers working with omics.</p

Formamide deionized accelerates the somatic embryogenesis of Cunninghamia lanceolata

Aim of the study: To improve the efficiency of the somatic embryogenesis (SE) in Cunninghamia lanceolata. Area of the study: The study was conducted at Nanjing Forestry University (Nanjing, China). Material and methods: Immature cones of C. lanceolata, genotype 01A1 which was planted in Yangkou State-owned Forest Farm (Fujian, China), were used to induced callus. These calli were used to induce SE, concentration gradients of 0 g/L, 0.01134 g/L, 0.1134 g/L, 1.1134 g/L and 11.34 g/L of FD was added, to explore the optimal concentration for promoting SE of C. lanceolata. Main results: Low concentration of FD promoted the maturation of somatic embryos, while high concentration of FD lead to browning of embryogenic callus. The seedling rate and rooting number of seedlings induced by different concentrations of FD were significantly different. Research highlights: This study may aid in the rapid maturation of C. lanceolata somatic embryos and is useful for accelerated C. lanceolata breeding. Keywords: C. lanceolata; Formamide Deionized; Somatic embryogenesis; Seedling rate. Abbreviations used: FD (Formamide Deionized), FD0 (the concentration of 0 g/L FD), FD0.01134 (the concentration of 0.01134 g/L FD), FD0.1134 (the concentration of 0.1134 g/L FD), FD1.134 (the concentration of 1.134 g/L FD), FD11.34 (the concentration of 11.34 g/L FD)

Genome-wide analysis of the GRAS gene family in Liriodendron chinense reveals the putative function in abiotic stress and plant development

IntroductionGRAS genes encode plant-specific transcription factors that play essential roles in plant growth and development. However, the members and the function of the GRAS gene family have not been reported in Liriodendron chinense. L. chinense, a tree species in the Magnolia family that produces excellent timber for daily life and industry. In addition, it is a good relict species for plant evolution research.MethodsTherefore, we conducted a genome-wide study of the LcGRAS gene family and identified 49 LcGRAS genes in L. chinense.ResultsWe found that LcGRAS could be divided into 13 sub-groups, among which there is a unique branch named HAM-t. We carried out RNA sequencing analysis of the somatic embryos from L. chinense and found that LcGRAS genes are mainly expressed after heart-stage embryo development, suggesting that LcGRAS may have a function during somatic embryogenesis. We also investigated whether GRAS genes are responsive to stress by carrying out RNA sequencing (RNA-seq) analysis, and we found that the genes in the PAT subfamily were activated upon stress treatment, suggesting that these genes may help plants survive stressful environments. We found that PIF was downregulated and COR was upregulated after the transient overexpression of PATs, suggesting that PAT may be upstream regulators of cold stress. DiscussionCollectively, LcGRAS genes are conserved and play essential roles in plant development and adaptation to abiotic stress

Halophyte Nitraria billardieri CIPK25 promotes photosynthesis in Arabidopsis under salt stress

The calcineurin B-like (CBL)-interacting protein kinases (CIPKs), a type of plant-specific genes in the calcium signaling pathway, function in response to adverse environments. However, few halophyte derived CIPKs have been studied for their role in plant physiological and developmental adaptation during abiotic stresses, which inhibits the potential application of these genes to improve environmental adaptability of glycophytes. In this study, we constructed Nitraria billardieri CIPK25 overexpressing Arabidopsis and analyzed the seedling development under salt treatment. Our results show that Arabidopsis with NbCIPK25 expression exhibits more vigorous growth than wild type plants under salt condition. To gain insight into the molecular mechanisms underlying salt tolerance, we profiled the transcriptome of WT and transgenic plants via RNA-seq. GO and KEGG analyses revealed that upregulated genes in NbCIPK25 overexpressing seedlings under salt stress are enriched in photosynthesis related terms; Calvin-cycle genes including glyceraldehyde-3-phosphate dehydrogenases (GAPDHs) are significantly upregulated in transgenic plants, which is consistent with a decreased level of NADPH (GAPDH substrate) and increased level of NADP+. Accordingly, NbCIPK25 overexpressing plants exhibited more efficient photosynthesis; soluble sugar and proteins, as photosynthesis products, showed a higher accumulation in transgenic plants. These results provide molecular insight into how NbCIPK25 promotes the expression of genes involved in photosynthesis, thereby maintaining plant growth under salt stress. Our finding supports the potential application of halophyte-derived NbCIPK25 in genetic modification for better salt adaptation

Unveiling Defect-Mediated Carrier Dynamics in Monolayer Semiconductors by Spatiotemporal Microwave Imaging

The optoelectronic properties of atomically thin transition-metal dichalcogenides are strongly correlated with the presence of defects in the materials, which are not necessarily detrimental for certain applications. For instance, defects can lead to an enhanced photoconduction, a complicated process involving charge generation and recombination in the time domain and carrier transport in the spatial domain. Here, we report the simultaneous spatial and temporal photoconductivity imaging in two types of WS2 monolayers by laser-illuminated microwave impedance microscopy. The diffusion length and carrier lifetime were directly extracted from the spatial profile and temporal relaxation of microwave signals respectively. Time-resolved experiments indicate that the critical process for photo-excited carriers is the escape of holes from trap states, which prolongs the apparent lifetime of mobile electrons in the conduction band. As a result, counterintuitively, the photoconductivity is stronger in CVD samples than exfoliated monolayers with a lower defect density. Our work reveals the intrinsic time and length scales of electrical response to photo-excitation in van der Waals materials, which is essential for their applications in novel optoelectronic devices.Comment: 21 pages, 4 figure

Superior photo-carrier diffusion dynamics in organic-inorganic hybrid perovskites revealed by spatiotemporal conductivity imaging

The outstanding performance of organic-inorganic metal trihalide solar cells benefits from the exceptional photo-physical properties of both electrons and holes in the material. Here, we directly probe the free-carrier dynamics in Cs-doped FAPbI3 thin films by spatiotemporal photoconductivity imaging. Using charge transport layers to selectively quench one type of carriers, we show that the two relaxation times on the order of 1 μs and 10 μs correspond to the lifetimes of electrons and holes in FACsPbI3, respectively. Strikingly, the diffusion map- ping indicates that the difference in electron/hole lifetimes is largely compensated by their disparate mobility. Consequently, the long diffusion lengths (3~5 μm) of both carriers are comparable to each other, a feature closely related to the unique charge trapping and de- trapping processes in hybrid trihalide perovskites. Our results unveil the origin of superior diffusion dynamics in this material, crucially important for solar-cell applications.The research at UT-Austin was primarily supported by the NSF through the Center for Dynamics and Control of Materials, an NSF Materials Research Science and Engineering Center (MRSEC) under Cooperative Agreement DMR-1720595. The authors also acknowledge the use of facilities and instrumentation supported by the NSF MRSEC. K.L. and X.M. acknowledge the support from Welch Foundation Grant F-1814. X. Li acknowledges the support from Welch Foundation Grant F-1662. The tip-scan iMIM setup was supported by the US Army Research Laboratory and the US Army Research Office under Grants W911NF-16-1-0276 and W911NF-17-1-0190. The work at NREL was supported by the US DOE under Contract No. DE-AC36-08GO28308 with Alliance for Sustainable Energy, Limited Liability Company (LLC), the Manager and Operator of the National Renewable Energy Laboratory. K.Z., J.H., X.C., X.W., and Y.Y. acknowledge the support on charge carrier dynamics study from the Center for Hybrid Organic-Inorganic Semiconductors for Energy (CHOISE), an Energy Frontier Research Center funded by the Office of Basic Energy Sciences, Office of Science within the US DOE. F.Z. acknowledges the support on devices fabrication and characterizations from the De-Risking Halide PSCs program of the National Center for Photovoltaics, funded by the US DOE, Office of Energy Efficiency and Renewable Energy, Solar Energy Technologies Office.Center for Dynamics and Control of Material

Comparative Analysis of the Chloroplast Genomic Information of Cunninghamia lanceolata (Lamb.) Hook with Sibling Species from the Genera Cryptomeria D. Don, Taiwania Hayata, and Calocedrus Kurz

Chinese fir (Cunninghamia lanceolata (Lamb.) Hook) is an important coniferous tree species for timber production, which accounts for ~40% of log supply from plantations in southern China. Chloroplast genetic engineering is an exciting field to engineer several valuable tree traits. In this study, we revisited the published complete Chinese fir (NC_021437) and four other coniferous species chloroplast genome sequence in Taxodiaceae. Comparison of their chloroplast genomes revealed three unique inversions found in the downstream of the gene clusters and evolutionary divergence were found, although overall the chloroplast genomic structure of the Cupressaceae linage was conserved. We also investigated the phylogenetic position of Chinese fir among conifers by examining gene functions, selection forces, substitution rates, and the full chloroplast genome sequence. Consistent with previous molecular systematics analysis, the results provided a well-supported phylogeny framework for the Cupressaceae that strongly confirms the “basal” position of Cunninghamia lanceolata. The structure of the Cunninghamia lanceolata chloroplast genome showed a partial lack of one IR copy, rearrangements clearly occurred and slight evolutionary divergence appeared among the cp genome of C. lanceolata, Taiwania cryptomerioides, Taiwania flousiana, Calocedrus formosana and Cryptomeria japonica. The information from sequence divergence and length variation of genes could be further considered for bioengineering research

Comprehensive Review of Safety Studies in Process Industrial Systems: Concepts, Progress, and Main Research Topics

This paper focuses on reviewing past progress in the advancement of definitions, methods, and models for safety analysis and assessment of process industrial systems and highlighting the main research topics. Based on the analysis of the knowledge with respect to process safety, the review covers the fact that the entire system does not have the ability to produce casualties, health deterioration, and other accidents, which ultimately cause human life threats and health damage. And, according to the comparison between safety and reliability, when a system is in an unreliable state, it must be in an unsafe state. Related works show that the main organizations and regulations are developed and grouped together, and these are also outlined in the literature. The progress and current research topics of the methods and models have been summarized and discussed in the analysis and assessment of safety for process industrial systems, which mainly illustrate that the dynamic operational safety assessment under the big data challenges will become the research direction, which will change the future study situation