208 research outputs found

    Reinforcement Learning Experience Reuse with Policy Residual Representation

    Full text link
    Experience reuse is key to sample-efficient reinforcement learning. One of the critical issues is how the experience is represented and stored. Previously, the experience can be stored in the forms of features, individual models, and the average model, each lying at a different granularity. However, new tasks may require experience across multiple granularities. In this paper, we propose the policy residual representation (PRR) network, which can extract and store multiple levels of experience. PRR network is trained on a set of tasks with a multi-level architecture, where a module in each level corresponds to a subset of the tasks. Therefore, the PRR network represents the experience in a spectrum-like way. When training on a new task, PRR can provide different levels of experience for accelerating the learning. We experiment with the PRR network on a set of grid world navigation tasks, locomotion tasks, and fighting tasks in a video game. The results show that the PRR network leads to better reuse of experience and thus outperforms some state-of-the-art approaches.Comment: Conference version appears in IJCAI 201

    Throughput maximization for RIS-UAV relaying communications

    Get PDF
    In this paper, we consider a reconfigurable intelligent surface (RIS) assisted unmanned aerial vehicle (UAV) relaying communication system, where the RIS is mounted on the UAV and can move at a high speed. Compared with the conventional static RIS, better performance and more flexibility can be achieved with the assistance of the mobile UAV. We maximize the average downlink throughput by jointly optimizing the UAV trajectory, RIS passive beamforming and source power allocation for each time slot. The formulated non-convex optimization problem is decomposed into three subproblems: passive beamforming optimization, trajectory optimization and power allocation optimization. An alternating iterative optimization algorithm of the three subproblems is proposed to achieve the suboptimal solutions. The numerical results indicate that the RIS-UAV relaying communication system with trajectory optimization can get higher throughput

    Effect of Attapulgite Nanorods and Calcium Sulfate Microwhiskers on the Reaction-Induced Phase Separation of Epoxy/PES Blends

    Get PDF
    The influence of two kinds of mesoscale inorganic rod fillers, nanoscale attapulgite and micron-sized CaSO4 whisker, on the reaction-induced phase separation of epoxy/aromatic amine/poly- (ether sulfone) (PES) blends has been investigated by optical microscopy (OM), scanning electron microscopy (SEM), and time resolved light scattering (TRLS). By varying the PES concentration and curing temperature, we found that the incorporation of attapulgite and CaSO4 had dramatic impact on the phase separation process and the final phase morphology of blends. In blends at higher content than critical concentration, the process of phase separation was retarded by the incorporation of nanoscale fillers but accelerated by that of the micron-sized fillers, mainly due to the enhanced viscoelastic effect and the preferential wettable effect, respectively. Meanwhile both mesoscale fillers could change the cocontinuous phase structure of blends with lower PES content than critical concentration into PES-rich dispersed structure due to the surface affinity of fillers to epoxy matrix

    Multimodality Molecular Imaging of Cardiovascular Disease Based on Nanoprobes

    Get PDF
    Recently, multimodality molecular imaging has evolved into a fast-growing research field with goals of detecting and measuring biological processes in vivo non-invasively. Researchers have come to realize that the complementary abilities of different imaging modalities over single modality could provide more precisely information for the diagnosis of diseases. At present, nanoparticles-based multimodal imaging probes have received significant attention because of their ease of preparation and straightforward integration of each modality into one entity. More importantly, nanotechnology has an increasing impact on multimodality molecular imaging of cardiovascular diseases, such as atherosclerosis and vulnerable plaque, myocardial infarction, angiogenesis, apoptosis and so on. In this review, we briefly summarize that various nanoprobes are exploited for targeted molecular imaging of cardiovascular diseases, as well as associated multimodality imaging approaches and their applications in the diagnosis and treatment of cardiovascular diseases

    Transcriptome analysis of Deinagkistrodon acutus venomous gland focusing on cellular structure and functional aspects using expressed sequence tags

    Get PDF
    BACKGROUND: The snake venom gland is a specialized organ, which synthesizes and secretes the complex and abundant toxin proteins. Though gene expression in the snake venom gland has been extensively studied, the focus has been on the components of the venom. As far as the molecular mechanism of toxin secretion and metabolism is concerned, we still knew a little. Therefore, a fundamental question being arisen is what genes are expressed in the snake venom glands besides many toxin components? RESULTS: To examine extensively the transcripts expressed in the venom gland of Deinagkistrodon acutus and unveil the potential of its products on cellular structure and functional aspects, we generated 8696 expressed sequence tags (ESTs) from a non-normalized cDNA library. All ESTs were clustered into 3416 clusters, of which 40.16% of total ESTs belong to recognized toxin-coding sequences; 39.85% are similar to cellular transcripts; and 20.00% have no significant similarity to any known sequences. By analyzing cellular functional transcripts, we found high expression of some venom related genes and gland-specific genes, such as calglandulin EF-hand protein gene and protein disulfide isomerase gene. The transcripts of creatine kinase and NADH dehydrogenase were also identified at high level. Moreover, abundant cellular structural proteins similar to mammalian muscle tissues were also identified. The phylogenetic analysis of two snake venom toxin families of group III metalloproteinase and serine protease in suborder Colubroidea showed an early single recruitment event in the viperids evolutionary process. CONCLUSION: Gene cataloguing and profiling of the venom gland of Deinagkistrodon acutus is an essential requisite to provide molecular reagents for functional genomic studies needed for elucidating mechanisms of action of toxins and surveying physiological events taking place in the very specialized secretory tissue. So this study provides a first global view of the genetic programs for the venom gland of Deinagkistrodon acutus described so far and an insight into molecular mechanism of toxin secreting. All sequences data reported in this paper have been submitted into the public database [GenBank: DV556511-DV565206]

    A Biomechanical Model of the Inner Ear: Numerical Simulation of the Caloric Test

    Get PDF
    Whether two vertical semicircular canals can receive thermal stimuli remains controversial. This study examined the caloric response in the three semicircular canals to the clinical hot caloric test using the finite element method. The results of the developed model showed the horizontal canal (HC) cupula maximally deflected to the utricle side by approximately 3 μm during the hot supine test. The anterior canal cupula began to receive the caloric stimuli about 20 s after the HC cupula, and it maximally deflected to the canal side by 0.55 μm. The posterior canal cupula did not receive caloric stimuli until approximately 40 s after the HC cupula, and it maximally deflected to the canal side by 0.34 μm. Although the endolymph flow and the cupular deformation change with respect to the head position during the test, the supine test ensures the maximal caloric response in the HC, but no substantial improvement for the responses of the two vertical canals was observed. In conclusion, while the usual supine test is the optimum test for evaluating the functions of the inner ear, more irrigation time is needed in order to effectively clinically examine the vertical canals

    Comparative genomics reveals adaptive evolution of Asian tapeworm in switching to a new intermediate host

    Get PDF
    Taenia saginata, Taenia solium and Taenia asiatica (beef, pork and Asian tapeworms, respectively) are parasitic flatworms of major public health and food safety importance. Among them, T. asiatica is a newly recognized species that split from T. saginata via an intermediate host switch ∼1.14 Myr ago. Here we report the 169- and 168-Mb draft genomes of T. saginata and T. asiatica. Comparative analysis reveals that high rates of gene duplications and functional diversifications might have partially driven the divergence between T. asiatica and T. saginata. We observe accelerated evolutionary rates, adaptive evolutions in homeostasis regulation, tegument maintenance and lipid uptakes, and differential/specialized gene family expansions in T. asiatica that may favour its hepatotropism in the new intermediate host. We also identify potential targets for developing diagnostic or intervention tools against human tapeworms. These data provide new insights into the evolution of Taenia parasites, particularly the recent speciation of T. asiatica

    Defining Key Genes Regulating Morphogenesis of Apocrine Sweat Gland in Sheepskin

    Get PDF
    The apocrine sweat gland is a unique skin appendage in humans compared to mouse and chicken models. The absence of apocrine sweat glands in chicken and murine skin largely restrains further understanding of the complexity of human skin biology and skin diseases, like hircismus. Sheep may serve as an additional system for skin appendage investigation owing to the distributions and histological similarities between the apocrine sweat glands of sheep trunk skin and human armpit skin. To understand the molecular mechanisms underlying morphogenesis of apocrine sweat glands in sheepskin, transcriptome analyses were conducted to reveal 1631 differentially expressed genes that were mainly enriched in three functional groups (cellular component, molecular function and biological process), particularly in gland, epithelial, hair follicle and skin development. There were 7 Gene Ontology (GO) terms enriched in epithelial cell migration and morphogenesis of branching epithelium that were potentially correlated with the wool follicle peg elongation. An additional 5 GO terms were enriched in gland morphogenesis (20 genes), gland development (42 genes), salivary gland morphogenesis and development (8 genes), branching involved in salivary gland morphogenesis (6 genes) and mammary gland epithelial cell differentiation (4 genes). The enriched gland-related genes and two Kyoto Encyclopedia of Genes and Genomes pathway genes (WNT and TGF-β) were potentially involved in the induction of apocrine sweat glands. Genes named BMPR1A, BMP7, SMAD4, TGFB3, WIF1, and WNT10B were selected to validate transcript expression by qRT-PCR. Immunohistochemistry was performed to localize markers for hair follicle (SOX2), skin fibroblast (PDGFRB), stem cells (SOX9) and BMP signaling (SMAD5) in sheepskin. SOX2 and PDGFRB were absent in apocrine sweat glands. SOX9 and SMAD5 were both observed in precursor cells of apocrine sweat glands and later in gland ducts. These results combined with the upregulation of BMP signaling genes indicate that apocrine sweat glands were originated from outer root sheath of primary wool follicle and positively regulated by BMP signaling. This report established the primary network regulating early development of apocrine sweat glands in sheepskin and will facilitate the further understanding of histology and pathology of apocrine sweat glands in human and companion animal skin