Hybrid Reinforcement Learning with Expert State Sequences

Existing imitation learning approaches often require that the complete demonstration data, including sequences of actions and states, are available. In this paper, we consider a more realistic and difficult scenario where a reinforcement learning agent only has access to the state sequences of an expert, while the expert actions are unobserved. We propose a novel tensor-based model to infer the unobserved actions of the expert state sequences. The policy of the agent is then optimized via a hybrid objective combining reinforcement learning and imitation learning. We evaluated our hybrid approach on an illustrative domain and Atari games. The empirical results show that (1) the agents are able to leverage state expert sequences to learn faster than pure reinforcement learning baselines, (2) our tensor-based action inference model is advantageous compared to standard deep neural networks in inferring expert actions, and (3) the hybrid policy optimization objective is robust against noise in expert state sequences.Comment: AAAI 2019; https://github.com/XiaoxiaoGuo/tensor4r

One-Shot Relational Learning for Knowledge Graphs

Knowledge graphs (KGs) are the key components of various natural language processing applications. To further expand KGs' coverage, previous studies on knowledge graph completion usually require a large number of training instances for each relation. However, we observe that long-tail relations are actually more common in KGs and those newly added relations often do not have many known triples for training. In this work, we aim at predicting new facts under a challenging setting where only one training instance is available. We propose a one-shot relational learning framework, which utilizes the knowledge extracted by embedding models and learns a matching metric by considering both the learned embeddings and one-hop graph structures. Empirically, our model yields considerable performance improvements over existing embedding models, and also eliminates the need of re-training the embedding models when dealing with newly added relations.Comment: EMNLP 201

A Petunia homeodomain-leucine zipper protein, PhHD-Zip, plays an important role in flower senescence.

Flower senescence is initiated by developmental and environmental signals, and regulated by gene transcription. A homeodomain-leucine zipper transcription factor, PhHD-Zip, is up-regulated during petunia flower senescence. Virus-induced gene silencing of PhHD-Zip extended flower life by 20% both in unpollinated and pollinated flowers. Silencing PhHD-Zip also dramatically reduced ethylene production and the abundance of transcripts of genes involved in ethylene (ACS, ACO), and ABA (NCED) biosynthesis. Abundance of transcripts of senescence-related genes (SAG12, SAG29) was also dramatically reduced in the silenced flowers. Over-expression of PhHD-Zip accelerated petunia flower senescence. Furthermore, PhHD-Zip transcript abundance in petunia flowers was increased by application of hormones (ethylene, ABA) and abiotic stresses (dehydration, NaCl and cold). Our results suggest that PhHD-Zip plays an important role in regulating petunia flower senescence

A basic helix-loop-helix transcription factor, PhFBH4, regulates flower senescence by modulating ethylene biosynthesis pathway in petunia.

The basic helix-loop-helix (bHLH) transcription factors (TFs) play important roles in regulating multiple biological processes in plants. However, there are few reports about the function of bHLHs in flower senescence. In this study, a bHLH TF, PhFBH4, was found to be dramatically upregulated during flower senescence. Transcription of PhFBH4 is induced by plant hormones and abiotic stress treatments. Silencing of PhFBH4 using virus-induced gene silencing or an antisense approach extended flower longevity, while transgenic petunia flowers with an overexpression construct showed a reduction in flower lifespan. Abundance of transcripts of senescence-related genes (SAG12, SAG29) was significantly changed in petunia PhFBH4 transgenic flowers. Furthermore, silencing or overexpression of PhFBH4 reduced or increased, respectively, transcript abundances of important ethylene biosynthesis-related genes, ACS1 and ACO1, thereby influencing ethylene production. An electrophoretic mobility shift assay showed that the PhFBH4 protein physically interacted with the G-box cis-element in the promoter of ACS1, suggesting that ACS1 was a direct target of the PhFBH4 protein. In addition, ectopic expression of this gene altered plant development including plant height, internode length, and size of leaves and flowers, accompanied by alteration of transcript abundance of the gibberellin biosynthesis-related gene GA2OX3. Our results indicate that PhFBH4 plays an important role in regulating plant growth and development through modulating the ethylene biosynthesis pathway

Quantitative profiling of selective Sox/POU pairing on hundreds of sequences in parallel by Coop-seq

© The Author(s) 2016. Published by Oxford University Press on behalf of Nucleic Acids Research. Cooperative binding of transcription factors is known to be important in the regulation of gene expression programs conferring cellular identities. However, current methods to measure cooperativity parameters have been laborious and therefore limited to studying only a few sequence variants at a time. We developed Coop-seq (cooperativity by sequencing) that is capable of efficiently and accurately determining the cooperativity parameters for hundreds of different DNA sequences in a single experiment. We apply Coop-seq to 12 dimer pairs from the Sox and POU families of transcription factors using 324 unique sequences with changed half-site orientation, altered spacing and discrete randomization within the binding elements. The study reveals specific dimerization profiles of different Sox factors with Oct4. By contrast, Oct4 and the three neural class III POU factors Brn2, Brn4 and Oct6 assemble with Sox2 in a surprisingly indistinguishable manner. Two novel half-site configurations can support functional Sox/Oct dimerization in addition to known composite motifs. Moreover, Coop-seq uncovers a nucleotide switch within the POU half-site when spacing is altered, which is mirrored in genomic loci bound by Sox2/Oct4 complexes.Link_to_subscribed_fulltex