On Efficient Training, Controllability and Compositional Generalization of Insertion-based Language Generators

Auto-regressive language models with the left-to-right generation order have been a predominant paradigm for language generation. Recently, out-of-order text generation beyond the traditional left-to-right paradigm has attracted extensive attention, with a notable variation of insertion-based generation, where a model is used to gradually extend the context into a complete sentence purely with insertion operations. However, since insertion operations disturb the position information of each token, it is often believed that each step of the insertion-based likelihood estimation requires a bi-directional \textit{re-encoding} of the whole generated sequence. This computational overhead prohibits the model from scaling up to generate long, diverse texts such as stories, news articles, and reports. To address this issue, we propose InsNet, an insertion-based sequence model that can be trained as efficiently as traditional transformer decoders while maintaining the same performance as that with a bi-directional context encoder. We evaluate InsNet on story generation and CleVR-CoGENT captioning, showing the advantages of InsNet in several dimensions, including computational costs, generation quality, the ability to perfectly incorporate lexical controls, and better compositional generalization

Long Text Generation via Adversarial Training with Leaked Information

Automatically generating coherent and semantically meaningful text has many applications in machine translation, dialogue systems, image captioning, etc. Recently, by combining with policy gradient, Generative Adversarial Nets (GAN) that use a discriminative model to guide the training of the generative model as a reinforcement learning policy has shown promising results in text generation. However, the scalar guiding signal is only available after the entire text has been generated and lacks intermediate information about text structure during the generative process. As such, it limits its success when the length of the generated text samples is long (more than 20 words). In this paper, we propose a new framework, called LeakGAN, to address the problem for long text generation. We allow the discriminative net to leak its own high-level extracted features to the generative net to further help the guidance. The generator incorporates such informative signals into all generation steps through an additional Manager module, which takes the extracted features of current generated words and outputs a latent vector to guide the Worker module for next-word generation. Our extensive experiments on synthetic data and various real-world tasks with Turing test demonstrate that LeakGAN is highly effective in long text generation and also improves the performance in short text generation scenarios. More importantly, without any supervision, LeakGAN would be able to implicitly learn sentence structures only through the interaction between Manager and Worker.Comment: 14 pages, AAAI 201

Texygen: A Benchmarking Platform for Text Generation Models

We introduce Texygen, a benchmarking platform to support research on open-domain text generation models. Texygen has not only implemented a majority of text generation models, but also covered a set of metrics that evaluate the diversity, the quality and the consistency of the generated texts. The Texygen platform could help standardize the research on text generation and facilitate the sharing of fine-tuned open-source implementations among researchers for their work. As a consequence, this would help in improving the reproductivity and reliability of future research work in text generation.Comment: 4 page

Acyl-CoA Dehydrogenase Drives Heat Adaptation by Sequestering Fatty Acids

Cells adapt to temperature shifts by adjusting levels of lipid desaturation and membrane fluidity. This fundamental process occurs in nearly all forms of life, but its mechanism in eukaryotes is unknown. We discovered that the evolutionarily conserved C. elegans gene acdh-11 (acyl CoAdehydrogenase, ACDH) facilitates heat adaptation by regulating the lipid desaturase FAT-7. Human ACDH deficiency causes the most common inherited disorders of fatty acid oxidation, with syndromes that are exacerbated by hyperthermia. Heat up-regulates acdh-11 expression to decrease fat-7 expression. We solved the high-resolution crystal structure of ACDH-11 and established the molecular basis of its selective and high-affinity binding to C11/C12-chain fatty acids. ACDH-11 sequesters C11/C12-chain fatty acids and prevents these fatty acids from activating nuclear hormone receptors and driving fat-7 expression. Thus, the ACDH-11 pathway drives heat adaptation by linking temperature shifts to regulation of lipid desaturase levels and membrane fluidity via an unprecedented mode of fatty acid signaling.National Institutes of Health (U.S.) (Grants GM24663 and K99HL11665)Charles A. King Trust (Postdoctoral Fellowship

MEPE/OF45 protects cells from DNA damage induced killing via stabilizing CHK1

Matrix extracellular phosphoglycoprotein/osteoblast factor 45 (MEPE/OF45) was cloned in 2000 with functions related to bone metabolism. We identified MEPE/OF45 for the first time as a new co-factor of CHK1 in mammalian cells to protect cells from DNA damage induced killing. We demonstrate here that MEPE/OF45 directly interacts with CHK1. Knocking down MEPE/OF45 decreases CHK1 levels and sensitizes the cells to DNA damage inducers such as ionizing radiation (IR) or camptothicin (CPT)-induced killing. Over-expressing wild-type MEPE/OF45, but not the mutant MEPE/OF45 (depleted the key domain to interact with CHK1) increases CHK1 levels in the cells and increases the resistance of the cells to IR or CPT. MEPE/OF45, interacting with CHK1, increases CHK1 half-life and decreases CHK1 degradation through the ubiquitine-mediated pathway. In addition, the interaction of MEPE/OF45 with CHK1 decreases CHK1 levels in the ubiquitin E3 ligases (Cul1 and Cul4A) complex, which suggests that MEPE/OF45 competes with the ubiquitin E3 ligases binding to CHK1 and thus decreases CHK1 from ubiquitin-mediated proteolysis. These findings reveal an important role of MEPE/OF45 in protecting cells from DNA damage induced killing through stabilizing CHK1, which would provide MEPE/OF45 as a new target for sensitizing tumor cells to radiotherapy or chemotherapy