Large Language Models Are Semi-Parametric Reinforcement Learning Agents

Inspired by the insights in cognitive science with respect to human memory and reasoning mechanism, a novel evolvable LLM-based (Large Language Model) agent framework is proposed as REMEMBERER. By equipping the LLM with a long-term experience memory, REMEMBERER is capable of exploiting the experiences from the past episodes even for different task goals, which excels an LLM-based agent with fixed exemplars or equipped with a transient working memory. We further introduce Reinforcement Learning with Experience Memory (RLEM) to update the memory. Thus, the whole system can learn from the experiences of both success and failure, and evolve its capability without fine-tuning the parameters of the LLM. In this way, the proposed REMEMBERER constitutes a semi-parametric RL agent. Extensive experiments are conducted on two RL task sets to evaluate the proposed framework. The average results with different initialization and training sets exceed the prior SOTA by 4% and 2% for the success rate on two task sets and demonstrate the superiority and robustness of REMEMBERER

ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought

Recently Large Language Models (LLMs) have been proven to have strong abilities in various domains and tasks. We study the problem of prompt designing in the text-to-SQL task and attempt to improve the LLMs' reasoning ability when generating SQL queries. Besides the trivial few-shot in-context learning setting, we design our chain-of-thought (CoT) prompt with a similar method to schema linking. We provide a method named ACT-SQL to automatically generate auto-CoT exemplars and thus the whole process doesn't need manual labeling. Our approach is cost-saving since we only use the LLMs' API call once when generating one SQL query. Furthermore, we extend our in-context learning method to the multi-turn text-to-SQL task. The experiment results show that the LLMs' performance can benefit from our ACT-SQL approach. Our approach achieves SOTA performance on the Spider dev set among existing in-context learning approaches

Zhuang handicraft in the Baise area: The gendered cultural significance of Mo-Mie

This research is of interest at local, Chinese national, and global levels. Locally, I examine the cultural production of the Zhuang people around Baise in the Guangxi Zhuang Autonomous Region, the Zhuang being one of the 56 official nationalities in China. Globally, there is increasing interest in China and its make-up, including the specific national cultural components. I attempt to contribute to clarity in attaching aesthetic and social value to cultural production that is often carried out by women and largely categorised as ‘craft’, rather than ‘art’. I adopt an empirical approach by engaging with the producers of such work and examine their relationship to the processes of its consumption. I found that the traditional family handcraft of Mo-Mie had been largely forgotten until coming back into focus in 2009 due to government efforts to explore China's national cultures. This short piece of research cannot ‘solve’ the problems that face the history of art. My attempt, in a limited way, is continue the process of examination and appreciation of unique forms of cultural production that exist, in order that we might begin to develop wider conclusions about the place of such creativity in appreciation of human heritage

Tissue factor pathway inhibitor-2 was repressed by CpG hypermethylation through inhibition of KLF6 binding in highly invasive breast cancer cells

<p>Abstract</p> <p>Background</p> <p>Tissue factor pathway inhibitor-2 (TFPI-2) is a matrix-associated Kunitz inhibitor that inhibits plasmin and trypsin-mediated activation of zymogen matrix metalloproteinases involved in tumor progression, invasion and metastasis. Here, we have investigated the mechanism of DNA methylation on the repression of TFPI-2 in breast cancer cell lines.</p> <p>Results</p> <p>We found that both protein and mRNA of TFPI-2 could not be detected in highly invasive breast cancer cell line MDA-MB-435. To further investigate the mechanism of TFPI-2 repression in breast cancer cells, 1.5 Kb TFPI-2 promoter was cloned, and several genetic variations were detected, but the promoter luciferase activities were not affected by the point mutation in the promoter region and the phenomena was further supported by deleted mutation. Scan mutation and informatics analysis identified a potential KLF6 binding site in TFPI-2 promoter. It was revealed, by bisulfite modified sequence, that the CpG island in TFPI-2 promoter region was hypermethylated in MDA-MB-435. Finally, using EMSA and ChIP assay, we demonstrated that the CpG methylation in the binding site of KLF-6 diminished the binding of KLF6 to TFPI-2 promoter.</p> <p>Conclusion</p> <p>In this study, we found that the CpG islands in TFPI-2 promoter was hypermethylated in highly invasive breast cancer cell line, and DNA methylation in the entire promoter region caused TFPI-2 repression by inducing inactive chromatin structure and decreasing KLF6 binding to its DNA binding sequence.</p

ASTormer: An AST Structure-aware Transformer Decoder for Text-to-SQL

Text-to-SQL aims to generate an executable SQL program given the user utterance and the corresponding database schema. To ensure the well-formedness of output SQLs, one prominent approach adopts a grammar-based recurrent decoder to produce the equivalent SQL abstract syntax tree (AST). However, previous methods mainly utilize an RNN-series decoder, which 1) is time-consuming and inefficient and 2) introduces very few structure priors. In this work, we propose an AST structure-aware Transformer decoder (ASTormer) to replace traditional RNN cells. The structural knowledge, such as node types and positions in the tree, is seamlessly incorporated into the decoder via both absolute and relative position embeddings. Besides, the proposed framework is compatible with different traversing orders even considering adaptive node selection. Extensive experiments on five text-to-SQL benchmarks demonstrate the effectiveness and efficiency of our structured decoder compared to competitive baselines

A BiRGAT Model for Multi-intent Spoken Language Understanding with Hierarchical Semantic Frames

Previous work on spoken language understanding (SLU) mainly focuses on single-intent settings, where each input utterance merely contains one user intent. This configuration significantly limits the surface form of user utterances and the capacity of output semantics. In this work, we first propose a Multi-Intent dataset which is collected from a realistic in-Vehicle dialogue System, called MIVS. The target semantic frame is organized in a 3-layer hierarchical structure to tackle the alignment and assignment problems in multi-intent cases. Accordingly, we devise a BiRGAT model to encode the hierarchy of ontology items, the backbone of which is a dual relational graph attention network. Coupled with the 3-way pointer-generator decoder, our method outperforms traditional sequence labeling and classification-based schemes by a large margin

Learning from Easy to Complex: Adaptive Multi-curricula Learning for Neural Dialogue Generation

Current state-of-the-art neural dialogue systems are mainly data-driven and are trained on human-generated responses. However, due to the subjectivity and open-ended nature of human conversations, the complexity of training dialogues varies greatly. The noise and uneven complexity of query-response pairs impede the learning efficiency and effects of the neural dialogue generation models. What is more, so far, there are no unified dialogue complexity measurements, and the dialogue complexity embodies multiple aspects of attributes---specificity, repetitiveness, relevance, etc. Inspired by human behaviors of learning to converse, where children learn from easy dialogues to complex ones and dynamically adjust their learning progress, in this paper, we first analyze five dialogue attributes to measure the dialogue complexity in multiple perspectives on three publicly available corpora. Then, we propose an adaptive multi-curricula learning framework to schedule a committee of the organized curricula. The framework is established upon the reinforcement learning paradigm, which automatically chooses different curricula at the evolving learning process according to the learning status of the neural dialogue generation model. Extensive experiments conducted on five state-of-the-art models demonstrate its learning efficiency and effectiveness with respect to 13 automatic evaluation metrics and human judgments.Comment: Accepted to AAAI 202

Probing Product Description Generation via Posterior Distillation

In product description generation (PDG), the user-cared aspect is critical for the recommendation system, which can not only improve user's experiences but also obtain more clicks. High-quality customer reviews can be considered as an ideal source to mine user-cared aspects. However, in reality, a large number of new products (known as long-tailed commodities) cannot gather sufficient amount of customer reviews, which brings a big challenge in the product description generation task. Existing works tend to generate the product description solely based on item information, i.e., product attributes or title words, which leads to tedious contents and cannot attract customers effectively. To tackle this problem, we propose an adaptive posterior network based on Transformer architecture that can utilize user-cared information from customer reviews. Specifically, we first extend the self-attentive Transformer encoder to encode product titles and attributes. Then, we apply an adaptive posterior distillation module to utilize useful review information, which integrates user-cared aspects to the generation process. Finally, we apply a Transformer-based decoding phase with copy mechanism to automatically generate the product description. Besides, we also collect a large-scare Chinese product description dataset to support our work and further research in this field. Experimental results show that our model is superior to traditional generative models in both automatic indicators and human evaluation