In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning

In this note, we explore inference-time alignment through in-context learning. We consider a vanilla pretrained language model Llama-2 before any fine-tuning and retrieve an average of 9 demonstration alignment examples when the model is prompted to follow chat-style instructions. Compared to direct prompting, the in-context alignment without changing model weights leads to a 7x increase in win-rate w.r.t. the text-davinci-003 model from OpenAI, making the vanilla language model comparable to strong baselines with alignment fine-tuning

Advances in liposome research in the field of antitumor

Liposomes, as biocompatible and safe nanocarriers with easily modified surfaces, can be well used in the field of antitumor. Their targeting properties have contributed to the reduction of drug dosage and non-target cell toxicity. To further exploit the targeting properties of liposomes, passive targeting liposomes, active targeting liposomes and physicochemical targeting liposomes have been constructed by surface modification. This paper summarizes the mechanisms of action of different types of targeted liposomes and describes the specific role of liposomes in overcoming tumor resistance, immunotherapy and helping drugs to cross the blood-brain barrier, and summarizes the current development issues and future directions

The moderation effect of social factors on marketing factors in consumer research

Consumer research tends to isolate the impact of marketing and social factors. Little has been done to include both. This paper aimed to find out what would happen when these two sets of factors are included. Two models were built in this paper, Model I with the marketing factors only and Model II with both the marketing and social factors. Data was collected in Ireland among more than 1473 transition year students in a personal survey regarding their willingness to learn Chinese. Data were analysed by using structural equation modelling (SEM). Results showed the two social variables, acculturation and intergenerational influence, significantly consolidated the effects of brand awareness on both brand trust and purchase behaviour; and they diminished the impact of brand trust on purchase behaviour. Empirical evidence suggested the worthiness for marketing researchers to examine both marketing and social factors in consumer research

SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control

Despite the growing success of diffusion models in continuous-valued domains (e.g., images), similar efforts for discrete domains such as text have yet to match the performance of autoregressive language models. In this work, we present SSD-LM -- a diffusion-based language model with two key design choices. First, SSD-LM is semi-autoregressive, iteratively generating blocks of text, allowing for flexible output length at decoding time while enabling local bidirectional context updates. Second, it is simplex-based, performing diffusion on the natural vocabulary space rather than a learned latent space, allowing us to incorporate classifier guidance and modular control using off-the-shelf classifiers without any adaptation. We evaluate SSD-LM on unconstrained text generation benchmarks, and show that it matches or outperforms strong autoregressive GPT-2 models across standard quality and diversity metrics, while vastly outperforming diffusion-based baselines. On controlled text generation, SSD-LM also outperforms competitive baselines, with an extra advantage in modularity.Comment: ACL 202

miR-181a Post-Transcriptionally Downregulates Oncogenic RalA and Contributes to Growth Inhibition and Apoptosis in Chronic Myelogenous Leukemia (CML)

MicroRNAs (miRNAs) are a class of short RNAs that regulate gene expression through either translational repression or mRNA cleavage. miRNA-181a (miR-181a), one of the many miRNAs conserved among vertebrates, is differentially expressed in a variety of leukemia. However, its function in leukemia, particularly chronic myelogenous leukemia (CML), is poorly understood. Here we have reported the identification of miR-181a targets by combining TargetScan software prediction and expression profiling through overexpression of miR-181a mimic in leukemic K562 cells. Four overlapping genes were found to be the likely targets of miR-181a. Among the four genes, RalA is a downstream molecule of bcr-abl fusion protein in ras signaling pathway. However, its role in CML remains elusive. Luciferase reporter and Western blot assays confirmed that RalA is a direct target of miR-181a. overexpression of miR-181a effectively suppresses cell growth and induces G2-phase arrest and apoptosis partially by targeting RalA in leukemic K562 cells. Using the KEGG database combined with recent publications, downstream signaling pathway of RalA was graphed by cytoscape software. Therefore, our study is the first to report that RalA is directly regulated by miR-181a and plays an important role in CML. The approach of computational prediction combined with expression profiling might be valuable for the identification of miRNA targets in animal

Understanding In-Context Learning via Supportive Pretraining Data

In-context learning (ICL) improves language models' performance on a variety of NLP tasks by simply demonstrating a handful of examples at inference time. It is not well understood why ICL ability emerges, as the model has never been specifically trained on such demonstrations. Unlike prior work that explores implicit mechanisms behind ICL, we study ICL via investigating the pretraining data. Specifically, we first adapt an iterative, gradient-based approach to find a small subset of pretraining data that supports ICL. We observe that a continued pretraining on this small subset significantly improves the model's ICL ability, by up to 18%. We then compare the supportive subset constrastively with random subsets of pretraining data and discover: (1) The supportive pretraining data to ICL do not have a higher domain relevance to downstream tasks. (2) The supportive pretraining data have a higher mass of rarely occurring, long-tail tokens. (3) The supportive pretraining data are challenging examples where the information gain from long-range context is below average, indicating learning to incorporate difficult long-range context encourages ICL. Our work takes a first step towards understanding ICL via analyzing instance-level pretraining data. Our insights have a potential to enhance the ICL ability of language models by actively guiding the construction of pretraining data in the future.Comment: ACL 202