Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning

Fine-tuning large pre-trained language models on various downstream tasks with whole parameters is prohibitively expensive. Hence, Parameter-efficient fine-tuning has attracted attention that only optimizes a few task-specific parameters with the frozen pre-trained model. In this work, we focus on prefix tuning, which only optimizes continuous prefix vectors (i.e. pseudo tokens) inserted into Transformer layers. Based on the observation that the learned syntax and semantics representation varies a lot at different layers, we argue that the adaptive prefix will be further tailored to each layer than the fixed one, enabling the fine-tuning more effective and efficient. Thus, we propose Adaptive Prefix Tuning (APT) to adjust the prefix in terms of both fine-grained token level and coarse-grained layer level with a gate mechanism. Experiments on the SuperGLUE and NER datasets show the effectiveness of APT. In addition, taking the gate as a probing, we validate the efficiency and effectiveness of the variable prefix.Comment: Accepted to ACL 2023 (Main conference

X-ray Observation and Analysis of The Composite Supernova Remnant G327.1-1.1

Based on the data from the observation of the SNR G327.1-1.1 by ASCA and ROSAT, we find that G327.1-1.1 is a composite remnant with both a nonthermal emission component and a diffuse thermal emission component. The nonthermal component is well fitted by a power-law model with photon index about 2.2. This component is attributed to the emission from the synchrotron nebula powered by an undiscovered central pulsar. The thermal component has a temperature of about 0.4 keV. We attribute it to the emission from the shock-heat swept-up ISM. Its age, explosion energy and density of ambient medium are derived from the observed thermal component. Some charactistics about the synchrotron nebula are also derived. We search for the pulsed signal, but has not found it. The soft X-ray(0.4 - 2 keV) and hard X-ray(2 - 10 keV) images are different, but they both elongate in the SE-NW direction. And this X-ray SE-NW elongation is in positional coincidence with the radio ridge in MOST 843MHz radio map. We present a possibility that the X-ray nonthermal emission mainly come from the trail produced by a quickly moving undiscoverd pulsar, and the long radio ridge is formed when the pulsar is moving out of the boundary of the plerionic structure.Comment: 20 pages, 4 Postscript figures, aasms4.sty and psfig.sty, to be published in Astrophysical Journal, January 20, 1999, Vol. 51

A chalcone derivative reactivates latent HIV-1 transcription through activating P-TEFb and promoting Tat-SEC interaction on viral promoter.

The principal barrier to the eradication of HIV/AIDS is the existence of latent viral reservoirs. One strategy to overcome this barrier is to use latency-reversing agents (LRAs) to reactivate the latent proviruses, which can then be eliminated by effective anti-retroviral therapy. Although a number of LRAs have been found to reactivate latent HIV, they have not been used clinically due to high toxicity and poor efficacy. In this study, we report the identification of a chalcone analogue called Amt-87 that can significantly reactivate the transcription of latent HIV provirses and act synergistically with known LRAs such as prostratin and JQ1 to reverse latency. Amt-87 works by activating the human transcriptional elongation factor P-TEFb, a CDK9-cyclin T1 heterodimer that is part of the super elongation complex (SEC) used by the viral encoded Tat protein to activate HIV transcription. Amt-87 does so by promoting the phosphorylation of CDK9 at the T-loop, liberating P-TEFb from the inactive 7SK snRNP, and inducing the formation of the Tat-SEC complex at the viral promoter. Together, our data reveal chalcones as a promising category of compounds that should be further explored to identify effective LRAs for targeted reversal of HIV latency

Reduced tolerance to abiotic stress in transgenic Arabidopsis overexpressing a Capsicum annuum multiprotein bridging factor 1

BACKGROUND: The pepper fruit is the second most consumed vegetable worldwide. However, low temperature affects the vegetative development and reproduction of the pepper, resulting in economic losses. To identify cold-related genes regulated by abscisic acid (ABA) in pepper seedlings, cDNA representational difference analysis was previously performed using a suppression subtractive hybridization method. One of the genes cloned from the subtraction was homologous to Solanum tuberosum MBF1 (StMBF1) encoding the coactivator multiprotein bridging factor 1. Here, we have characterized this StMBF1 homolog (named CaMBF1) from Capsicum annuum and investigated its role in abiotic stress tolerance. RESULTS: Tissue expression profile analysis using quantitative RT-PCR showed that CaMBF1 was expressed in all tested tissues, and high-level expression was detected in the flowers and seeds. The expression of CaMBF1 in pepper seedlings was dramatically suppressed by exogenously supplied salicylic acid, high salt, osmotic and heavy metal stresses. Constitutive overexpression of CaMBF1 in Arabidopsis aggravated the visible symptoms of leaf damage and the electrolyte leakage of cell damage caused by cold stress in seedlings. Furthermore, the expression of RD29A, ERD15, KIN1, and RD22 in the transgenic plants was lower than that in the wild-type plants. On the other hand, seed germination, cotyledon greening and lateral root formation were more severely influenced by salt stress in transgenic lines compared with wild-type plants, indicating that CaMBF1-overexpressing Arabidopsis plants were hypersensitive to salt stress. CONCLUSIONS: Overexpression of CaMBF1 in Arabidopsis displayed reduced tolerance to cold and high salt stress during seed germination and post-germination stages. CaMBF1 transgenic Arabidopsis may reduce stress tolerance by downregulating stress-responsive genes to aggravate the leaf damage caused by cold stress. CaMBF1 may be useful for genetic engineering of novel pepper cultivars in the future

Enhancement of polar phases in PVDF by forming PVDF/SiC nanowire composite

Different contents of silicon carbide (SiC) nanowires were mixed with Poly(vinylidene fluoride) (PVDF) to facilitate the polar phase crystallization. It was shown that the annealing temperature and SiC content affected on the phase and crystalline structures of PVDF/SiC samples. Furthermore, the addition of SiC nanowire enhanced the transformation of non-polar α phase to polar phases and increased the relative fraction of β phase in PVDF. Due to the nucleating agent mechanism of SiC nanowires, the ion-dipole interaction between the negatively charged surface of SiC nanowires and the positive CH2 groups in PVDF facilitated the formation of polar phases in PVDF

Study on Microclimate Characteristics and Vertical Variation of Potential Evapotranspiration of the Robinia pseudoacacia

With the water and soil conservation forests of Robinia pseudoacacia in the Malian beach of Hongqi farm, Ji County, Shanxi province, as the research object, this study estimated the potential evapotranspiration in the open space outside the forest and at the heights of 3 m, 6 m, and 10 m in the forests with the climate data during 2011-2012 and the upgraded Penman-Monteith formula; then, this study explored the microclimate characteristics inside and outside the Robinia pseudoacacia forest and thereafter revealed the vertical variation rules of potential evapotranspiration of Robinia pseudoacacia. The results indicate that the air temperature, relative humidity, wind speed, and potential evapotranspiration at different heights above the ground surface showed similar changing trends, but with some variation during different periods. In addition, the weather also had impacts on the potential evapotranspiration. In April, July, and October, the change of potential evapotranspiration of the Robinia pseudoacacia forests showed a bimodal curve in the sunny days and rainy days, while it showed a single-peak curve with quite small fluctuation in the rainy days. However, it showed a single-peak pattern even in the sunny days in January, and it showed no fluctuation in the snowy days in January

Dynamics analysis of a novel two-translation parallel mechanism

In this paper, we take a novel two-translation parallel mechanism based on variable length parallelogram structure as the research object, and carry out the research on the dynamic characteristics of the mechanism. Based on the principle of virtual work, the dynamic model of the new two-translation parallel mechanism is established. By numerically solving the dynamic model, the dynamic behavior of the new mechanism with impact characteristics is found, which provides a theoretical basis for optimizing this new mechanism