Making Small Language Models Better Multi-task Learners with Mixture-of-Task-Adapters

Recently, Large Language Models (LLMs) have achieved amazing zero-shot learning performance over a variety of Natural Language Processing (NLP) tasks, especially for text generative tasks. Yet, the large size of LLMs often leads to the high computational cost of model training and online deployment. In our work, we present ALTER, a system that effectively builds the multi-tAsk Learners with mixTure-of-task-adaptERs upon small language models (with <1B parameters) to address multiple NLP tasks simultaneously, capturing the commonalities and differences between tasks, in order to support domain-specific applications. Specifically, in ALTER, we propose the Mixture-of-Task-Adapters (MTA) module as an extension to the transformer architecture for the underlying model to capture the intra-task and inter-task knowledge. A two-stage training method is further proposed to optimize the collaboration between adapters at a small computational cost. Experimental results over a mixture of NLP tasks show that our proposed MTA architecture and the two-stage training method achieve good performance. Based on ALTER, we have also produced MTA-equipped language models for various domains

Inhibition of SARS Pseudovirus Cell Entry by Lactoferrin Binding to Heparan Sulfate Proteoglycans

It has been reported that lactoferrin (LF) participates in the host immune response against Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV) invasion by enhancing NK cell activity and stimulating neutrophil aggregation and adhesion. We further investigated the role of LF in the entry of SARS pseudovirus into HEK293E/ACE2-Myc cells. Our results reveal that LF inhibits SARS pseudovirus infection in a dose-dependent manner. Further analysis suggested that LF was able to block the binding of spike protein to host cells at 4°C, indicating that LF exerted its inhibitory function at the viral attachment stage. However, LF did not disrupt the interaction of spike protein with angiotensin-converting enzyme 2 (ACE2), the functional receptor of SARS-CoV. Previous studies have shown that LF colocalizes with the widely distributed cell-surface heparan sulfate proteoglycans (HSPGs). Our experiments have also confirmed this conclusion. Treatment of the cells with heparinase or exogenous heparin prevented binding of spike protein to host cells and inhibited SARS pseudovirus infection, demonstrating that HSPGs provide the binding sites for SARS-CoV invasion at the early attachment phase. Taken together, our results suggest that, in addition to ACE2, HSPGs are essential cell-surface molecules involved in SARS-CoV cell entry. LF may play a protective role in host defense against SARS-CoV infection through binding to HSPGs and blocking the preliminary interaction between SARS-CoV and host cells. Our findings may provide further understanding of SARS-CoV pathogenesis and aid in treatment of this deadly disease

Massively parallel reporter perturbation assays uncover temporal regulatory architecture during neural differentiation

神経細胞を作る遺伝子制御の構造を解明. 京都大学プレスリリース. 2022-03-24.Hundreds of gene regulatory motifs cooperate and conflict to make brain cells. 京都大学プレスリリース. 2022-03-24.Gene regulatory elements play a key role in orchestrating gene expression during cellular differentiation, but what determines their function over time remains largely unknown. Here, we perform perturbation-based massively parallel reporter assays at seven early time points of neural differentiation to systematically characterize how regulatory elements and motifs within them guide cellular differentiation. By perturbing over 2, 000 putative DNA binding motifs in active regulatory regions, we delineate four categories of functional elements, and observe that activity direction is mostly determined by the sequence itself, while the magnitude of effect depends on the cellular environment. We also find that fine-tuning transcription rates is often achieved by a combined activity of adjacent activating and repressing elements. Our work provides a blueprint for the sequence components needed to induce different transcriptional patterns in general and specifically during neural differentiation

Study of Dispersion Characteristics of Mercerized Pulp

Mercerized pulp is widely used in the filter paper industry. But the major challenge facing users of the pulp is its difficult dispersion in water. It was found that by applying a suitable degree of beating it was possible to achieve better dispersion than the original pulp. The beating degree before and after beating was almost the same. But the properties of filter paper were greatly improved after beating, especially for the formation index and burst index. The morphology of beaten fibers was analyzed by SEM with both the freeze-drying and air drying sample preparation process. The results showed that the primary cell wall of the beaten mercerized pulp fibers were swollen and partly peeled from the fiber main body after beating, as revealed by micrographs obtained after freeze-drying. The results suggest that the improvement of the fiber dispersion in water was caused by these changes on the fiber surface

Simultaneous Determination of Xanthine and Hypoxanthine Using Polyglycine/rGO-Modified Glassy Carbon Electrode

A novel electrochemical sensor was developed for selective and sensitive determination of xanthine (XT) and hypoxanthine (HX) based on polyglycine (p-Gly) and reduced graphene oxide (rGO) modified glassy carbon electrode (GCE). A mixed dispersion of 7 μL of 5 mM glycine and 1 mg/mL GO was dropped on GCE for the fabrication of p-Gly/rGO/GCE, followed by cyclic voltammetric sweeping in 0.1 M phosphate buffer solution within −0.45~1.85 V at a scanning rate of 100 mV·s−1. The morphological and electrochemical features of p-Gly/rGO/GCE were investigated by scanning electron microscopy and cyclic voltammetry. Under optimal conditions, the linear relationship was acquired for the simultaneous determination of XT and HX in 1–100 μM. The preparation of the electrode was simple and efficient. Additionally, the sensor combined the excellent conductivity of rGO and the polymerization of Gly, demonstrating satisfying simultaneous sensing performance to both XT and HX

A High-Temperature MEMS Surface Fence for Wall-Shear-Stress Measurement in Scramjet Flow

A new variant of MEMS surface fence is proposed for shear-stress estimation under high-speed, high-temperature flow conditions. Investigation of high-temperature resistance including heat-resistant mechanism and process, in conjunction with high-temperature packaging design, enable the sensor to be used in environment up to 400 °C. The packaged sensor is calibrated over a range of ~65 Pa and then used to examine the development of the transient flow of the scramjet ignition process (Mach 2 airflow, stagnation pressure, and a temperature of 0.8 MPa and 950 K, respectively). The results show that the sensor is able to detect the transient flow conditions of the scramjet ignition process including shock impact, flow correction, steady state, and hydrogen off

Massively parallel reporter perturbation assays uncover temporal regulatory architecture during neural differentiation.

Gene regulatory elements play a key role in orchestrating gene expression during cellular differentiation, but what determines their function over time remains largely unknown. Here, we perform perturbation-based massively parallel reporter assays at seven early time points of neural differentiation to systematically characterize how regulatory elements and motifs within them guide cellular differentiation. By perturbing over 2,000 putative DNA binding motifs in active regulatory regions, we delineate four categories of functional elements, and observe that activity direction is mostly determined by the sequence itself, while the magnitude of effect depends on the cellular environment. We also find that fine-tuning transcription rates is often achieved by a combined activity of adjacent activating and repressing elements. Our work provides a blueprint for the sequence components needed to induce different transcriptional patterns in general and specifically during neural differentiation

Error-Based Observer of a Charge Couple Device Tracking Loop for Fast Steering Mirror

The charge couple device (CCD) tracking loop of a fast steering mirror (FSM) is usually used to stabilize line of sight (LOS). High closed-loop bandwidth facilitates good performance. However, low-rate sample and time delay of the CCD greatly limit the high control bandwidth. This paper proposes an error-based observer (EBO) to improve the low-frequency performance of the CCD tracking system. The basic idea is by combining LOS error from the CCD and the controller output to produce the high-gain observer, forwarding into the originally closed-loop control system. This proposed EBO can improve the system both in target tracking and disturbance suppression due to LOS error from the CCD’s sensing of the two signals. From a practical engineering view, the closed-loop stability and robustness of the EBO system are investigated on the condition of gain margin and phase margin of the open-loop transfer function. Two simulations of CCD experiments are provided to verify the benefits of the proposed algorithm

Transcription factor binding site orientation and order are major drivers of gene regulatory activity.

The gene regulatory code and grammar remain largely unknown, precluding our ability to link phenotype to genotype in regulatory sequences. Here, using a massively parallel reporter assay (MPRA) of 209,440 sequences, we examine all possible pair and triplet combinations, permutations and orientations of eighteen liver-associated transcription factor binding sites (TFBS). We find that TFBS orientation and order have a major effect on gene regulatory activity. Corroborating these results with genomic analyses, we find clear human promoter TFBS orientation biases and similar TFBS orientation and order transcriptional effects in an MPRA that tested 164,307 liver candidate regulatory elements. Additionally, by adding TFBS orientation to a model that predicts expression from sequence we improve performance by 7.7%. Collectively, our results show that TFBS orientation and order have a significant effect on gene regulatory activity and need to be considered when analyzing the functional effect of variants on the activity of these sequences