NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework

Pretrained language models have become the standard approach for many NLP tasks due to strong performance, but they are very expensive to train. We propose a simple and efficient learning framework, TLM, that does not rely on large-scale pretraining. Given some labeled task data and a large general corpus, TLM uses task data as queries to retrieve a tiny subset of the general corpus and jointly optimizes the task objective and the language modeling objective from scratch. On eight classification datasets in four domains, TLM achieves results better than or similar to pretrained language models (e.g., RoBERTa-Large) while reducing the training FLOPs by two orders of magnitude. With high accuracy and efficiency, we hope TLM will contribute to democratizing NLP and expediting its development.Comment: 14 pages, 5 figure

2'-Fucosyllactose Remits Colitis-Induced Liver Oxygen Stress through the Gut-Liver-Metabolites Axis.

peer reviewedLiver oxygen stress is one of the main extraintestinal manifestations of colitis and 5% of cases develop into a further liver injury and metabolic disease. 2′-fucosyllactose (2′-FL), a main member of human milk oligosaccharides (HMOs), has been found to exert efficient impacts on remitting colitis. However, whether 2′-FL exerts the function to alleviate colitis-induced liver injury and how 2′-FL influences the metabolism via regulating gut microbiota remain unknown. Herein, in our study, liver oxygen stress was measured by measuring liver weight and oxygen-stress-related indicators. Then, 16S full-length sequencing analysis and non-target metabolome in feces were performed to evaluate the overall responses of metabolites and intestinal bacteria after being treated with 2′-FL (400 mg/kg b.w.) in colitis mice. The results showed that, compared with the control group, the liver weight of colitis mice was significantly decreased by 18.30% (p < 0.05). After 2′-FL treatment, the liver weight was significantly increased by 12.65% compared with colitis mice (p < 0.05). Meanwhile, they exhibited higher levels of oxidation in liver tissue with decreasing total antioxidant capacity (T-AOC) (decreased by 17.15%) and glutathione (GSH) levels (dropped by 22.68%) and an increasing malondialdehyde (MDA) level (increased by 36.24%), and 2′-FL treatment could reverse those tendencies. Full-length 16S rRNA sequencing revealed that there were 39 species/genera differentially enriched in the control, dextran sulphate sodium (DSS), and DSS + 2′-FL groups. After treatment with 2′-FL, the intestinal metabolic patterns, especially glycometabolism and the lipid-metabolism-related process, in DSS mice were strikingly altered with 33 metabolites significantly down-regulated and 26 metabolites up-regulated. Further analysis found DSS induced a 40.01%, 41.12%, 43.81%, and 39.86% decline in acetic acid, propionic acid, butyric acid, and total short chain fatty acids (SCFAs) in colitis mice (all p < 0.05), respectively, while these were up-regulated to different degrees in the DSS + 2′-FL group. By co-analyzing the data of gut microbiota and metabolites, glycometabolism and lipid-metabolism-associated metabolites exhibited strong positive/negative relationships with Akkermansia_muciniphila (all p < 0.01) and Paraprevotella spp. (all p < 0.01), suggesting that the two species might play crucial roles in the process of 2′-FL alleviating colitis-induced liver oxygen stress. In conclusion, in the gut−liver−microbiotas axis, 2′-FL mediated in glucose and lipid-related metabolism and alleviated liver oxygen stress via regulating gut microbiota in the DSS-induced colitis model. The above results provide a new perspective to understand the probiotic function of 2′-FL

High yield expression of an AHL-lactonase from Bacillus sp. B546 in Pichia pastoris and its application to reduce Aeromonas hydrophila mortality in aquaculture

<p>Abstract</p> <p>Background</p> <p><it>Aeromonas hydrophila </it>is a serious pathogen and can cause hemorrhagic septicemia in fish. To control this disease, antibiotics and chemicals are widely used which can consequently result in "superbugs" and chemical accumulation in the food chain. Though vaccine against <it>A. hydrophila </it>is available, its use is limited due to multiple serotypes of this pathogen and problems of safety and efficacy. Another problem with vaccination is the ability to apply it to small fish especially in high numbers. In this study, we tried a new way to attenuate the <it>A. hydrophila </it>infection by using a quorum quenching strategy with a recombinant AHL-lactonase expressed in <it>Pichia pastoris</it>.</p> <p>Results</p> <p>The AHL-lactonase (AiiA_B546) from <it>Bacillus </it>sp. B546 was produced extracellularly in <it>P. pastoris </it>with a yield of 3,558.4 ± 81.3 U/mL in a 3.7-L fermenter when using 3-oxo-C8-HSL as the substrate. After purification with a HiTrap Q Sepharose column, the recombinant homogenous protein showed a band of 33.6 kDa on SDS-PAGE, higher than the calculated molecular mass (28.14 kDa). Deglycosylation of AiiA_B546with Endo H confirmed the occurrence of <it>N</it>-glycosylation. The purified recombinant AiiA_B546showed optimal activity at pH 8.0 and 20°C, exhibited excellent stability at pH 8.0-12.0 and thermal stability at 70°C, was firstly confirmed to be significantly protease-resistant, and had wide substrate specificity. In application test, when co-injected with A. <it>hydrophila </it>in common carp, recombinant AiiA_B546decreased the mortality rate and delayed the mortality time of fish.</p> <p>Conclusions</p> <p>Our results not only indicate the possibility of mass-production of AHL-lactonase at low cost, but also open up a promising foreground of application of AHL-lactonase in fish to control <it>A. hydrophila </it>disease by regulating its virulence. To our knowledge, this is the first report on heterologous expression of AHL-lactonase in <it>P. pastoris </it>and attenuating <it>A. hydrophila </it>virulence by co-injection with AHL-lactonase.</p

Chemical constituents from Ampelopsis sinica var. hancei prevent liver damage

Antihepatotoxic chemical constituents from the roots of Ampelopsis sinica var. hancei (pl.) W.T. Wang was investigated- Chromatography was used to isolate chemical constituents and their structures were elucidated on the basis of spectroscopic analysis. Antihepatotoxic activity of these compounds in rats was carried out after the establishment of CCl4 induced liver injury. Phytochemical investigation on the roots of Ampelopsis sinica var. hancei (pl.) W. T. Wang resulted in the isolation of eight compounds including β-sitosterol (1), β-daucosterol (2), lupeol (3), trans-resveratrol (4), piceid (5), gallic acid (6), n-butyl gallate (7) and (+)-catechin (8). Rats treated with the compounds 6-8 showed significant (p < 0.05) protection of liver as evidence from normal AST and ALT levels. LDH levels were significantly (p < 0.05) reduced by the treatment with the compounds 5, 7 and 8. In addition, MDA levels were significantly (p < 0.05) increased with gallic acid (6) and (+)-catechin (8). All the chemical constituents were isolated from Ampelopsis sinica var. hancei (pl.) W.T. Wang for the first time. Compounds 5-8 showed significantly antihepatotoxtic activity in CCl4 -induced liver damage rats.Colegio de Farmacéuticos de la Provincia de Buenos Aire