Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages

Pre-trained multilingual language models underpin a large portion of modern NLP tools outside of English. A strong baseline for specializing these models for specific languages is Language-Adaptive Pre-Training (LAPT). However, retaining a large cross-lingual vocabulary and embedding matrix comes at considerable excess computational cost during adaptation. In this study, we propose several simple techniques to replace a cross-lingual vocabulary with a compact, language-specific one. Namely, we address strategies for re-initializing the token embedding matrix after vocabulary specialization. We then provide a systematic experimental comparison of our techniques, in addition to the recently-proposed Focus method. We demonstrate that: 1) Embedding-replacement techniques in the monolingual transfer literature are inadequate for adapting multilingual models. 2) Replacing cross-lingual vocabularies with smaller specialized ones provides an efficient method to improve performance in low-resource languages. 3) Simple embedding re-initialization techniques based on script-wise sub-distributions rival techniques such as Focus, which rely on similarity scores obtained from an auxiliary model

Activation of Cytosolic Phospholipase A\u3csub\u3e2\u3c/sub\u3eα in Resident Peritoneal Macrophages by Listeria Monocytogenes Involves Listeriolysin O and TLR2

Eicosanoid production by macrophages is an early response to microbial infection that promotes acute inflammation. The intracellular pathogen Listeria monocytogenes stimulates arachidonic acid release and eicosanoid production from resident mouse peritoneal macrophages through activation of group IVA cytosolic phospholipase A2 (cPLA2α). The ability of wild type L. monocytogenes (WTLM) to stimulate arachidonic acid release is partially dependent on the virulence factor listeriolysin O; however, WTLM and L. monocytogenes lacking listeriolysin O (ΔhlyLM) induce similar levels of cyclooxygenase 2. Arachidonic acid release requires activation of MAPKs by WTLM and ΔhlyLM. The attenuated release of arachidonic acid that is observed in TLR2-/- and MyD88-/- macrophages infected with WTLM and ΔhlyLM correlates with diminished MAPK activation. WTLM but not ΔhlyLM increases intracellular calcium, which is implicated in regulation of cPLA2α. Prostaglandin E2, prostaglandin I 2, and leukotriene C4 are produced by cPLA 2α+/+ but not cPLA2α-/- macrophages in response to WTLM and ΔhlyLM. Tumor necrosis factor (TNF)-α production is significantly lower in cPLA2α +/+ than in cPLA2α-/- macrophages infected with WTLM and ΔhlyLM. Treatment of infected cPLA 2α+/+ macrophages with the cyclooxygenase inhibitor indomethacin increases TNFα production to the level produced by cPLA 2α-/- macrophages implicating prostaglandins in TNFα down-regulation. Therefore activation of cPLA2α in macrophages may impact immune responses to L. monocytogenes

Insulin regulates carboxypeptidase E by modulating translation initiation scaffolding protein eIF4G1 in pancreatic β cells

Insulin resistance, hyperinsulinemia, and hyperproinsulinemia occur early in the pathogenesis of type 2 diabetes (T2D). Elevated levels of proinsulin and proinsulin intermediates are markers of β-cell dysfunction and are strongly associated with development of T2D in humans. However, the mechanism(s) underlying β-cell dysfunction leading to hyperproinsulinemia is poorly understood. Here, we show that disruption of insulin receptor (IR) expression in β cells has a direct impact on the expression of the convertase enzyme carboxypeptidase E (CPE) by inhibition of the eukaryotic translation initiation factor 4 gamma 1 translation initiation complex scaffolding protein that is mediated by the key transcription factors pancreatic and duodenal homeobox 1 and sterol regulatory element-binding protein 1, together leading to poor proinsulin processing. Reexpression of IR or restoring CPE expression each independently reverses the phenotype. Our results reveal the identity of key players that establish a previously unknown link between insulin signaling, translation initiation, and proinsulin processing, and provide previously unidentified mechanistic insight into the development of hyperproinsulinemia in insulin-resistant states

Status of the differential transformation method

Further to a recent controversy on whether the differential transformation method (DTM) for solving a differential equation is purely and solely the traditional Taylor series method, it is emphasized that the DTM is currently used, often only, as a technique for (analytically) calculating the power series of the solution (in terms of the initial value parameters). Sometimes, a piecewise analytic continuation process is implemented either in a numerical routine (e.g., within a shooting method) or in a semi-analytical procedure (e.g., to solve a boundary value problem). Emphasized also is the fact that, at the time of its invention, the currently-used basic ingredients of the DTM (that transform a differential equation into a difference equation of same order that is iteratively solvable) were already known for a long time by the "traditional"-Taylor-method users (notably in the elaboration of software packages --numerical routines-- for automatically solving ordinary differential equations). At now, the defenders of the DTM still ignore the, though much better developed, studies of the "traditional"-Taylor-method users who, in turn, seem to ignore similarly the existence of the DTM. The DTM has been given an apparent strong formalization (set on the same footing as the Fourier, Laplace or Mellin transformations). Though often used trivially, it is easily attainable and easily adaptable to different kinds of differentiation procedures. That has made it very attractive. Hence applications to various problems of the Taylor method, and more generally of the power series method (including noninteger powers) has been sketched. It seems that its potential has not been exploited as it could be. After a discussion on the reasons of the "misunderstandings" which have caused the controversy, the preceding topics are concretely illustrated.Comment: To appear in Applied Mathematics and Computation, 29 pages, references and further considerations adde

Expression of the Splicing Factor Gene SFRS10 Is Reduced in Human Obesity and Contributes to Enhanced Lipogenesis

SummaryAlternative mRNA splicing provides transcript diversity and may contribute to human disease. We demonstrate that expression of several genes regulating RNA processing is decreased in both liver and skeletal muscle of obese humans. We evaluated a representative splicing factor, SFRS10, downregulated in both obese human liver and muscle and in high-fat-fed mice, and determined metabolic impact of reduced expression. SFRS10-specific siRNA induces lipogenesis and lipid accumulation in hepatocytes. Moreover, Sfrs10 heterozygous mice have increased hepatic lipogenic gene expression, VLDL secretion, and plasma triglycerides. We demonstrate that LPIN1, a key regulator of lipid metabolism, is a splicing target of SFRS10; reduced SFRS10 favors the lipogenic β isoform of LPIN1. Importantly, LPIN1β-specific siRNA abolished lipogenic effects of decreased SFRS10 expression. Together, our results indicate that reduced expression of SFRS10, as observed in tissues from obese humans, alters LPIN1 splicing, induces lipogenesis, and therefore contributes to metabolic phenotypes associated with obesity

Intergenerational Transmission of Glucose Intolerance and Obesity by In Utero Undernutrition in Mice

OBJECTIVE—Low birth weight (LBW) is associated with increased risk of obesity, diabetes, and cardiovascular disease during adult life. Moreover, this programmed disease risk can progress to subsequent generations. We previously described a mouse model of LBW, produced by maternal caloric undernutrition (UN) during late gestation. LBW offspring (F1-UN generation) develop progressive obesity and impaired glucose tolerance (IGT) with aging. We aimed to determine whether such metabolic phenotypes can be transmitted to subsequent generations in an experimental model, even in the absence of altered nutrition during the second pregnancy

Immunological and Cardiometabolic Risk Factors in the Prediction of Type 2 Diabetes and Coronary Events: MONICA/KORA Augsburg Case-Cohort Study

BACKGROUND: This study compares inflammation-related biomarkers with established cardiometabolic risk factors in the prediction of incident type 2 diabetes and incident coronary events in a prospective case-cohort study within the population-based MONICA/KORA Augsburg cohort. METHODS AND FINDINGS: Analyses for type 2 diabetes are based on 436 individuals with and 1410 individuals without incident diabetes. Analyses for coronary events are based on 314 individuals with and 1659 individuals without incident coronary events. Mean follow-up times were almost 11 years. Areas under the receiver-operating characteristic curve (AUC), changes in Akaike's information criterion (ΔAIC), integrated discrimination improvement (IDI) and net reclassification index (NRI) were calculated for different models. A basic model consisting of age, sex and survey predicted type 2 diabetes with an AUC of 0.690. Addition of 13 inflammation-related biomarkers (CRP, IL-6, IL-18, MIF, MCP-1/CCL2, IL-8/CXCL8, IP-10/CXCL10, adiponectin, leptin, RANTES/CCL5, TGF-β1, sE-selectin, sICAM-1; all measured in nonfasting serum) increased the AUC to 0.801, whereas addition of cardiometabolic risk factors (BMI, systolic blood pressure, ratio total/HDL-cholesterol, smoking, alcohol, physical activity, parental diabetes) increased the AUC to 0.803 (ΔAUC [95% CI] 0.111 [0.092-0.149] and 0.113 [0.093-0.149], respectively, compared to the basic model). The combination of all inflammation-related biomarkers and cardiometabolic risk factors yielded a further increase in AUC to 0.847 (ΔAUC [95% CI] 0.044 [0.028-0.066] compared to the cardiometabolic risk model). Corresponding AUCs for incident coronary events were 0.807, 0.825 (ΔAUC [95% CI] 0.018 [0.013-0.038] compared to the basic model), 0.845 (ΔAUC [95% CI] 0.038 [0.028-0.059] compared to the basic model) and 0.851 (ΔAUC [95% CI] 0.006 [0.003-0.021] compared to the cardiometabolic risk model), respectively. CONCLUSIONS: Inclusion of multiple inflammation-related biomarkers into a basic model and into a model including cardiometabolic risk factors significantly improved the prediction of type 2 diabetes and coronary events, although the improvement was less pronounced for the latter endpoint

LPS-induced NF??B enhanceosome requires TonEBP/NFAT5 without DNA binding

NF??B is a central mediator of inflammation. Present inhibitors of NF??B are mostly based on inhibition of essential machinery such as proteasome and protein kinases, or activation of nuclear receptors; as such, they are of limited therapeutic use due to severe toxicity. Here we report an LPS-induced NF??B enhanceosome in which TonEBP is required for the recruitment of p300. Increased expression of TonEBP enhances the NF??B activity and reduced TonEBP expression lowers it. Recombinant TonEBP molecules incapable of recruiting p300 do not stimulate NF??B. Myeloid-specific deletion of TonEBP results in milder inflammation and sepsis. We discover that a natural small molecule cerulenin specifically disrupts the enhanceosome without affecting the activation of NF??B itself. Cerulenin suppresses the pro-inflammatory activation of macrophages and sepsis without detectable toxicity. Thus, the NF??B enhanceosome offers a promising target for useful anti-inflammatory agents.ope