BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting

The BLOOM model is a large publicly available multilingual language model, but its pretraining was limited to 46 languages. To extend the benefits of BLOOM to other languages without incurring prohibitively large costs, it is desirable to adapt BLOOM to new languages not seen during pretraining. In this work, we apply existing language adaptation strategies to BLOOM and benchmark its zero-shot prompting performance on eight new languages in a resource-constrained setting. We find language adaptation to be effective at improving zero-shot performance in new languages. Surprisingly, we find that adapter-based finetuning is more effective than continued pretraining for large models. In addition, we discover that prompting performance is not significantly affected by language specifics, such as the writing system. It is primarily determined by the size of the language adaptation data. We also add new languages to BLOOMZ, which is a multitask finetuned version of BLOOM capable of following task instructions zero-shot. We find including a new language in the multitask fine-tuning mixture to be the most effective method to teach BLOOMZ a new language. We conclude that with sufficient training data language adaptation can generalize well to diverse languages. Our code is available at https://github.com/bigscience-workshop/multilingual-modeling

Hepatocyte Growth Factor Modulates Interleukin-6 Production in Bone Marrow Derived Macrophages: Implications for Inflammatory Mediated Diseases

The generation of the pro-inflammatory cytokines IL-6, TNF-α, and IL-1β fuel the acute phase response (APR). To maintain body homeostasis, the increase of inflammatory proteins is resolved by acute phase proteins via presently unknown mechanisms. Hepatocyte growth factor (HGF) is transcribed in response to IL-6. Since IL-6 production promotes the generation of HGF and induces the APR, we posited that accumulating HGF might be a likely candidate for quelling excess inflammation under non-pathological conditions. We sought to assess the role of HGF and how it influences the regulation of inflammation utilizing a well-defined model of inflammatory activation, lipopolysaccharide (LPS)-stimulation of bone marrow derived macrophages (BMM). BMM were isolated from C57BL6 mice and were stimulated with LPS in the presence or absence of HGF. When HGF was present, there was a decrease in production of the pro-inflammatory cytokine IL-6, along with an increase in the anti-inflammatory cytokine IL-10. Altered cytokine production correlated with an increase in phosphorylated GSK3β, increased retention of the phosphorylated NFκB p65 subunit in the cytoplasm, and an enhanced interaction between CBP and phospho-CREB. These changes were a direct result of signaling through the HGF receptor, MET, as effects were reversed in the presence of a selective inhibitor of MET (SU11274) or when using BMM from macrophage-specific conditional MET knockout mice. Combined, these data provide compelling evidence that under normal circumstances, HGF acts to suppress the inflammatory response