PerPLM: Personalized Fine-tuning of Pretrained Language Models via Writer-specific Intermediate Learning and Prompts

The meanings of words and phrases depend not only on where they are used (contexts) but also on who use them (writers). Pretrained language models (PLMs) are powerful tools for capturing context, but they are typically pretrained and fine-tuned for universal use across different writers. This study aims to improve the accuracy of text understanding tasks by personalizing the fine-tuning of PLMs for specific writers. We focus on a general setting where only the plain text from target writers are available for personalization. To avoid the cost of fine-tuning and storing multiple copies of PLMs for different users, we exhaustively explore using writer-specific prompts to personalize a unified PLM. Since the design and evaluation of these prompts is an underdeveloped area, we introduce and compare different types of prompts that are possible in our setting. To maximize the potential of prompt-based personalized fine-tuning, we propose a personalized intermediate learning based on masked language modeling to extract task-independent traits of writers' text. Our experiments, using multiple tasks, datasets, and PLMs, reveal the nature of different prompts and the effectiveness of our intermediate learning approach.Comment: 11 page

Whole-exome sequencing of fibroblast and its iPS cell lines derived from a patient diagnosed with xeroderma pigmentosum

AbstractCells from a patient with a DNA repair-deficiency disorder are anticipated to bear a large number of somatic mutations. Because such mutations occur independently in each cell, there is a high degree of mosaicism in patients' tissues. While major mutations that have been expanded in many cognate cells are readily detected by sequencing, minor ones are overlaid with a large depth of non-mutated alleles and are not detected. However, cell cloning enables us to observe such cryptic mutations as well as major mutations. In the present study, we focused on a fibroblastic cell line that is derived from a patient diagnosed with xeroderma pigmentosum (XP), which is an autosomal recessive disorder caused by a deficiency in nucleotide excision repair. By making a list of somatic mutations, we can expect to see a characteristic pattern of mutations caused by the hereditary disorder. We cloned a cell by generating an iPS cell line and performed a whole-exome sequencing analysis of the progenitor and its iPS cell lines. Unexpectedly, we failed to find causal mutations in the XP-related genes, but we identified many other mutations including homozygous deletion of GSTM1 and GSTT1. In addition, we found that the long arm of chromosome 9 formed uniparental disomy in the iPS cell line, which was also confirmed by a structural mutation analysis using a SNP array. Type and number of somatic mutations were different from those observed in XP patients. Taken together, we conclude that the patient might be affected by a different type of the disorder and that some of the mutations that we identified here may be responsible for exhibiting the phenotype. Sequencing and SNP-array data have been submitted to SRA and GEO under accession numbers SRP059858 and GSE55520, respectively

病原菌シグナルによるエンドウ原形質膜におけるホスファチジルイノシトールリン脂質のリン酸化とリゾリン脂質生成の制御

Effects of elicitor and suppressor from a pea pathogen, Mycosphaerella pinodes, on Pl etabolism in pea plasma membrane were examined in vitro. The elicitor induced rapid phosphorylation of phosphatidylinositols as well as production of lysophospholipid in plasma membranes, but these responses were severely inhibited by the suppressor. These results indicate that a membrane-associated phospholipase A is regulated coordinately by fungal signals, together with Pl metabolism, and that it may participate in signal transduction pathways leading to defense responses. To evaluate a possible rote of phospholipase A activation in induction of a pea defense response, the effect of free fatty acid on induction of a phytoalexin accumulation was also examined. When pea leaves were treated with linoleic- or linolenic acid, most commonly released in plant cells by phospholipase A, the accumulation of pisatin was induced even in the absence of the elicitor. It is, therefore, conceivable that free fatty acid(s) released from plasma membrane is also implicated in the early stage of elicitor-signal transduction in pea.エンドウの上胚軸組織により分離した原形質膜画分を褐紋病菌の生産するエリシターで処理すると、ホスファチジルイノシトールリン脂質の急速なリン酸化とリゾリン脂質の生成が誘導されたが、同菌より調製したサプレッサーの共存下では双方とも著しく阻害された。本結果は、ポリホスホイノシチド代謝系と同調的に作動するホリパーゼA活性化が存在すること、さらに、原形質膜における病原菌シグナルの受容・応答には複数の資質代謝系が介在する可能性を示唆している。一方、ホスホリパーゼAの活性化の役割を調べる目的で、本酵素によって原形質膜から生成されると考えられる脂肪酸(リノール酸ならびにリノレン酸)をエンドウ葉に処理したところ、エリシターの非存在下においてもファイトアレキシンであるピサチンの生成が誘導されることが示された。以上から、ポリホスホイノシチド代謝系と同調的に働くホスホリパーゼAがエリシターシグナルの初期伝達に深く関連しているものと考えられた