PerPLM: Personalized Fine-tuning of Pretrained Language Models via Writer-specific Intermediate Learning and Prompts

The meanings of words and phrases depend not only on where they are used (contexts) but also on who use them (writers). Pretrained language models (PLMs) are powerful tools for capturing context, but they are typically pretrained and fine-tuned for universal use across different writers. This study aims to improve the accuracy of text understanding tasks by personalizing the fine-tuning of PLMs for specific writers. We focus on a general setting where only the plain text from target writers are available for personalization. To avoid the cost of fine-tuning and storing multiple copies of PLMs for different users, we exhaustively explore using writer-specific prompts to personalize a unified PLM. Since the design and evaluation of these prompts is an underdeveloped area, we introduce and compare different types of prompts that are possible in our setting. To maximize the potential of prompt-based personalized fine-tuning, we propose a personalized intermediate learning based on masked language modeling to extract task-independent traits of writers' text. Our experiments, using multiple tasks, datasets, and PLMs, reveal the nature of different prompts and the effectiveness of our intermediate learning approach.Comment: 11 page

Dose uncertainty due to energy dependence in dual-energy computed tomography

Purpose: To evaluate the absolute dose uncertainty at 2 different energies and for the large and small bowtie filters in dual-energy computed tomography (DECT). Material and methods: Measurements were performed using DECT at 80 kV and 140 kilovoltage peak (kVp), and single-energy computed tomography (CT) at 120 kV. The absolute dose was calculated from the mass-energy absorption obtained from the half-value layer (HVL) of aluminium. Results: The difference in the water-to-air ratio of the mean mass energy-absorption coefficients at 80 kV and 140 kV was 2.0% for the small bow-tie filter and 3.0% for the large bow-tie filter. At lower tube voltages, the difference in the absorbed dose with the large and small bow-tie filters was larger. Conclusions: The absolute dose uncertainty due to energy dependence was 3.0%, which could be reduced with single-energy beams at 120 kV or by using the average effective energy measurement with dual-energy beams

Transforming growth factor α protects against Fas-mediated liver apoptosis in mice

AbstractThe Fas/Fas ligand interaction plays a crucial role in various liver diseases, and administration of agonistic anti-Fas antibody to mice causes massive hepatic apoptosis and fulminant hepatic failure. Several growth factors have recently been found to function in preventing apoptosis. In this study, we demonstrated that overexpression of transforming growth factor α (TGFα) has a dramatic protective effect on Fas-mediated hepatic apoptosis at the biochemical and histological levels. Moreover, 85.7% (six out of seven) of TGFα transgenic mice survived the lethal liver damage, whereas all wild-type mice died. Expression of Bcl-xL, an anti-apoptotic protein, was greatly increased in the transgenic mice. Taken together, our findings suggest that TGFα protects against Fas-mediated liver apoptosis in vivo and up-regulation of Bcl-xL may participate in protective effect of TGFα

Impacts of direct release and river discharge on oceanic 137Cs derived from the Fukushima Dai-ichi Nuclear Power Plant accident

A series of accidents at the Fukushima Dai-ichi Nuclear Power Plant (1F NPP) following the Great East Japan Earthquake and tsunami of 11 March 2011 resulted in the release of radioactive materials to the ocean. We used the Regional Ocean Model System (ROMS) to simulate the 137Cs activity in the oceanic area off Fukushima, with the sources of radioactivity being direct release, atmospheric deposition, river discharge, and inflow across the domain boundary. The direct release rate of 137Cs after the accident until the end of 2016 was estimated by comparing simulated results with measured 137Cs activities adjacent to the 1F NPP. River discharge rates of 137Cs were estimated by multiplying simulated river flow rates by the dissolved 137Cs activities, which were estimated by an empirical function. Inflow of 137Cs across the domain boundary was set according to the results of a North Pacific Ocean model. Because the spatiotemporal variability of 137Cs activity was large, the simulated results were compared with the annual averaged observed 137Cs activity distribution. Normalized annual averaged 137Cs activity distributions in the regional ocean were similar for each year from 2013 to 2016. This result suggests that the annual averaged distribution is predictable. Simulated 137Cs activity attributable to direct release was in good agreement with measurement data from the coastal zone adjacent to the 1F NPP. Comparison of the simulated results with measured activity in the offshore area indicated that the simulation slightly underestimated the activity attributable to inflow across the domain boundary. This result suggests that recirculation of subducted 137Cs to the surface layer was underestimated by the North Pacific model. During the study period, the effect of river discharge on oceanic 137Cs activity was small compared to the effect of directly released 137Cs

Gremlin Enhances the Determined Path to Cardiomyogenesis

BACKGROUND: The critical event in heart formation is commitment of mesodermal cells to a cardiomyogenic fate, and cardiac fate determination is regulated by a series of cytokines. Bone morphogenetic proteins (BMPs) and fibroblast growth factors have been shown to be involved in this process, however additional factors needs to be identified for the fate determination, especially at the early stage of cardiomyogenic development. METHODOLOGY/PRINCIPAL FINDINGS: Global gene expression analysis using a series of human cells with a cardiomyogenic potential suggested Gremlin (Grem1) is a candidate gene responsible for in vitro cardiomyogenic differentiation. Grem1, a known BMP antagonist, enhanced DMSO-induced cardiomyogenesis of P19CL6 embryonal carcinoma cells (CL6 cells) 10-35 fold in an area of beating differentiated cardiomyocytes. The Grem1 action was most effective at the early differentiation stage when CL6 cells were destined to cardiomyogenesis, and was mediated through inhibition of BMP2. Furthermore, BMP2 inhibited Wnt/beta-catenin signaling that promoted CL6 cardiomyogenesis. CONCLUSIONS/SIGNIFICANCE: Grem1 enhances the determined path to cardiomyogenesis in a stage-specific manner, and inhibition of the BMP signaling pathway is involved in initial determination of Grem1-promoted cardiomyogenesis. Our results shed new light on renewal of the cardiovascular system using Grem1 in human

Midkine promoter-based conditionally replicative adenovirus therapy for midkine-expressing human pancreatic cancer

<p>Abstract</p> <p>Background</p> <p>To develop a novel therapeutic strategy for human pancreatic cancer using a midkine promoter-based conditionally replicating adenovirus.</p> <p>Methods</p> <p>We examined midkine mRNA expression and midkine protein expression by seven human pancreatic cancer cell lines (AsPC-1, BxPC-3, CFPAC-1, HPAC, MIAPaCa-2, PANC-1, and Suit-2), as well as by non-cancerous pancreatic tissue and pancreatic cancers. Midkine promoter activity was measured in cancer cell lines by the dual luciferase reporter assay. Adenoviral transduction efficiency was assessed by fluorescent staining of cancer cell lines using adenovirus type 5 containing the green fluorescent protein gene (Ad5GFP). Replication of adenovirus type 5 containing the 0.6 kb midkne promoter (Ad5MK) was assessed by the detection of E1 protein in cancer cell lines. The cytotoxicity of Ad5MK for cancer cells was evaluated from the extent of growth inhibition after viral infection. Infection and replication were also assessed in nude mice with subcutaneous Suit-2 tumors by intratumoral injection of Ad5MK, Ad5GFP, or vehicle. E1a mRNA expression in the treated tumors and expression of the replication-specific adenoviral hexon protein were evaluated. Finally, the anti-tumor activity of Ad5MK against intraperitoneal xenografts of Suit-2 pancreatic cancer cells was examined after intraperitoneal injection of the virus.</p> <p>Results</p> <p>Both midkine mRNA expression and midkine protein expression were strong in AsPC-1 and CFPAC-1 cell liens, moderate in BxPC-3, HPAC, and Suit-2 cell lines, and weak in PANC-1 and MIAPaCa-2 cell lines. Expression of midkine mRNA was significantly stronger in pancreatic cancers than in non-cancerous pancreatic tissues. The relative luciferase activity mediated by the 0.6 kb midkne fragment in AsPC-1, PANC-1, and Suit-2 cell lines was approximately 6 to 20 times greater than that in midkne-negative MIAPaCa-2 cell lines. Pancreatic cancer cell lines exhibited a heterogeneous adenoviral transduction profile. E1A expression was higher in cell lines with strong midkine expression than in cell lines with weak midkine expression. Ad5MK showed much greater cytotoxicity for midkine-expressing Suit-2 and PANC-1 cell lines than for midkine-negative MIAPaCa-2 cell lines. In the Suit-2 subcutaneous xenograft model, expression of E1A was detected in Ad5MK-treated tumors, but not in untreated and Ad5GFP-treated tumors. In the Suit-2 intraperitoneal xenograft model, the Ad5MK group survived for significantly longer than the Ad5GFP, PBS, and untreated groups.</p> <p>Conclusion</p> <p>Ad5MK has an anti-tumor effect against human pancreatic cancer cell lines that express midkine mRNA. Midkine promoter-based conditionally replicative adenovirus might be a promising new gene therapy for pancreatic cancer.</p

Gradient-based parameter optimization method to determine membrane ionic current composition in human induced pluripotent stem cell-derived cardiomyocytes

Premature cardiac myocytes derived from human induced pluripotent stem cells (hiPSC-CMs) show heterogeneous action potentials (APs), probably due to different expression patterns of membrane ionic currents. We developed a method for determining expression patterns of functional channels in terms of whole-cell ionic conductance (Gx) using individual spontaneous AP configurations. It has been suggested that apparently identical AP configurations can be obtained using different sets of ionic currents in mathematical models of cardiac membrane excitation. If so, the inverse problem of Gx estimation might not be solved. We computationally tested the feasibility of the gradient-based optimization method. For a realistic examination, conventional 'cell-specific models' were prepared by superimposing the model output of AP on each experimental AP recorded by conventional manual adjustment of Gxs of the baseline model. Gxs of 4–6 major ionic currents of the 'cell-specific models' were randomized within a range of ± 5–15% and used as an initial parameter set for the gradient-based automatic Gxs recovery by decreasing the mean square error (MSE) between the target and model output. Plotting all data points of the MSE–Gx relationship during optimization revealed progressive convergence of the randomized population of Gxs to the original value of the cell-specific model with decreasing MSE. The absence of any other local minimum in the global search space was confirmed by mapping the MSE by randomizing Gxs over a range of 0.1–10 times the control. No additional local minimum MSE was obvious in the whole parameter space, in addition to the global minimum of MSE at the default model parameter

A novel efficient feeder-free culture system for the derivation of human induced pluripotent stem cells

Nakagawa, M.et al. A novel efficient feeder-free culture system forthe derivation of human induced pluripotent stem cells.Sci. Rep.4, 3594; DOI:10.1038/srep03594 (2014)