Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging

While Reinforcement Learning from Human Feedback (RLHF) aligns Large Language Models (LLMs) with general, aggregate human preferences, it is suboptimal for learning diverse, individual perspectives. In this work, we study Reinforcement Learning from Personalized Human Feedback (RLPHF) problem, wherein LLMs are aligned to multiple (sometimes conflicting) preferences by modeling alignment as a Multi-Objective Reinforcement Learning (MORL) problem. Compared to strong single-objective baselines, we show that we can achieve personalized alignment by decomposing preferences into multiple dimensions. These dimensions are defined based on personalizations that are declared as desirable by the user. In this work, we show that they can be efficiently trained independently in a distributed manner and combined effectively post-hoc through parameter merging. The code is available at https://github.com/joeljang/RLPHF.Comment: Preprin

Discrimination between target and non-target interactions on the viral surface by merging fluorescence emission into Rayleigh scattering

Direct and quantitative determination of antibodies or cellular receptors dynamically binding to the surface of viral particles is the key issue for predicting the efficacy of therapeutic materials or host susceptibility to a new emerging pathogen. However, targeted visualization of infectious viruses is still highly challenging owing to their nanoscopic sizes and uncontrollable nonspecific interactions with loading molecules responsible for false signals. Here we present a multimodal single-molecule and single-particle (SMSP) visualization capable of simultaneously yet independently tracking Rayleigh scattering and fluorescence that, respectively, are generated from viruses (approximately 100 nm) and labeled interacting molecules. By analyzing real-time trajectories of fluorescent antibodies against a virus surface protein with reference to single virus-derived Rayleigh scattering, we determined heterogeneous binding stoichiometry of virus-antibody couplings irrespective of the nonspecific binder population. Therefore, our multimodal (or multi-level) SMSP assay visually identifies and selectively quantifies specific interactions between them with single binding event accuracy. As a 'specific-binding quantifier' to assess variable host susceptibility to a virus, it was further applied for distinguishing ratiometric bindings and spontaneous dissociation kinetics of synthesized isomeric receptors to influenza virus. The present framework could offer a solid analytical foundation for the development of a direct-acting antiviral agent inhibiting an integral viral enveloped protein and for nanobiological investigation for dissecting spatiotemporal nanoparticle-molecule interactions, which have been scarcely explored compared to those among plasmonic nanoparticles or among molecules only

Insulin Secretion and Incretin Hormone Concentration in Women with Previous Gestational Diabetes Mellitus

BackgroundWe examined the change in the levels of incretin hormone and effects of glucose-dependent insulinotropic polypeptide (GIP) and glucagon-like peptide 1 (GLP-1) on insulin secretion in women with previous gestational diabetes (pGDM).MethodsA 75-g oral glucose tolerance test (OGTT) was conducted on 34 women with pGDM. In addition, 11 women with normal glucose tolerance, matched for age, height and weight, were also tested. The insulin, GIP, GLP-1, and glucagon concentrations were measured, and their anthropometric and biochemical markers were also measured.ResultsAmong 34 women with pGDM, 18 had normal glucose tolerance, 13 had impaired glucose tolerance (IGT) and 1 had diabetes. No significant differences were found in GLP-1 concentration between the pGDM and control group. However, a significantly high level of glucagon was present in the pGDM group at 30 minutes into the OGTT. The GIP concentration was elevated at 30 minutes and 60 minutes in the pGDM group. With the exception of the 30-minute timepoint, women with IGT had significantly high blood glucose from 0 to 120 minutes. However, there was no significant difference in insulin or GLP-1 concentration. The GIP level was significantly high from 0 to 90 minutes in patients diagnosed with IGT.ConclusionGLP-1 secretion does not differ between pGDM patients and normal women. GIP was elevated, but that does not seem to induce in increase in insulin secretion. Therefore, we conclude that other factors such as heredity and environment play important roles in the development of type 2 diabetes

Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning

Large language models excel at a variety of language tasks when prompted with examples or instructions. Yet controlling these models through prompting alone is limited. Tailoring language models through fine-tuning (e.g., via reinforcement learning) can be effective, but it is expensive and requires model access. We propose Inference-time Policy Adapters (IPA), which efficiently tailors a language model such as GPT-3 without fine-tuning it. IPA guides a large base model during decoding time through a lightweight policy adaptor trained to optimize an arbitrary user objective with reinforcement learning. On five challenging text generation tasks, such as toxicity reduction and open-domain generation, IPA consistently brings significant improvements over off-the-shelf language models. It outperforms competitive baseline methods, sometimes even including expensive fine-tuning. In particular, tailoring GPT-2 with IPA can outperform GPT-3, while tailoring GPT- 3 with IPA brings a major performance boost over GPT-3 (and sometimes even over GPT-4). Our promising results highlight the potential of IPA as a lightweight alternative to tailoring extreme-scale language models

Syndecan transmembrane domain specifically regulates downstream signaling events of the transmembrane receptor cytoplasmic domain

Despite the known importance of the transmembrane domain (TMD) of syndecan receptors in cell adhesion and signaling, the molecular basis for syndecan TMD function remains un-known. Using in vivo invertebrate models, we found that mammalian syndecan-2 rescued both the guidance defects in C. elegans hermaphrodite-specific neurons and the impaired development of the midline axons of Drosophila caused by the loss of endogenous syndecan. These compensatory ef-fects, however, were reduced significantly when syndecan-2 dimerization-defective TMD mutants were introduced. To further investigate the role of the TMD, we generated a chimera, 2eTPC, com-prising the TMD of syndecan-2 linked to the cytoplasmic domain of platelet-derived growth factor receptor (PDGFR). This chimera exhibited SDS-resistant dimer formation that was lost in the corre-sponding dimerization-defective syndecan-2 TMD mutant, 2eT(GL)PC. Moreover, 2eTPC specifically enhanced Tyr 579 and Tyr 857 phosphorylation in the PDGFR cytoplasmic domain, while the TMD mutant failed to support such phosphorylation. Finally, 2eTPC, but not 2eT(GL)PC, induced phosphorylation of Src and PI3 kinase (known downstream effectors of Tyr 579 phosphorylation) and promoted Src-mediated migration of NIH3T3 cells. Taken together, these data suggest that the TMD of a syndecan-2 specifically regulates receptor cytoplasmic domain function and subsequent downstream signaling events controlling cell behavior. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.1

Vertical distributions of organic matter components in sea ice near Cambridge Bay, Dease Strait, Canadian Archipelago

Ice algae thriving within sea ice play a crucial role in transferring energy to higher trophic levels and influencing biogeochemical processes in polar oceans; however, the distribution of organic matter within the ice interior is not well understood. This study aimed to investigate the vertical distribution of organic matter, including chlorophyll a (Chl-a), particulate organic carbon and nitrogen (POC and PON), carbohydrates (CHO), proteins (PRT), lipids (LIP), and food material (FM), within the sea ice. Samples were collected from the bottom, middle, and top sections of the sea ice column near Cambridge Bay during the spring of 2018. Based on the δ13C signature, biochemical composition, and POC contribution of biopolymeric carbon (BPC), the organic substances within the sea ice were predominantly attributed to marine autotrophs. While the highest concentrations of each parameter were observed at the sea ice bottom, notable concentrations were also found in the upper sections. The average sea ice column-integrated Chl-a concentration was 5.05 ± 2.26 mg m−2, with the bottom ice section contributing 59% (S.D. = ± 10%) to the total integration. The column-integrated concentrations of FM, BPC, POC, and PON were 2.05 ± 0.39, 1.10 ± 0.20, 1.47 ± 0.25, and 0.09 ± 0.03 g m−2, respectively. Contributions of the bottom ice section to these column-integrated concentrations varied for each parameter, with values of 20 ± 6, 21 ± 7, 19 ± 5, and 28 ± 7%, respectively. While the bottom ice section exhibited a substantial Chl-a contribution in line with previous studies, significantly higher contributions of the other parameters were observed in the upper sea ice sections. This suggests that the particulate matter within the interior of the sea ice could potentially serve as an additional food source for higher trophic grazers or act as a seeding material for a phytoplankton bloom during the ice melting season. Our findings highlight the importance of comprehensive field measurements encompassing the entire sea ice section to better understand the distribution of organic carbon pools within the sea ice in the Arctic Ocean

Corazon Genome Annotation Project

Corazon Genome Annotation Project Mycobacterium phage, Corazon, was found at Lafayette College in Easton, Pennsylvania in 2017. Its plaque was small, round and clear with siphoviridae morphology type. The approximate length of Corazon is 64931 bp with 3’ sticky overhang. The overhang had 11 base length with ‘GCGCGCAGCGC’ sequence. To perform a manual inspection of Corazon, we performed gene annotation by reviewing and revising the prediction and identifying any missing genes using DNA Annotation programs such as DNA Master, GeneMark, BLAST, Phamerator, and HHPred. We annotated the Corazon genome in three distinct steps. First, we established a relationship with our phage, Corazon, and other phages to understand the overall genomic architectures. Second, we ran automated gene prediction proteins and functional data on the predicted gene. Third, we reviewed the prediction and made necessary changes to delete or identify any missing genes. The group annotated the mycobacterium Corazon genes 33-48. Based on the evidence supported by the programs, most of the Corazon genes from 33 to 48 were forward genes with functions such as terminase, portal protein, capsid maturation protease, scaffolding protein, head decorate protein, major capsid protein, portal protein, major tail protein, and tail assembly chaperone. Tail assembly chaperones are only shown in frameshift genes, in this case, Gene 47. Gene 47 is a frameshift gene which a nucleotide reads more than once or omitted to have two genes in the same space. Since Gene 47 is a frameshift gene, Gene 46 and 47 are overlapping. Based on the predicted functions, the segment of genes 33 to 48 of Corazon mainly consists of structural genes that are largely responsible for the formation of the phage structure. By annotating the gene in this project, it will determine the known functions of the newly discovered phage, and it will contribute to the exploration of phage genomes. Keywords: Corazon, DNA Annotation, DNA Master, Genemark, gene, portal protein, major tail protein, tail assembly chaperone, capsid maturation protease

Development of Stem Cell-laden Spheroids Fabrication Using 3D Bioprinting Technology

Stem cell therapy is next generation therapeutic method to treat many intractable diseases such as ischemic heart disease. However, one of the key points to be solved in order to use this therapy is how to deliver the stem cells because of their poor transfer and survival rate. Encapsulating stem cell into bioink which contains decellularized extracellular matrix(dECM) can be the method to deliver stem cell into impaired tissues. Furthermore spheroid environment is niche condition so that cells were viable as shown as numerical superiority of green fluorescence in Live/Dead image.2

Biofabrication of Injectable Stem Cell-laden Spheroids for the Treatment of Ischemic Heart Diseases

Ischemic heart disease remains the leading cause of death in worldwide for several decades. It is classified as an incurable disease due to the limited regenerative capability of the ischemic tissues. Although stem cell therapy has emerged to overcome this disease, it has limitation about low rate of adhesion and differentiation of transplanted cells. In this study, three-dimensional spheroids including human c-kit+ mesenchymal stem cells and endothelial progenitor cells were fabricated by using 3D bioprinting technique. Bioink, mixture of heart decellularized extracellular matrix (hdECM), stem cells, and alginate, was prepared and was dropped and cross linked in a dish which is containing calcium chloride in a programmed manner. Optimal composition ratios were found by varying the composition of the materials to be mixed, and various sizes of spheroids were produced by changing the parameters such as pneumatic pressure, pressure time, and nozzle size. Also, predominant green fluorescence evidenced dominated population of live cells and clearly indicates that cells were viable and environment in the spheroid was biocompatible niche condition. It is expected that this can be used in the future to prevent or treat heart-related diseases.2