Locating and Mitigating Gender Bias in Large Language Models

Large language models(LLM) are pre-trained on extensive corpora to learn facts and human cognition which contain human preferences. However, this process can inadvertently lead to these models acquiring biases and stereotypes prevalent in society. Prior research has typically tackled the issue of bias through a one-dimensional perspective, concentrating either on locating or mitigating it. This limited perspective has created obstacles in facilitating research on bias to synergistically complement and progressively build upon one another. In this study, we integrate the processes of locating and mitigating bias within a unified framework. Initially, we use causal mediation analysis to trace the causal effects of different components' activation within a large language model. Building on this, we propose the LSDM (Least Square Debias Method), a knowledge-editing based method for mitigating gender bias in occupational pronouns, and compare it against two baselines on three gender bias datasets and seven knowledge competency test datasets. The experimental results indicate that the primary contributors to gender bias are the bottom MLP modules acting on the last token of occupational pronouns and the top attention module acting on the final word in the sentence. Furthermore, LSDM mitigates gender bias in the model more effectively than the other baselines, while fully preserving the model's capabilities in all other aspects.Comment: 23 pages, 5 figure

Changes in the small noncoding RNA transcriptome in osteosarcoma cells

Abstract Background Osteosarcoma has the highest incidence among bone malignant tumors and mainly occurs in adolescents and the elderly, but the pathological mechanism is still unclear, which makes early diagnosis and treatment very difficult. Bone marrow mesenchymal stem cells (BMSCs) are considered to be one of the sources of osteosarcoma cells. Therefore, a full understanding of the gene expression differences between BMSCs and osteosarcoma cells is very important to explore the pathogenesis of osteosarcoma and facilitate the early diagnosis and treatment of osteosarcoma. Small noncoding RNAs (sncRNAs) are a class of RNAs that do not encode proteins but directly play biological functions at the RNA level. SncRNAs mainly include Piwi-interacting RNAs (piRNAs), small nucleolar RNAs (snoRNAs), small nuclear RNAs (snRNAs), repeat RNAs and microRNAs (miRNAs). Methods In this study, we compared the expression of sncRNAs in BMSCs and osteosarcoma cells by high-throughput sequencing and qPCR and looked for differentially expressed sncRNAs. CCK-8, clone formation and transwell assay were used to detect the effect of sncRNA in MG63 cells. Results We found that 66 piRNAs were significantly upregulated and 70 piRNAs were significantly downregulated in MG63 cells. As for snoRNAs, 71 snoRNAs were significantly upregulated and 117 snoRNAs were significantly downregulated in MG63 cells. As for snRNAs, 35 snRNAs were significantly upregulated and 17 snRNAs were significantly downregulated in MG63 cells. As for repeat RNAs, 6 repeat RNAs were significantly upregulated and 7 repeat RNAs were significantly downregulated in MG63 cells. As for miRNAs, 326 miRNAs were significantly upregulated and 281 miRNAs were significantly downregulated in MG63 cells. Overexpression of piRNA DQ596225, snoRNA ENST00000364830.2, snRNA ENST00000410533.1 and miRNA hsa-miR-369-5p inhibited the proliferation and migration of MG63 cells. Conclusions Our results provide a theoretical basis for the pathogenesis, early diagnosis and treatment of osteosarcoma

Hierarchically porous surface of PEEK/nMCS composite created by femtosecond laser and incorporation of resveratrol exhibiting antibacterial performances and osteogenic activity in vitro

To further improve the surface properties of polyetheretherketone (PEEK)/nanoporous magnesium calcium silicate (nMCS) composite (PMC), hierarchically porous surface of PMC (PMCF) were created by femtosecond laser, and resveratrol (RV) was incorporated into the porous surface of PMCF (RV@PMCF). Compared to PMC with flat surface, PMCF contained not only two types of micropores with different sizes (around 20 μm and 0.5 μm) but also nanopores (around 4 nm), which exhibited remarkably increase in surface roughness and protein adsorption. In addition, PMCF displayed a slow-release of RV while PMC showed a burst-release of RV into cell cultured medium. Moreover, compared with PMC and PMCF, RV@PMCF with antibacterial performances inhibited the growth of E. coli and S. aureus thanks to the release of RV. In addition, compared with PMC, PMCF and RV@PMCF significantly promoted adhesion and proliferation of rat bone mesenchymal stem cells (BMSC). Furthermore, compared with PMCF, RV@PMCF obviously enhanced the proliferation and osteogenic differentiation as well as bone related genes expressions of BMSC. The results demonstrated that PMCF with hierarchically porous surface and incorporating of RV displayed antibacterial performances and osteogenic activity in vitro, Therefore, as a drug-loaded implant, RV@PMCF with good cytocompatibility would have a big potential for applications in orthopaedic fields. © 2020 Elsevier LtdNational Natural Science Foundation of Youth in China [81601866, 81771990]; Shanghai municipal health commission clinical special project [20184Y0124]; Key Medical Program of Science and Technology Development of Shanghai [19441906100, 17441900600, 17441902000]; Ministry of Education, Youth and Sports of the Czech Republic-Program NPUI [L01504

Additional file 1 of Changes in the small noncoding RNA transcriptome in osteosarcoma cells

Additional file 1. Table S1

Loss of FMRP Impaired Hippocampal Long-Term Plasticity and Spatial Learning in Rats

Fragile X syndrome (FXS) is a neurodevelopmental disorder caused by mutations in the FMR1 gene that inactivate expression of the gene product, the fragile X mental retardation 1 protein (FMRP). In this study, we used clustered regularly interspaced short palindromic repeats (CRISPR)/CRISPR-associated protein 9 (Cas9) technology to generate Fmr1 knockout (KO) rats by disruption of the fourth exon of the Fmr1 gene. Western blotting analysis confirmed that the FMRP was absent from the brains of the Fmr1 KO rats (Fmr1(exon4-KO)). Electrophysiological analysis revealed that the theta-burst stimulation (TBS)-induced long-term potentiation (LTP) and the low-frequency stimulus (LFS)-induced long-term depression (LTD) were decreased in the hippocampal Schaffer collateral pathway of the Fmr1(exon4-KO) rats. Short-term plasticity, measured as the paired-pulse ratio, remained normal in the KO rats. The synaptic strength mediated by the a -amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPAR) was also impaired. Consistent with previous reports, the Fmr1(exon4-KO) rats demonstrated an enhanced 3,5-dihydroxyphenylglycine (DHPG)-induced LTD in the present study, and this enhancement is insensitive to protein translation. In addition, the Fmr1(exon4-KO) rats showed deficits in the probe trial in the Morris water maze test. These results demonstrate that deletion of the Fmr1 gene in rats specifically impairs long-term synaptic plasticity and hippocampus-dependent learning in a manner resembling the key symptoms of FXS. Furthermore, the Fmr1(exon4-KO) rats displayed impaired social interaction and macroorchidism, the results consistent with those observed in patients with FXS. Thus, Fmr1exon4 KO rats constitute a novel rat model of FXS that complements existing mouse models.</p