On the Effectiveness of Out-of-Distribution Data in Self-Supervised Long-Tail Learning

Though Self-supervised learning (SSL) has been widely studied as a promising technique for representation learning, it doesn't generalize well on long-tailed datasets due to the majority classes dominating the feature space. Recent work shows that the long-tailed learning performance could be boosted by sampling extra in-domain (ID) data for self-supervised training, however, large-scale ID data which can rebalance the minority classes are expensive to collect. In this paper, we propose an alternative but easy-to-use and effective solution, Contrastive with Out-of-distribution (OOD) data for Long-Tail learning (COLT), which can effectively exploit OOD data to dynamically re-balance the feature space. We empirically identify the counter-intuitive usefulness of OOD samples in SSL long-tailed learning and principally design a novel SSL method. Concretely, we first localize the `head' and `tail' samples by assigning a tailness score to each OOD sample based on its neighborhoods in the feature space. Then, we propose an online OOD sampling strategy to dynamically re-balance the feature space. Finally, we enforce the model to be capable of distinguishing ID and OOD samples by a distribution-level supervised contrastive loss. Extensive experiments are conducted on various datasets and several state-of-the-art SSL frameworks to verify the effectiveness of the proposed method. The results show that our method significantly improves the performance of SSL on long-tailed datasets by a large margin, and even outperforms previous work which uses external ID data. Our code is available at https://github.com/JianhongBai/COLT

Robustness-Guided Image Synthesis for Data-Free Quantization

Quantization has emerged as a promising direction for model compression. Recently, data-free quantization has been widely studied as a promising method to avoid privacy concerns, which synthesizes images as an alternative to real training data. Existing methods use classification loss to ensure the reliability of the synthesized images. Unfortunately, even if these images are well-classified by the pre-trained model, they still suffer from low semantics and homogenization issues. Intuitively, these low-semantic images are sensitive to perturbations, and the pre-trained model tends to have inconsistent output when the generator synthesizes an image with poor semantics. To this end, we propose Robustness-Guided Image Synthesis (RIS), a simple but effective method to enrich the semantics of synthetic images and improve image diversity, further boosting the performance of downstream data-free compression tasks. Concretely, we first introduce perturbations on input and model weight, then define the inconsistency metrics at feature and prediction levels before and after perturbations. On the basis of inconsistency on two levels, we design a robustness optimization objective to enhance the semantics of synthetic images. Moreover, we also make our approach diversity-aware by forcing the generator to synthesize images with small correlations in the label space. With RIS, we achieve state-of-the-art performance for various settings on data-free quantization and can be extended to other data-free compression tasks.Comment: Accepted at AAAI 202

FLT3L and Plerixafor Combination Increases Hematopoietic Stem Cell Mobilization and Leads to Improved Transplantation Outcome

AbstractHematopoietic stem cell (HSC) transplantation has curative potential for patients with hematological malignancies. Clinically, HSCs derived from mobilized peripheral blood are used more frequently than bone marrow. However, current standard mobilizing agents yield grafts that may not contain sufficient HSCs. Here, using murine models, we discovered that FLT3L synergized with plerixafor to mobilize phenotypically defined HSCs and their combination (FP) was superior to granulocyte colony-stimulating factor (G-CSF) alone or in combination with plerixafor (GP). Additionally, FP mobilized more regulatory T cells, natural killer cells, and plasmacytoid dendritic cells compared with G-CSF alone or GP. Both syngeneic and allogeneic grafts mobilized by FP led to long-term survival in transplanted mice. Collectively, FP represents a promising novel and potent mobilization regimen with potential clinical application in both the autologous and allogeneic transplantation settings

Integrated analysis of single-cell RNA-seq and bulk RNA-seq reveals RNA N6-methyladenosine modification associated with prognosis and drug resistance in acute myeloid leukemia

IntroductionAcute myeloid leukemia (AML) is a type of blood cancer that is identified by the unrestricted growth of immature myeloid cells within the bone marrow. Despite therapeutic advances, AML prognosis remains highly variable, and there is a lack of biomarkers for customizing treatment. RNA N6-methyladenosine (m6A) modification is a reversible and dynamic process that plays a critical role in cancer progression and drug resistance.MethodsTo investigate the m6A modification patterns in AML and their potential clinical significance, we used the AUCell method to describe the m6A modification activity of cells in AML patients based on 23 m6A modification enzymes and further integrated with bulk RNA-seq data.ResultsWe found that m6A modification was more effective in leukemic cells than in immune cells and induced significant changes in gene expression in leukemic cells rather than immune cells. Furthermore, network analysis revealed a correlation between transcription factor activation and the m6A modification status in leukemia cells, while active m6A-modified immune cells exhibited a higher interaction density in their gene regulatory networks. Hierarchical clustering based on m6A-related genes identified three distinct AML subtypes. The immune dysregulation subtype, characterized by RUNX1 mutation and KMT2A copy number variation, was associated with a worse prognosis and exhibited a specific gene expression pattern with high expression level of IGF2BP3 and FMR1, and low expression level of ELAVL1 and YTHDF2. Notably, patients with the immune dysregulation subtype were sensitive to immunotherapy and chemotherapy.DiscussionCollectively, our findings suggest that m6A modification could be a potential therapeutic target for AML, and the identified subtypes could guide personalized therapy

Profiling analysis of long non-coding RNAs in early postnatal mouse hearts

Mammalian cardiomyocytes undergo a critical hyperplastic-to-hypertrophic growth transition at early postnatal age, which is important in establishing normal physiological function of postnatal hearts. In the current study, we intended to explore the role of long non-coding (lnc) RNAs in this transitional stage. We analyzed lncRNA expression profiles in mouse hearts at postnatal day (P) 1, P7 and P28 via microarray. We identified 1,146 differentially expressed lncRNAs with more than 2.0-fold change when compared the expression profiles of P1 to P7, P1 to P28, and P7 to P28. The neighboring genes of these differentially expressed lncRNAs were mainly involved in DNA replication-associated biological processes. We were particularly interested in one novel cardiac-enriched lncRNA, ENSMUST00000117266, whose expression was dramatically down-regulated from P1 to P28 and was also sensitive to hypoxia, paraquat, and myocardial infarction. Knockdown ENSMUST00000117266 led to a significant increase of neonatal mouse cardiomyocytes in G0/G1 phase and reduction in G2/M phase, suggesting that ENSMUST00000117266 is involved in regulating cardiomyocyte proliferative activity and is likely associated with hyperplastic-to-hypertrophic growth transition. In conclusion, our data have identified a large group of lncRNAs presented in the early postnatal mouse heart. Some of these lncRNAs may have important functions in cardiac hyperplastic-to-hypertrophic growth transition

Anthropogenic Activities Generate High-Refractory Black Carbon along the Yangtze River Continuum

12 pages, 7 figuresCombustion-driven particulate black carbon (PBC) is a crucial slow-cycling pool in the organic carbon flux from rivers to oceans. Since the refractoriness of PBC stems from the association of non-homologous char and soot, the composition and source of char and soot must be considered when investigating riverine PBC. Samples along the Yangtze River continuum during different hydrological periods were collected in this study to investigate the association and asynchronous combustion drive of char and soot in PBC. The results revealed that PBC in the Yangtze River, with higher refractory nature, accounts for 13.73 ± 6.89% of particulate organic carbon, and soot occupies 37.53 ± 11.00% of PBC. The preponderant contribution of fossil fuel combustion to soot (92.57 ± 3.20%) compared to char (27.55 ± 5.92%), suggested that fossil fuel combustion is a crucial driver for PBC with high soot percentage. Redundancy analysis and structural equation modeling confirmed that the fossil fuel energy used by anthropogenic activities promoting soot is the crucial reason for high-refractory PBC. We estimated that the Yangtze River transported 0.15–0.23 Tg of soot and 0.15–0.25 Tg of char to the ocean annually, and the export of large higher refractory PBC to the ocean can form a long-term sink and prolong the residence time of terrigenous carbonThis study was supported by grants from the National Natural Science Foundation of China (nos. 42277214, 42207256, and 41971286), major programs of the National Social Science Foundation of China (grant nos. 22&ZD136), the Special Science and Technology Innovation Program for Carbon Peak and Carbon Neutralization of Jiangsu Province (grant no. BE2022612)Peer reviewe

Genomic, Pathway Network, and Immunologic Features Distinguishing Squamous Carcinomas

This integrated, multiplatform PanCancer Atlas study co-mapped and identified distinguishing molecular features of squamous cell carcinomas (SCCs) from five sites associated with smokin

Qwen Technical Report

Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans. In this work, we introduce Qwen, the first installment of our large language model series. Qwen is a comprehensive language model series that encompasses distinct models with varying parameter counts. It includes Qwen, the base pretrained language models, and Qwen-Chat, the chat models finetuned with human alignment techniques. The base language models consistently demonstrate superior performance across a multitude of downstream tasks, and the chat models, particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are highly competitive. The chat models possess advanced tool-use and planning capabilities for creating agent applications, showcasing impressive performance even when compared to bigger models on complex tasks like utilizing a code interpreter. Furthermore, we have developed coding-specialized models, Code-Qwen and Code-Qwen-Chat, as well as mathematics-focused models, Math-Qwen-Chat, which are built upon base language models. These models demonstrate significantly improved performance in comparison with open-source models, and slightly fall behind the proprietary models.Comment: 59 pages, 5 figure

Pan-Cancer Analysis of lncRNA Regulation Supports Their Targeting of Cancer Genes in Each Tumor Context

Long noncoding RNAs (lncRNAs) are commonly dys-regulated in tumors, but only a handful are known toplay pathophysiological roles in cancer. We inferredlncRNAs that dysregulate cancer pathways, onco-genes, and tumor suppressors (cancer genes) bymodeling their effects on the activity of transcriptionfactors, RNA-binding proteins, and microRNAs in5,185 TCGA tumors and 1,019 ENCODE assays.Our predictions included hundreds of candidateonco- and tumor-suppressor lncRNAs (cancerlncRNAs) whose somatic alterations account for thedysregulation of dozens of cancer genes and path-ways in each of 14 tumor contexts. To demonstrateproof of concept, we showed that perturbations tar-geting OIP5-AS1 (an inferred tumor suppressor) andTUG1 and WT1-AS (inferred onco-lncRNAs) dysre-gulated cancer genes and altered proliferation ofbreast and gynecologic cancer cells. Our analysis in-dicates that, although most lncRNAs are dysregu-lated in a tumor-specific manner, some, includingOIP5-AS1, TUG1, NEAT1, MEG3, and TSIX, synergis-tically dysregulate cancer pathways in multiple tumorcontexts