NADiffuSE: Noise-aware Diffusion-based Model for Speech Enhancement

The goal of speech enhancement (SE) is to eliminate the background interference from the noisy speech signal. Generative models such as diffusion models (DM) have been applied to the task of SE because of better generalization in unseen noisy scenes. Technical routes for the DM-based SE methods can be summarized into three types: task-adapted diffusion process formulation, generator-plus-conditioner (GPC) structures and the multi-stage frameworks. We focus on the first two approaches, which are constructed under the GPC architecture and use the task-adapted diffusion process to better deal with the real noise. However, the performance of these SE models is limited by the following issues: (a) Non-Gaussian noise estimation in the task-adapted diffusion process. (b) Conditional domain bias caused by the weak conditioner design in the GPC structure. (c) Large amount of residual noise caused by unreasonable interpolation operations during inference. To solve the above problems, we propose a noise-aware diffusion-based SE model (NADiffuSE) to boost the SE performance, where the noise representation is extracted from the noisy speech signal and introduced as a global conditional information for estimating the non-Gaussian components. Furthermore, the anchor-based inference algorithm is employed to achieve a compromise between the speech distortion and noise residual. In order to mitigate the performance degradation caused by the conditional domain bias in the GPC framework, we investigate three model variants, all of which can be viewed as multi-stage SE based on the preprocessing networks for Mel spectrograms. Experimental results show that NADiffuSE outperforms other DM-based SE models under the GPC infrastructure. Audio samples are available at: https://square-of-w.github.io/NADiffuSE-demo/

Research hotspots and trends of fresh e-commerce in China: A knowledge mapping analysis based on bibliometrics

The fresh e-commerce industry has seen a sudden and substantial rise since the outbreak of COVID-19. The rapid development of this industry calls for a comprehensive and systematic review of its research status, hotspots and future trends, which will have significant implications for researchers in related fields. This paper first conducts a current situation analysis of the core literature on fresh e-commerce retrieved from four databases – CNKI, CSSCI, Wanfang and VIP – to categorize the research status of fresh e-commerce in three dimensions: the year of publication, article sources, and distribution of subjects. CiteSpace is then used to perform a bibliometric analysis of the data and to create visualized knowledge maps. The results show that the research on fresh e-commerce can be divided into three stages: rapid development (2012-2015), exploration and transformation (2016-2019), maturity and upgrade (2020-present). At each stage, the research evolves toward diversity and maturity with policy developments and changes in the external environment. Cold chain logistics, business models, freshness-keeping of products and e-commerce are ongoing research hotspots in fresh produce e-commerce, while later studies focus more on the transformation and upgrade of products, logistics, distribution and platforms to better serve consumers’ consumption habits and environmental requirements. This study provides valuable insights for researchers and enterprises who are engaged in the industry and for those who are interested in the development of fresh e-commerce in China

Sales forecasting of stores in shopping malls: A study based on external data and transaction data

To improve the forecast accuracy of the sales of stores in shopping malls, this paper proposes a prediction method based on deep learning that comprehensively considers the external data, such as online review data of shopping mall stores, weather data, weekday/weekend data, and historical transaction data of the stores. To begin with, the online review data of the stores are pre-trained with BERT (Bidirectional Encoder Representations from Transformers) to complete the multi-label sentiment classification and obtain the intensity index of perceived sentiment of reviews. The index, together with other external data, such as online ratings, weather, weekday/weekend differences, and historical transactions of the stores, is pre-processed. At last, the Long Short-Term Memory (LSTM) and the Attention models are used to predict the sales volume of stores in a certain shopping mall. The results show that the addition of external data – weather, weekday/weekend, online ratings and intensity index of sentiment of reviews – to the historical sales data-based model can effectively improve the forecast accuracy of store sales

Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model

Integrating large language models (LLMs) into healthcare presents potential but faces challenges. Directly pre-training LLMs for domains like medicine is resource-heavy and sometimes unfeasible. Sole reliance on Supervised Fine-tuning (SFT) can result in overconfident predictions and may not tap into domain specific insights. Addressing these challenges, we present a multi-stage training method combining Domain-specific Continued Pre-training (DCPT), SFT, and Direct Preference Optimization (DPO). A notable contribution of our study is the introduction of a 3Gb Chinese Medicine (ChiMed) dataset, encompassing medical question answering, plain texts, knowledge graphs, and dialogues, segmented into three training stages. The medical LLM trained with our pipeline, Qilin-Med, exhibits significant performance boosts. In the CPT and SFT phases, it achieves 38.4% and 40.0% accuracy on the CMExam, surpassing Baichuan-7B's 33.5%. In the DPO phase, on the Huatuo-26M test set, it scores 16.66 in BLEU-1 and 27.44 in ROUGE1, outperforming the SFT's 12.69 and 24.21. This highlights the strength of our training approach in refining LLMs for medical applications

ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding

Spoken language understanding (SLU) is a fundamental task in the task-oriented dialogue systems. However, the inevitable errors from automatic speech recognition (ASR) usually impair the understanding performance and lead to error propagation. Although there are some attempts to address this problem through contrastive learning, they (1) treat clean manual transcripts and ASR transcripts equally without discrimination in fine-tuning; (2) neglect the fact that the semantically similar pairs are still pushed away when applying contrastive learning; (3) suffer from the problem of Kullback-Leibler (KL) vanishing. In this paper, we propose Mutual Learning and Large-Margin Contrastive Learning (ML-LMCL), a novel framework for improving ASR robustness in SLU. Specifically, in fine-tuning, we apply mutual learning and train two SLU models on the manual transcripts and the ASR transcripts, respectively, aiming to iteratively share knowledge between these two models. We also introduce a distance polarization regularizer to avoid pushing away the intra-cluster pairs as much as possible. Moreover, we use a cyclical annealing schedule to mitigate KL vanishing issue. Experiments on three datasets show that ML-LMCL outperforms existing models and achieves new state-of-the-art performance

Rethinking Multi-Interest Learning for Candidate Matching in Recommender Systems

Existing research efforts for multi-interest candidate matching in recommender systems mainly focus on improving model architecture or incorporating additional information, neglecting the importance of training schemes. This work revisits the training framework and uncovers two major problems hindering the expressiveness of learned multi-interest representations. First, the current training objective (i.e., uniformly sampled softmax) fails to effectively train discriminative representations in a multi-interest learning scenario due to the severe increase in easy negative samples. Second, a routing collapse problem is observed where each learned interest may collapse to express information only from a single item, resulting in information loss. To address these issues, we propose the REMI framework, consisting of an Interest-aware Hard Negative mining strategy (IHN) and a Routing Regularization (RR) method. IHN emphasizes interest-aware hard negatives by proposing an ideal sampling distribution and developing a Monte-Carlo strategy for efficient approximation. RR prevents routing collapse by introducing a novel regularization term on the item-to-interest routing matrices. These two components enhance the learned multi-interest representations from both the optimization objective and the composition information. REMI is a general framework that can be readily applied to various existing multi-interest candidate matching methods. Experiments on three real-world datasets show our method can significantly improve state-of-the-art methods with easy implementation and negligible computational overhead. The source code will be released.Comment: RecSys 202

Histone deacetylase 11 regulates oligodendrocyte-specific gene expression and cell development in OL-1 oligodendroglia cells

Both in vivo and in vitro studies indicate a correlation between reduced acetylation of histone core proteins and oligodendrocyte development. The nature of these histone modifications and the mechanisms mediating them remain undefined. To address these issues we utilized OL-1 cells, a rat non-transformed oligodendrocyte cell line, and primary oligodendrocyte cultures. We found that the acetylated histone H3 at lysine 9 and lysine 14 (H3K9/K14ac) is reduced in both the myelin basic protein (MBP) and proteolipid protein (PLP) genes of maturing oligodendroglial OL-1 cells, and furthermore, this temporally correlates with increases in MBP, PLP, and histone deacetylase (HDAC) 11 expression. Disruption of developmentally-regulated histone H3 deacetylation within the MBP and PLP genes by the HDAC inhibitor trichostatin A blunts MBP and PLP expression. With its increased expression, interaction of HDAC 11 with acetylated histone H3 and recruitment of HDAC 11 to the MBP and PLP genes markedly increases in maturing OL-1 cells. Moreover, suppressing HDAC 11 expression with small interfering RNA significantly: 1) increases H3K9/K14ac globally and within the MBP and PLP genes, 2) decreases MBP and PLP mRNA expression, and 3) blunts the morphological changes associated with oligodendrocyte development. Our data strongly support a specific role for HDAC 11 in histone deacetylation and in turn the regulation of oligodendrocyte-specific protein gene expression and oligodendrocyte development

Equivariant Contrastive Learning for Sequential Recommendation

Contrastive learning (CL) benefits the training of sequential recommendation models with informative self-supervision signals. Existing solutions apply general sequential data augmentation strategies to generate positive pairs and encourage their representations to be invariant. However, due to the inherent properties of user behavior sequences, some augmentation strategies, such as item substitution, can lead to changes in user intent. Learning indiscriminately invariant representations for all augmentation strategies might be suboptimal. Therefore, we propose Equivariant Contrastive Learning for Sequential Recommendation (ECL-SR), which endows SR models with great discriminative power, making the learned user behavior representations sensitive to invasive augmentations (e.g., item substitution) and insensitive to mild augmentations (e.g., featurelevel dropout masking). In detail, we use the conditional discriminator to capture differences in behavior due to item substitution, which encourages the user behavior encoder to be equivariant to invasive augmentations. Comprehensive experiments on four benchmark datasets show that the proposed ECL-SR framework achieves competitive performance compared to state-of-the-art SR models. The source code is available at https://github.com/Tokkiu/ECL.Comment: Accepted by RecSys 202

β-catenin mediates insulin-like growth factor-I actions to promote cyclin D1 mRNA expression, cell proliferation and survival in oligodendroglial cultures

By promoting cell proliferation, survival and maturation insulin-like growth factor (IGF)-I is essential to the normal growth and development of the central nervous system. It is clear that IGF-I actions are primarily mediated by the type I IGF receptor (IGF1R), and that phosphoinositide 3 (PI3)-Akt kinases and MAP kinases signal many of IGF-I-IGF1R actions in neural cells, including oligodendrocyte lineage cells. The precise downstream targets of these signaling pathways, however, remain to be defined. We studied oligodendroglial cells to determine whether β-catenin, a molecule that is a downstream target of glycogen synthase kinase-3β (GSK3β) and plays a key role in the Wnt canonical signaling pathway, mediates IGF-I actions. We found that IGF-I increases β-catenin protein abundance within an hour after IGF-I-induced phosphorylation of Akt and GSK3β. Inhibiting the PI3-Akt pathway suppressed IGF-I-induced increases in β-catenin and cyclin D1 mRNA, while suppression of GSK3β activity simulated IGF-I actions. Knocking-down β-catenin mRNA by RNA interference suppressed IGF-I-stimulated increases in the abundance of cyclin D1 mRNA, cell proliferation, and cell survival. Our data suggest that β-catenin is an important downstream molecule in the PI3-Akt-GSK3β pathway, and as such it mediates IGF-I upregulation of cyclin D1 mRNA and promotion of cell proliferation and survival in oligodendroglial cells

Developmental expression of histone deacetylase 11 in the murine brain

Recent studies indicate that neural cell development in the central nervous system (CNS) correlates with a reduction in acetylation of histone core proteins. Moreover, histone hypoacetylation is thought to be important to oligodendrocyte lineage development. The mechanisms mediating the reduction in acetylation during postnatal neural development remain to be defined. To begin to understand these mechanisms, we investigated the expression of histone deacetylase 11 (HDAC11), a newly identified HDAC, in mouse brain during postnatal development. We show that HDAC11 was widely expressed in the brain and that this expression gradually increased in a region-specific pattern between birth and 4 weeks of age. At the cellular level HDAC11 protein was predominately localized in the nuclei of mature oligodendrocytes but only minimally in astrocytes. Although dentate gyrus granule neurons abundantly expressed HDAC11, granule neuron precursors in the subgranule layer exhibited little HDAC11 immunoreactivity. Double-immunostaining of the corpus callosum and dentate gyrus demonstrated that HDAC11 and Ki67, a cell-proliferating marker, are rarely colocalized in same cells. Our data show that HDAC11 was expressed in the developing brain in a temporal and spatial pattern that correlates with the maturation of neural cells, including cells of the oligodendrocyte lineage. These findings support a role for HDAC11 in CNS histone deacetylation and the development of oligodendrocytes and neurons during postnatal development