One Model for All: Large Language Models are Domain-Agnostic Recommendation Systems

The purpose of sequential recommendation is to utilize the interaction history of a user and predict the next item that the user is most likely to interact with. While data sparsity and cold start are two challenges that most recommender systems are still facing, many efforts are devoted to utilizing data from other domains, called cross-domain methods. However, general cross-domain methods explore the relationship between two domains by designing complex model architecture, making it difficult to scale to multiple domains and utilize more data. Moreover, existing recommendation systems use IDs to represent item, which carry less transferable signals in cross-domain scenarios, and user cross-domain behaviors are also sparse, making it challenging to learn item relationship from different domains. These problems hinder the application of multi-domain methods to sequential recommendation. Recently, large language models (LLMs) exhibit outstanding performance in world knowledge learning from text corpora and general-purpose question answering. Inspired by these successes, we propose a simple but effective framework for domain-agnostic recommendation by exploiting the pre-trained LLMs (namely LLM-Rec). We mix the user's behavior across different domains, and then concatenate the title information of these items into a sentence and model the user's behaviors with a pre-trained language model. We expect that by mixing the user's behaviors across different domains, we can exploit the common knowledge encoded in the pre-trained language model to alleviate the problems of data sparsity and cold start problems. Furthermore, we are curious about whether the latest technical advances in nature language processing (NLP) can transfer to the recommendation scenarios.Comment: 10 pages, 7 figures, 6 table

Rethinking Memory and Communication Cost for Efficient Large Language Model Training

Recently, various distributed strategies for large language model training have been proposed. However, these methods provided limited solutions for the trade-off between memory consumption and communication cost. In this paper, we rethink the impact of memory consumption and communication costs on the training speed of large language models, and propose a memory-communication balanced strategy set Partial Redundancy Optimizer (PaRO). PaRO provides comprehensive options which reduces the amount and frequency of inter-group communication with minor memory redundancy by fine-grained sharding strategy, thereby improving the training efficiency in various training scenarios. Additionally, we propose a Hierarchical Overlapping Ring (HO-Ring) communication topology to enhance communication efficiency between nodes or across switches in large language model training. Our experiments demonstrate that PaRO significantly improves training throughput by 1.19x-2.50x compared to the SOTA method and achieves a near-linear scalability. The HO-Ring algorithm improves communication efficiency by 36.5% compared to the traditional Ring algorithm

高血清流行人群新生儿先天性巨细胞病毒感染筛查策略对比：一项母婴队列研究

巨细胞病毒（Cytomegalovirus，CMV）感染极为常见，感染人体后将造成终身潜伏带毒、机会性活跃，大多不引起明显的临床症状，但孕妇的CMV病毒活跃可能形成垂直传播而导致新生儿宫内先天感染，损伤胎盘并在胎儿神经细胞中复制。CMV是全球儿童感音神经性耳聋的最主要病因。及时筛查确诊新生儿CMV先天感染是改善患儿临床结局的关键。该研究团体通过多中心母婴队列观察，系统对比了基于新生儿唾液和尿液标本进行先天性巨细胞病毒感染筛查的多种策略，提出了初次筛查和再次确认检测的最优样本采集时间窗口和样本类型，对包括中国以及大多数发展中国家在内的广大巨细胞病毒感染高流行区的先天性巨细胞病毒感染防控工作具有重要指导意义。 我校博士生黄悦、硕士生王晗、高级工程师李廷栋和新密市妇幼保健院李彩红主任医师为该论文的共同第一作者。我校张军教授、葛胜祥教授和美国德克萨斯大学休斯顿健康科学中心傅通明博士为该论文的共同通讯作者。Background: Universal screening of congenital cytomegalovirus (cCMV) infection is important for monitoring and intervention during critical stages of speech and language development. This study aimed to explore the optimal detection strategy for cCMV infection screening. Methods: Serum samples from pregnant women and saliva and urine samples from their newborns were collected for the anti-CMV IgG and CMV DNA PCR tests, respectively. The sensitivity, specificity, and predictive values as well as the likelihood ratios of 12 potential screening strategies for cCMV infection, based on tests for saliva, urine, and their combination, were evaluated. Findings: A total of 6729 pregnant women were enrolled, and the seroprevalence was 98.1%. Among 6350 newborns that were followed up, 49 were defined as having cCMV infection. In the screening test, the CMV DNA positivity rate remained similar from day 0 to day 5, increased slowly from day 6 to day 13, and became high in newborns beyond 13 days of birth. In the confirmatory testing, the positive rates increased significantly beyond day 21. For the 49 newborns with cCMV infection, the proportion of agreement between saliva and urine testing was poor. Upon evaluating alternative screening strategies, using saliva and urine screening with saliva and urine confirmation as the reference strategy, saliva screening with saliva and urine confirmation showed good diagnostic accuracy and feasibility, with sensitivity, specificity, positive predictive and negative predictive values of 85.7%, 100.0%, 100.0% and 99.9%, respectively. Interpretation: In populations with high seroprevalence, saliva screening with saliva and urine confirmation might be an alternative strategy for screening cCMV infections. The suggested timeframes for screening and confirmation are within 13 (ideally 5) and 21 (ideally 13) days of birth, respectively.This work was supported by the National Natural Science Foundation of China (grants 81973058 and 81672111), the National Science and Technology Major Project of China (2017ZX10302201-002-003) and Merck & Co., Inc., Kenilworth, New Jersey, U.S.A. 该研究获得国家自然科学基金、传染病重大专项、厦门大学校长基金及美国默沙东公司支持