PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information

With the increase of content pages and interactive buttons in online services such as online-shopping and video-watching websites, industrial-scale recommender systems face challenges in multi-domain and multi-task recommendations. The core of multi-task and multi-domain recommendation is to accurately capture user interests in multiple scenarios given multiple user behaviors. In this paper, we propose a plug-and-play \textit{\textbf{P}arameter and \textbf{E}mbedding \textbf{P}ersonalized \textbf{Net}work (\textbf{PEPNet})} for multi-domain and multi-task recommendation. PEPNet takes personalized prior information as input and dynamically scales the bottom-level Embedding and top-level DNN hidden units through gate mechanisms. \textit{Embedding Personalized Network (EPNet)} performs personalized selection on Embedding to fuse features with different importance for different users in multiple domains. \textit{Parameter Personalized Network (PPNet)} executes personalized modification on DNN parameters to balance targets with different sparsity for different users in multiple tasks. We have made a series of special engineering optimizations combining the Kuaishou training framework and the online deployment environment. By infusing personalized selection of Embedding and personalized modification of DNN parameters, PEPNet tailored to the interests of each individual obtains significant performance gains, with online improvements exceeding 1\% in multiple task metrics across multiple domains. We have deployed PEPNet in Kuaishou apps, serving over 300 million users every day.Comment: Accepted by KDD 202

Understanding and Modeling Passive-Negative Feedback for Short-video Sequential Recommendation

Sequential recommendation is one of the most important tasks in recommender systems, which aims to recommend the next interacted item with historical behaviors as input. Traditional sequential recommendation always mainly considers the collected positive feedback such as click, purchase, etc. However, in short-video platforms such as TikTok, video viewing behavior may not always represent positive feedback. Specifically, the videos are played automatically, and users passively receive the recommended videos. In this new scenario, users passively express negative feedback by skipping over videos they do not like, which provides valuable information about their preferences. Different from the negative feedback studied in traditional recommender systems, this passive-negative feedback can reflect users' interests and serve as an important supervision signal in extracting users' preferences. Therefore, it is essential to carefully design and utilize it in this novel recommendation scenario. In this work, we first conduct analyses based on a large-scale real-world short-video behavior dataset and illustrate the significance of leveraging passive feedback. We then propose a novel method that deploys the sub-interest encoder, which incorporates positive feedback and passive-negative feedback as supervision signals to learn the user's current active sub-interest. Moreover, we introduce an adaptive fusion layer to integrate various sub-interests effectively. To enhance the robustness of our model, we then introduce a multi-task learning module to simultaneously optimize two kinds of feedback -- passive-negative feedback and traditional randomly-sampled negative feedback. The experiments on two large-scale datasets verify that the proposed method can significantly outperform state-of-the-art approaches. The code is released at https://github.com/tsinghua-fib-lab/RecSys2023-SINE.Comment: Accepted by RecSys'2

Lithosphere thinning beneath west North China Craton: Evidence from geochemical and Sr-Nd-Hf isotope compositions of Jining basalts

This study shows lithosphere evolution history in the west North China Craton (NCC) from the early Cretaceous to Quaternary by studying the major element, trace element and Sr-Nd-Hf isotope compositions in Jining basalts of 119.6-108.6. Ma, 23.5-21.9. Ma and 1.3-0.11. Ma.The early Cretaceous basalts (119.6-108.6Ma) display enriched characteristics with high contents of incompatible elements, high 87Sr/86Sri, low εNd(t) and low εHf(t). These basalts resulted from partial melting of ancient metasomatized lithospheric mantle, and we consider the 119.6-108.6Ma magmatism as indicating lithosphere thinning in the west NCC. Although the Pacific slab seen seismically in the mantle transition zone beneath eastern China is no older than 60Ma, there exists convincing evidence for the presence of the Paleo-Pacific slab in the transition-zone in the Mesozoic. Thus we propose that the water released from the transition-zone slab hydrated the overlying lithosphere and further converted the base of the lithosphere into asthenosphere. This is the most likely mechanism responsible for the lithosphere thinning in the west NCC and the petrogenesis of the Jining 119.6-108.6Ma basalts.The Jining 23.5-21.9Ma basalts also have high contents of incompatible elements, but they display high εNd(t), high εHf(t) and variably low 87Sr/86Sri. We propose that these Miocene basalts were derived from the asthenosphere with contributions from ancient metasomatized lithospheric mantle during melt ascent. The Jining Quaternary basalts (1.3-0.11Ma) represent the melt of upwelling asthenosphere with low 87Sr/86Sri, high εNd(t) and high εHf(t). Upwelling and decompression melting of the eastward flowing asthenosphere from beneath western plateaus to beneath eastern hilly plains in the Cenozoic is the most plausible mechanism for the petrogenesis of Jining Cenozoic basalts (both of 23.5-21.9Ma and 1.3-0.11Ma), but the Jining 1.3-0.11Ma basalts must have been produced beneath even thinner lithosphere.Taken together geophysical studies and our petrological and geochemical studies of all these three episodes of the Jining basalts, we propose that the lithosphere in the west NCC has been thinning since the early Cretaceous and the thinning continues to the present

TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou

Life-long user behavior modeling, i.e., extracting a user's hidden interests from rich historical behaviors in months or even years, plays a central role in modern CTR prediction systems. Conventional algorithms mostly follow two cascading stages: a simple General Search Unit (GSU) for fast and coarse search over tens of thousands of long-term behaviors and an Exact Search Unit (ESU) for effective Target Attention (TA) over the small number of finalists from GSU. Although efficient, existing algorithms mostly suffer from a crucial limitation: the \textit{inconsistent} target-behavior relevance metrics between GSU and ESU. As a result, their GSU usually misses highly relevant behaviors but retrieves ones considered irrelevant by ESU. In such case, the TA in ESU, no matter how attention is allocated, mostly deviates from the real user interests and thus degrades the overall CTR prediction accuracy. To address such inconsistency, we propose \textbf{TWo-stage Interest Network (TWIN)}, where our Consistency-Preserved GSU (CP-GSU) adopts the identical target-behavior relevance metric as the TA in ESU, making the two stages twins. Specifically, to break TA's computational bottleneck and extend it from ESU to GSU, or namely from behavior length $10^2$ to length $10^4-10^5$, we build a novel attention mechanism by behavior feature splitting. For the video inherent features of a behavior, we calculate their linear projection by efficient pre-computing \& caching strategies. And for the user-item cross features, we compress each into a one-dimentional bias term in the attention score calculation to save the computational cost. The consistency between two stages, together with the effective TA-based relevance metric in CP-GSU, contributes to significant performance gain in CTR prediction.Comment: Accepted by KDD 202

New Superhard Carbon Phases Between Graphite and Diamond

Two new carbon allotropes (H-carbon and S-carbon) are proposed, as possible candidates for the intermediate superhard phases between graphite and diamond obtained in the process of cold compressing graphite, based on the results of first-principles calculations. Both H-carbon and S-carbon are more stable than previously proposed M-carbon and W-carbon and their bulk modulus are comparable to that of diamond. H-carbon is an indirect-band-gap semiconductor with a gap of 4.459 eV and S-carbon is a direct-band-gap semiconductor with a gap of 4.343 eV. The transition pressure from cold compressing graphite is 10.08 GPa and 5.93 Gpa for H-carbon and S-carbon, respectively, which is in consistent with the recent experimental report.Comment: 5pages,4figures,submitted to Phys.Rev.Lett on 18Jan12, transfer to Phys.Rev.B on 25Mar12; Solid State Communications(2012), http://dx.doi.org/10.1016/j.ssc.2012.05.02

Mixed Attention Network for Cross-domain Sequential Recommendation

In modern recommender systems, sequential recommendation leverages chronological user behaviors to make effective next-item suggestions, which suffers from data sparsity issues, especially for new users. One promising line of work is the cross-domain recommendation, which trains models with data across multiple domains to improve the performance in data-scarce domains. Recent proposed cross-domain sequential recommendation models such as PiNet and DASL have a common drawback relying heavily on overlapped users in different domains, which limits their usage in practical recommender systems. In this paper, we propose a Mixed Attention Network (MAN) with local and global attention modules to extract the domain-specific and cross-domain information. Firstly, we propose a local/global encoding layer to capture the domain-specific/cross-domain sequential pattern. Then we propose a mixed attention layer with item similarity attention, sequence-fusion attention, and group-prototype attention to capture the local/global item similarity, fuse the local/global item sequence, and extract the user groups across different domains, respectively. Finally, we propose a local/global prediction layer to further evolve and combine the domain-specific and cross-domain interests. Experimental results on two real-world datasets (each with two domains) demonstrate the superiority of our proposed model. Further study also illustrates that our proposed method and components are model-agnostic and effective, respectively. The code and data are available at https://github.com/Guanyu-Lin/MAN.Comment: WSDM 202