Multi-Relational Contrastive Learning for Recommendation

Personalized recommender systems play a crucial role in capturing users' evolving preferences over time to provide accurate and effective recommendations on various online platforms. However, many recommendation models rely on a single type of behavior learning, which limits their ability to represent the complex relationships between users and items in real-life scenarios. In such situations, users interact with items in multiple ways, including clicking, tagging as favorite, reviewing, and purchasing. To address this issue, we propose the Relation-aware Contrastive Learning (RCL) framework, which effectively models dynamic interaction heterogeneity. The RCL model incorporates a multi-relational graph encoder that captures short-term preference heterogeneity while preserving the dedicated relation semantics for different types of user-item interactions. Moreover, we design a dynamic cross-relational memory network that enables the RCL model to capture users' long-term multi-behavior preferences and the underlying evolving cross-type behavior dependencies over time. To obtain robust and informative user representations with both commonality and diversity across multi-behavior interactions, we introduce a multi-relational contrastive learning paradigm with heterogeneous short- and long-term interest modeling. Our extensive experimental studies on several real-world datasets demonstrate the superiority of the RCL recommender system over various state-of-the-art baselines in terms of recommendation accuracy and effectiveness.Comment: This paper has been published as a full paper at RecSys 202

Leveraging Multi-level Dependency of Relational Sequences for Social Spammer Detection

Much recent research has shed light on the development of the relation-dependent but content-independent framework for social spammer detection. This is largely because the relation among users is difficult to be altered when spammers attempt to conceal their malicious intents. Our study investigates the spammer detection problem in the context of multi-relation social networks, and makes an attempt to fully exploit the sequences of heterogeneous relations for enhancing the detection accuracy. Specifically, we present the Multi-level Dependency Model (MDM). The MDM is able to exploit user's long-term dependency hidden in their relational sequences along with short-term dependency. Moreover, MDM fully considers short-term relational sequences from the perspectives of individual-level and union-level, due to the fact that the type of short-term sequences is multi-folds. Experimental results on a real-world multi-relational social network demonstrate the effectiveness of our proposed MDM on multi-relational social spammer detection

A Survey on Knowledge Graphs: Representation, Acquisition and Applications

Human knowledge provides a formal understanding of the world. Knowledge graphs that represent structural relations between entities have become an increasingly popular research direction towards cognition and human-level intelligence. In this survey, we provide a comprehensive review of knowledge graph covering overall research topics about 1) knowledge graph representation learning, 2) knowledge acquisition and completion, 3) temporal knowledge graph, and 4) knowledge-aware applications, and summarize recent breakthroughs and perspective directions to facilitate future research. We propose a full-view categorization and new taxonomies on these topics. Knowledge graph embedding is organized from four aspects of representation space, scoring function, encoding models, and auxiliary information. For knowledge acquisition, especially knowledge graph completion, embedding methods, path inference, and logical rule reasoning, are reviewed. We further explore several emerging topics, including meta relational learning, commonsense reasoning, and temporal knowledge graphs. To facilitate future research on knowledge graphs, we also provide a curated collection of datasets and open-source libraries on different tasks. In the end, we have a thorough outlook on several promising research directions

Serialized Knowledge Enhanced Multi-objective Person-job Matching Recommendation in a High Mobility Job Market

In a high mobility job market, accumulated historical sequences information from persons and jobs bring opportunities and challenges to person-job matching recommendation, where the latent preferences may significantly determine the success of person-job matching. Moreover, the sparse labels further limit the learning performance of recommendation methods. To this end, we propose a novel serialized knowledge enhancement multi-objective person-job matching recommendation method, namely SMP-JM. The key idea is to design a serialized multi-objective method from “intention-delivery-review”, which effectively solves the problem of sparsity through the transmission of information and the serialization constraints between objectives. Specifically, we design various attention modules, such as self-attention, cross-attention and an orthogonal multi-head attention, to identify correlations between diversified features. Furthermore, a multi-granularity convolutional filtering module is design to extract personal latent preference from the historical sequential behaviors. Finally, the experimental results on a real-world dataset validate the performance of SMP-JM over the baseline methods

토큰 단위 분류모델을 위한 중요 토큰 포착 및 시퀀스 인코더 설계 방법

학위논문(박사) -- 서울대학교대학원 : 공과대학 전기·정보공학부, 2022. 8. 정교민.With the development of internet, a great of volume of data have accumulated over time. Therefore, dealing long sequential data can become a core problem in web services. For example, streaming services such as YouTube, Netflx and Tictoc have used the user's viewing history sequence to recommend videos that users may like. Such systems have replaced the user's viewed video with each item or token to predict what item or token will be viewed next. These tasks have been defined as Token-Level Classification (TLC) tasks. Given the sequence of tokens, TLC identifies the labels of tokens in the required portion of this sequence. As mentioned above, TLC can be applied to various recommendation Systems. In addition, most of Natural Language Processing (NLP) tasks can also be formulated as TLC problem. For example, sentence and each word within the sentence can be expressed as token-level sequence. In particular, in the case of information extraction, it can be changed to a TLC task that distinguishes whether a specific word span in the sentence is information. The characteristics of TLC datasets are that they are very sparse and long. Therefore, it is a very important problem to extract only important information from the sequences and properly encode them. In this thesis, we propose the method to solve the two academic questions of TLC in Recommendation Systems and information extraction: 1) How to capture important tokens from the token sequence and 2) How to encode a token sequence into model. As deep neural networks (DNNs) have shown outstanding performance in various web application tasks, we design the RNN and Transformer-based model for recommendation systems, and information extractions. In this dissertation, we propose novel models that can extract important tokens for recommendation systems and information extraction systems. In recommendation systems, we design a BART-based system that can capture important portion of token sequence through self-attention mechanisms and consider both bidirectional and left-to-right directional information. In information systems, we present relation network-based models to focus important parts such as opinion target and neighbor words.인터넷의 발달로, 많은 양의 데이터가 시간이 지남에 따라 축적되었다. 이로인해 긴 순차적 데이터를 처리하는 것은 웹 서비스의 핵심 문제가 되었다. 예를 들어, 유튜브, 넷플릭스, 틱톡과 같은 스트리밍 서비스는 사용자의 시청 기록 시퀀스를 사용하여 사용자가 좋아할 만한 비디오를 추천한다. 이러한 시스템은 다음에 어떤 항목이나 토큰을 볼 것인지를 예측하기 위해 사용자가 본 비디오를 각 항목 또는 토큰으로 대체하여 사용할 수 있다. 이러한 작업은 토큰 수준 분류(TLC) 작업으로 정의한다. 토큰 시퀀스가 주어지면, TLC는 이 시퀀스의 필요한 부분에서 토큰의 라벨을 식별한다. 이렇게와 같이, TLC는 다양한 추천 시스템에 적용될 수 있다. 또한 대부분의 자연어 처리(NLP) 작업은 TLC 문제로 공식화될 수 있다. 예를 들어, 문장과 문장 내의 각 단어는 토큰 레벨 시퀀스로 표현될 수 있다. 특히 정보 추출의 경우 문장의 특정 단어 간격이 정보인지 여부를 구분하는 TLC 작업으로 바뀔 수 있다. TLC 데이터 세트의 특징은 매우 희박(Sparse)하고 길다는 것이다. 따라서 시퀀스에서 중요한 정보만 추출하여 적절히 인코딩하는 것은 매우 중요한 문제이다. 본 논문에서는 권장 시스템과 정보 추출에서 TLC의 두 가지 학문적 질문- 1) 토큰 시퀀스에서 중요한 토큰을 캡처하는 방법 및 2) 토큰 시퀀스를 모델로 인코딩하는 방법 을 해결하는 방법을 제안한다. 심층 신경망(DNN)이 다양한 웹 애플리케이션 작업에서 뛰어난 성능을 보여 왔기 때문에 추천 시스템 및 정보 추출을 위한 RNN 및 트랜스포머 기반 모델을 설계한다. 먼저 우리는 자기 주의 메커니즘을 통해 토큰 시퀀스의 중요한 부분을 포착하고 양방향 및 좌우 방향 정보를 모두 고려할 수 있는 BART 기반 추천 시스템을 설계한다. 정보 시스템에서, 우리는 의견 대상과 이웃 단어와 같은 중요한 부분에 초점을 맞추기 위해 관계 네트워크 기반 모델을 제시한다.1. Introduction 1 2. Token-level Classification in Recommendation Systems 8 2.1 Overview 8 2.2 Hierarchical RNN-based Recommendation Systems 19 2.3 Entangled Bidirectional Encoder to Auto-regressive Decoder for Sequential Recommendation 27 3. Token-level Classification in Information Extraction 39 3.1 Overview 39 3.2 RABERT: Relation-Aware BERT for Target-Oriented Opinion Words Extraction 49 3.3 Gated Relational Target-aware Encoder and Local Context-aware Decoder for Target-oriented Opinion Words Extraction 58 4. Conclusion 79박