그래프 학습시 콜드 스타트 문제 해결을 위한 멀티테스크 학습 전략

학위논문(석사) -- 서울대학교대학원 : 자연과학대학 협동과정 뇌과학전공, 2021.8. 장병탁.Data in real world can commonly be represented as graph data. From molecules, social networks, Internet, and even natural language can be represented as graph. Therefore, generating fine representations from the graph data is critical in machine learning. One of the common attributes of most real-world graphs is that graphs are dynamic and can eventually face the cold start problem. A fundamental question is how the new cold nodes acquire initial information in order to be adapted into the existing graph. Here we postulates the cold start problem as a fundamental issue in graph learning and propose a new learning setting, “Expanded Semi-supervised Learning.” In expanded semi-supervised learning we extend the original semi-supervised learning setting even to new cold nodes that are disconnected from the graph. To this end, we propose Cold- Expand model that classifies the cold nodes based on link prediction with mul- tiple goals to tackle. We experimentally prove that by adding additional goal to existing link prediction method, our method outperforms the baseline in both expanded semi-supervised link prediction (at most 24%) and node classifica- tion tasks (at most 15%). To the best of our knowledge this is the first study to address expansion of semi-supervised learning to unseen nodes.대다수의 실제하는 그래프 데이터는 시간에 따라 노드와 그 연결이 변화하고 필연 적으로 콜드스타트 문제를 직면하게 된다. 콜드스타트 문제를 해결하기 위해서는 새로 발생한 콜드 노드들이 기존 그래프에 포함되기 위해 어떻게 초기 정보를 얻을 수 있는지가 매우 중요하다. 본 논문에서는 그래프 학습에서의 콜드스타트 문제를 제기하고 새롭고 현실적인 학습 세팅인 ”확장된 준지도학습”을 제안한다. 확장된 준지도학습에서는 기존의 준지도 학습을 새로 소개되어 기존 그래프와의 연결성 이 없는 콜드 노드들에도 적용한다. 새로운 학습 세팅을 위해 논문에서는 콜드 노드들의 분류를 위해 링크 예측을 함께 멀티테스크 목적으로 갖는 ColdExpand 모델을 제시한다. 또한 실험적으로 우리 모델이 추가적인 목적을 모델에 더해줌 으로써 콜드 노드들의 링크 예측과 분류 문제에서 과거 모델들을 크게 상회하는 결과가 나옴을 증명한다.Chapter 1 Introduction 1 Chapter 2 Related Works 5 2.1 Semi-supervisedNodeClassification 5 2.2 LinkPrediction 6 2.3 ColdStartProblem 8 2.4 Multi-taskLearningStrategy 9 Chapter 3 Method 11 3.1 ProblemDefinition 11 3.2 ColdExpand: Semi-Supervised Graph Learning in Cold Start 12 3.2.1 LinkPredictionofColdNodes 12 3.2.2 NodeClassificationofColdNodes 13 Chapter 4 Experiment 17 4.1 Dataset 17 4.2 Baseline 18 4.2.1 Expanded Semi-supervised Link Prediction 18 4.2.2 Expanded Semi-supervised Node Classification 18 4.3 Results. 19 4.3.1 Results of Link Prediction on Cold Nodes 19 4.3.2 Results of Semi-supervised Node Classification on Cold Nodes. 20 4.3.3 Qualitative Results for Node Classification in Cold Start Environment 23 4.3.4 ColdExpand also Improves Conventional Semi-supervised NodeClassification 23 4.3.5 AblationStudy 25 Chapter 5 Discussion 28석

Deep Learning for Link Prediction in Dynamic Networks using Weak Estimators

Link prediction is the task of evaluating the probability that an edge exists in a network, and it has useful applications in many domains. Traditional approaches rely on measuring the similarity between two nodes in a static context. Recent research has focused on extending link prediction to a dynamic setting, predicting the creation and destruction of links in networks that evolve over time. Though a difficult task, the employment of deep learning techniques have shown to make notable improvements to the accuracy of predictions. To this end, we propose the novel application of weak estimators in addition to the utilization of traditional similarity metrics to inexpensively build an effective feature vector for a deep neural network. Weak estimators have been used in a variety of machine learning algorithms to improve model accuracy, owing to their capacity to estimate changing probabilities in dynamic systems. Experiments indicate that our approach results in increased prediction accuracy on several real-world dynamic networks

Node Embedding over Temporal Graphs

In this work, we present a method for node embedding in temporal graphs. We propose an algorithm that learns the evolution of a temporal graph's nodes and edges over time and incorporates this dynamics in a temporal node embedding framework for different graph prediction tasks. We present a joint loss function that creates a temporal embedding of a node by learning to combine its historical temporal embeddings, such that it optimizes per given task (e.g., link prediction). The algorithm is initialized using static node embeddings, which are then aligned over the representations of a node at different time points, and eventually adapted for the given task in a joint optimization. We evaluate the effectiveness of our approach over a variety of temporal graphs for the two fundamental tasks of temporal link prediction and multi-label node classification, comparing to competitive baselines and algorithmic alternatives. Our algorithm shows performance improvements across many of the datasets and baselines and is found particularly effective for graphs that are less cohesive, with a lower clustering coefficient