Unsupervised Semantic Representation Learning of Scientific Literature Based on Graph Attention Mechanism and Maximum Mutual Information

Since most scientific literature data are unlabeled, this makes unsupervised graph-based semantic representation learning crucial. Therefore, an unsupervised semantic representation learning method of scientific literature based on graph attention mechanism and maximum mutual information (GAMMI) is proposed. By introducing a graph attention mechanism, the weighted summation of nearby node features make the weights of adjacent node features entirely depend on the node features. Depending on the features of the nearby nodes, different weights can be applied to each node in the graph. Therefore, the correlations between vertex features can be better integrated into the model. In addition, an unsupervised graph contrastive learning strategy is proposed to solve the problem of being unlabeled and scalable on large-scale graphs. By comparing the mutual information between the positive and negative local node representations on the latent space and the global graph representation, the graph neural network can capture both local and global information. Experimental results demonstrate competitive performance on various node classification benchmarks, achieving good results and sometimes even surpassing the performance of supervised learning

진료 내역 데이터를 활용한 딥러닝 기반의 건강보험 남용 탐지

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 산업공학과, 2020. 8. 조성준.As global life expectancy increases, spending on healthcare grows in accordance in order to improve quality of life. However, due to expensive price of medical care, the bare cost of healthcare services would inevitably places great financial burden to individuals and households. In this light, many countries have devised and established their own public healthcare insurance systems to help people receive medical services at a lower price. Since reimbursements are made ex-post, unethical practices arise, exploiting the post-payment structure of the insurance system. The archetypes of such behavior are overdiagnosis, the act of manipulating patients diseases, and overtreatments, prescribing unnecessary drugs for the patient. These abusive behaviors are considered as one of the main sources of financial loss incurred in the healthcare system. In order to detect and prevent abuse, the national healthcare insurance hires medical professionals to manually examine whether the claim filing is medically legitimate or not. However, the review process is, unquestionably, very costly and time-consuming. In order to address these limitations, data mining techniques have been employed to detect problematic claims or abusive providers showing an abnormal billing pattern. However, these cases only used coarsely grained information such as claim-level or provider-level data. This extracted information may lead to degradation of the model's performance. In this thesis, we proposed abuse detection methods using the medical treatment data, which is the lowest level information of the healthcare insurance claim. Firstly, we propose a scoring model based on which abusive providers are detected and show that the review process with the proposed model is more efficient than that with the previous model which uses the provider-level variables as input variables. At the same time, we devise the evaluation metrics to quantify the efficiency of the review process. Secondly, we propose the method of detecting overtreatment under seasonality, which reflects more reality to the model. We propose a model embodying multiple structures specific to DRG codes selected as important for each given department. We show that the proposed method is more robust to the seasonality than the previous method. Thirdly, we propose an overtreatment detection model accounting for heterogeneous treatment between practitioners. We proposed a network-based approach through which the relationship between the diseases and treatments is considered during the overtreatment detection process. Experimental results show that the proposed method classify the treatment well which does not explicitly exist in the training set. From these works, we show that using treatment data allows modeling abuse detection at various levels: treatment, claim, and provider-level.사람들의 기대수명이 증가함에 따라 삶의 질을 향상시키기 위해 보건의료에 소비하는 금액은 증가하고 있다. 그러나, 비싼 의료 서비스 비용은 필연적으로 개인과 가정에게 큰 재정적 부담을 주게된다. 이를 방지하기 위해, 많은 국가에서는 공공 의료 보험 시스템을 도입하여 사람들이 적절한 가격에 의료서비스를 받을 수 있도록 하고 있다. 일반적으로, 환자가 먼저 서비스를 받고 나서 일부만 지불하고 나면, 보험 회사가 사후에 해당 의료 기관에 잔여 금액을 상환을 하는 제도로 운영된다. 그러나 이러한 제도를 악용하여 환자의 질병을 조작하거나 과잉진료를 하는 등의 부당청구가 발생하기도 한다. 이러한 행위들은 의료 시스템에서 발생하는 주요 재정 손실의 이유 중 하나로, 이를 방지하기 위해, 보험회사에서는 의료 전문가를 고용하여 의학적 정당성여부를 일일히 검사한다. 그러나, 이러한 검토과정은 매우 비싸고 많은 시간이 소요된다. 이러한 검토과정을 효율적으로 하기 위해, 데이터마이닝 기법을 활용하여 문제가 있는 청구서나 청구 패턴이 비정상적인 의료 서비스 공급자를 탐지하는 연구가 있어왔다. 그러나, 이러한 연구들은 데이터로부터 청구서 단위나 공급자 단위의 변수를 유도하여 모델을 학습한 사례들로, 가장 낮은 단위의 데이터인 진료 내역 데이터를 활용하지 못했다. 이 논문에서는 청구서에서 가장 낮은 단위의 데이터인 진료 내역 데이터를 활용하여 부당청구를 탐지하는 방법론을 제안한다. 첫째, 비정상적인 청구 패턴을 갖는 의료 서비스 제공자를 탐지하는 방법론을 제안하였다. 이를 실제 데이터에 적용하였을 때, 기존의 공급자 단위의 변수를 사용한 방법보다 더 효율적인 심사가 이루어 짐을 확인하였다. 이 때, 효율성을 정량화하기 위한 평가 척도도 제안하였다. 둘째로, 청구서의 계절성이 존재하는 상황에서 과잉진료를 탐지하는 방법을 제안하였다. 이 때, 진료 과목단위로 모델을 운영하는 대신 질병군(DRG) 단위로 모델을 학습하고 평가하는 방법을 제안하였다. 그리고 실제 데이터에 적용하였을 때, 제안한 방법이 기존 방법보다 계절성에 더 강건함을 확인하였다. 셋째로, 동일 환자에 대해서 의사간의 상이한 진료 패턴을 갖는 환경에서의 과잉진료 탐지 방법을 제안하였다. 이는 환자의 질병과 진료내역간의 관계를 네트워크 기반으로 모델링하는것을 기반으로 한다. 실험 결과 제안한 방법이 학습 데이터에서 나타나지 않는 진료 패턴에 대해서도 잘 분류함을 알 수 있었다. 그리고 이러한 연구들로부터 진료 내역을 활용하였을 때, 진료내역, 청구서, 의료 서비스 제공자 등 다양한 레벨에서의 부당 청구를 탐지할 수 있음을 확인하였다.Chapter 1 Introduction 1 Chapter 2 Detection of Abusive Providers by department with Neural Network 9 2.1 Background 9 2.2 Literature Review 12 2.2.1 Abnormality Detection in Healthcare Insurance with Datamining Technique 12 2.2.2 Feed-Forward Neural Network 17 2.3 Proposed Method 21 2.3.1 Calculating the Likelihood of Abuse for each Treatment with Deep Neural Network 22 2.3.2 Calculating the Abuse Score of the Provider 25 2.4 Experiments 26 2.4.1 Data Description 27 2.4.2 Experimental Settings 32 2.4.3 Evaluation Measure (1): Relative Efficiency 33 2.4.4 Evaluation Measure (2): Precision at k 37 2.5 Results 38 2.5.1 Results in the test set 38 2.5.2 The Relationship among the Claimed Amount, the Abused Amount and the Abuse Score 40 2.5.3 The Relationship between the Performance of the Treatment Scoring Model and Review Efficiency 41 2.5.4 Treatment Scoring Model Results 42 2.5.5 Post-deployment Performance 44 2.6 Summary 45 Chapter 3 Detection of overtreatment by Diagnosis-related Group with Neural Network 48 3.1 Background 48 3.2 Literature review 51 3.2.1 Seasonality in disease 51 3.2.2 Diagnosis related group 52 3.3 Proposed method 54 3.3.1 Training a deep neural network model for treatment classi fication 55 3.3.2 Comparing the Performance of DRG-based Model against the department-based Model 57 3.4 Experiments 60 3.4.1 Data Description and Preprocessing 60 3.4.2 Performance Measures 64 3.4.3 Experimental Settings 65 3.5 Results 65 3.5.1 Overtreatment Detection 65 3.5.2 Abnormal Claim Detection 67 3.6 Summary 68 Chapter 4 Detection of overtreatment with graph embedding of disease-treatment pair 70 4.1 Background 70 4.2 Literature review 72 4.2.1 Graph embedding methods 73 4.2.2 Application of graph embedding methods to biomedical data analysis 79 4.2.3 Medical concept embedding methods 87 4.3 Proposed method 88 4.3.1 Network construction 89 4.3.2 Link Prediction between the Disease and the Treatment 90 4.3.3 Overtreatment Detection 93 4.4 Experiments 96 4.4.1 Data Description 97 4.4.2 Experimental Settings 99 4.5 Results 102 4.5.1 Network Construction 102 4.5.2 Link Prediction between the Disease and the Treatment 104 4.5.3 Overtreatment Detection 105 4.6 Summary 106 Chapter 5 Conclusion 108 5.1 Contribution 108 5.2 Future Work 110 Bibliography 112 국문초록 129Docto

TITAN: A Spatiotemporal Feature Learning Framework for Traffic Incident Duration Prediction

Critical incident stages identification and reasonable prediction of traffic incident duration are essential in traffic incident management. In this paper, we propose a traffic incident duration prediction model that simultaneously predicts the impact of the traffic incidents and identifies the critical groups of temporal features via a multi-task learning framework. First, we formulate a sparsity optimization problem that extracts low-level temporal features based on traffic speed readings and then generalizes higher level features as phases of traffic incidents. Second, we propose novel constraints on feature similarity exploiting prior knowledge about the spatial connectivity of the road network to predict the incident duration. The proposed problem is challenging to solve due to the orthogonality constraints, non-convexity objective, and non-smoothness penalties. We develop an algorithm based on the alternating direction method of multipliers (ADMM) framework to solve the proposed formulation. Extensive experiments and comparisons to other models on real-world traffic data and traffic incident records justify the efficacy of our model

Latent Representation and Sampling in Network: Application in Text Mining and Biology.

In classical machine learning, hand-designed features are used for learning a mapping from raw data. However, human involvement in feature design makes the process expensive. Representation learning aims to learn abstract features directly from data without direct human involvement. Raw data can be of various forms. Network is one form of data that encodes relational structure in many real-world domains. Therefore, learning abstract features for network units is an important task. In this dissertation, we propose models for incorporating temporal information given as a collection of networks from subsequent time-stamps. The primary objective of our models is to learn a better abstract feature representation of nodes and edges in an evolving network. We show that the temporal information in the abstract feature improves the performance of link prediction task substantially. Besides applying to the network data, we also employ our models to incorporate extra-sentential information in the text domain for learning better representation of sentences. We build a context network of sentences to capture extra-sentential information. This information in abstract feature representation of sentences improves various text-mining tasks substantially over a set of baseline methods. A problem with the abstract features that we learn is that they lack interpretability. In real-life applications on network data, for some tasks, it is crucial to learn interpretable features in the form of graphical structures. For this we need to mine important graphical structures along with their frequency statistics from the input dataset. However, exact algorithms for these tasks are computationally expensive, so scalable algorithms are of urgent need. To overcome this challenge, we provide efficient sampling algorithms for mining higher-order structures from network(s). We show that our sampling-based algorithms are scalable. They are also superior to a set of baseline algorithms in terms of retrieving important graphical sub-structures, and collecting their frequency statistics. Finally, we show that we can use these frequent subgraph statistics and structures as features in various real-life applications. We show one application in biology and another in security. In both cases, we show that the structures and their statistics significantly improve the performance of knowledge discovery tasks in these domains