263 research outputs found
An Attention-based Collaboration Framework for Multi-View Network Representation Learning
Learning distributed node representations in networks has been attracting
increasing attention recently due to its effectiveness in a variety of
applications. Existing approaches usually study networks with a single type of
proximity between nodes, which defines a single view of a network. However, in
reality there usually exists multiple types of proximities between nodes,
yielding networks with multiple views. This paper studies learning node
representations for networks with multiple views, which aims to infer robust
node representations across different views. We propose a multi-view
representation learning approach, which promotes the collaboration of different
views and lets them vote for the robust representations. During the voting
process, an attention mechanism is introduced, which enables each node to focus
on the most informative views. Experimental results on real-world networks show
that the proposed approach outperforms existing state-of-the-art approaches for
network representation learning with a single view and other competitive
approaches with multiple views.Comment: CIKM 201
λμ’ , μ΄μ’ , κ·Έλ¦¬κ³ λ무 ννμ κ·Έλνλ₯Ό μν λΉμ§λ νν νμ΅
νμλ
Όλ¬Έ(λ°μ¬) -- μμΈλνκ΅λνμ : 곡과λν μ κΈ°Β·μ 보곡νλΆ, 2022. 8. μ΅μ§μ.κ·Έλν λ°μ΄ν°μ λν λΉμ§λ νν νμ΅μ λͺ©μ μ κ·Έλνμ ꡬ쑰μ λ
Έλμ μμ±μ μ λ°μνλ μ μ©ν λ
Έλ λ¨μ νΉμ κ·Έλν λ¨μμ λ²‘ν° νν ννμ νμ΅νλ κ²μ΄λ€. μ΅κ·Ό, κ·Έλν λ°μ΄ν°μ λν΄ κ°λ ₯ν νν νμ΅ λ₯λ ₯μ κ°μΆ κ·Έλν μ κ²½λ§μ νμ©ν λΉμ§λ κ·Έλν νν νμ΅ λͺ¨λΈμ μ€κ³κ° μ£Όλͺ©μ λ°κ³ μλ€. λ§μ λ°©λ²λ€μ ν μ’
λ₯μ μ£μ§μ ν μ’
λ₯μ λ
Έλκ° μ‘΄μ¬νλ λμ’
κ·Έλνμ λν νμ΅μ μ§μ€μ νλ€. νμ§λ§ μ΄ μΈμμ μλ§μ μ’
λ₯μ κ΄κ³κ° μ‘΄μ¬νκΈ° λλ¬Έμ, κ·Έλν λν ꡬ쑰μ , μλ―Έλ‘ μ μμ±μ ν΅ν΄ λ€μν μ’
λ₯λ‘ λΆλ₯ν μ μλ€. κ·Έλμ, κ·Έλνλ‘λΆν° μ μ©ν ννμ νμ΅νκΈ° μν΄μλ λΉμ§λ νμ΅ νλ μμν¬λ μ
λ ₯ κ·Έλνμ νΉμ§μ μ λλ‘ κ³ λ €ν΄μΌλ§ νλ€. λ³Έ νμλ
Όλ¬Έμμ μ°λ¦¬λ λ리 μ ν μ μλ μΈκ°μ§ κ·Έλν κ΅¬μ‘°μΈ λμ’
κ·Έλν, νΈλ¦¬ ννμ κ·Έλν, κ·Έλ¦¬κ³ μ΄μ’
κ·Έλνμ λν κ·Έλν μ κ²½λ§μ νμ©νλ λΉμ§λ νμ΅ λͺ¨λΈλ€μ μ μνλ€.
μ²μμΌλ‘, μ°λ¦¬λ λμ’
κ·Έλνμ λ
Έλμ λνμ¬ μ μ°¨μ ννμ νμ΅νλ κ·Έλν 컨볼루μ
μ€ν μΈμ½λ λͺ¨λΈμ μ μνλ€. κΈ°μ‘΄μ κ·Έλν μ€ν μΈμ½λλ ꡬ쑰μ μ μ²΄κ° νμ΅μ΄ λΆκ°λ₯ν΄μ μ νμ μΈ νν νμ΅ λ₯λ ₯μ κ°μ§ μ μλ λ°λ©΄μ, μ μνλ μ€ν μΈμ½λλ λ
Έλμ νΌμ³λ₯Ό 볡μνλ©°,ꡬ쑰μ μ μ²΄κ° νμ΅μ΄ κ°λ₯νλ€. λ
Έλμ νΌμ³λ₯Ό 볡μνκΈ° μν΄μ, μ°λ¦¬λ μΈμ½λ λΆλΆμ μν μ΄ μ΄μν λ
ΈλλΌλ¦¬ μ μ¬ν ννμ κ°μ§κ² νλ λΌνλΌμμ μ€λ¬΄λ©μ΄λΌλ κ²μ μ£Όλͺ©νμ¬ λμ½λ λΆλΆμμλ μ΄μ λ
Έλμ ννκ³Ό λ©μ΄μ§κ² νλ λΌνλΌμμ μ€νλμ νλλ‘ μ€κ³νμλ€. λν λΌνλΌμμ μ€νλμ κ·Έλλ‘ μ μ©νλ©΄ λΆμμ μ±μ μ λ°ν μ μκΈ° λλ¬Έμ, μ£μ§μ κ°μ€μΉ κ°μ μμ κ°μ μ€ μ μλ λΆνΈν κ·Έλνλ₯Ό νμ©νμ¬ μμ μ μΈ λΌνλΌμμ μ€νλμ ννλ₯Ό μ μνμλ€. λμ’
κ·Έλνμ λν λ
Έλ ν΄λ¬μ€ν°λ§κ³Ό λ§ν¬ μμΈ‘ μ€νμ ν΅νμ¬ μ μνλ λ°©λ²μ΄ μμ μ μΌλ‘ μ°μν μ±λ₯μ 보μμ νμΈνμλ€.
λμ§Έλ‘, μ°λ¦¬λ νΈλ¦¬μ ννλ₯Ό κ°μ§λ κ³μΈ΅μ μΈ κ΄κ³λ₯Ό κ°μ§κ³ μλ κ·Έλνμ λ
Έλ ννμ μ ννκ² νμ΅νκΈ° μνμ¬ μ곑μ 곡κ°μμ λμνλ μ€ν μΈμ½λ λͺ¨λΈμ μ μνλ€. μ ν΄λ¦¬λμΈ κ³΅κ°μ νΈλ¦¬λ₯Ό μ¬μνκΈ°μ λΆμ μ νλ€λ μ΅κ·Όμ λΆμμ ν΅νμ¬, μ곑μ 곡κ°μμ κ·Έλν μ κ²½λ§μ λ μ΄μ΄λ₯Ό νμ©νμ¬ λ
Έλμ μ μ°¨μ ννμ νμ΅νκ² λλ€. μ΄ λ, κ·Έλν μ κ²½λ§μ΄ μ곑μ κΈ°ννμμ κ³μΈ΅ μ 보λ₯Ό λ΄κ³ μλ 거리μ κ°μ νμ©νμ¬ λ
Έλμ μ΄μμ¬μ΄μ μ€μλλ₯Ό νμ©νλλ‘ μ€κ³νμλ€. μ°λ¦¬λ λ
Όλ¬Έ μΈμ© κ΄κ³ λ€νΈμν¬, κ³ν΅λ, μ΄λ―Έμ§ μ¬μ΄μ λ€νΈμν¬λ±μ λν΄ μ μν λͺ¨λΈμ μ μ©νμ¬ λ
Έλ ν΄λ¬μ€ν°λ§κ³Ό λ§ν¬ μμΈ‘ μ€νμ νμμΌλ©°, νΈλ¦¬μ ννλ₯Ό κ°μ§λ κ·Έλνμ λν΄μ μ μν λͺ¨λΈμ΄ μ ν΄λ¦¬λμΈ κ³΅κ°μμ μννλ λͺ¨λΈμ λΉν΄ ν₯μλ μ±λ₯μ 보μλ€λ κ²μ νμΈνμλ€.
λ§μ§λ§μΌλ‘, μ°λ¦¬λ μ¬λ¬ μ’
λ₯μ λ
Έλμ μ£μ§λ₯Ό κ°μ§λ μ΄μ’
κ·Έλνμ λν λμ‘° νμ΅ λͺ¨λΈμ μ μνλ€. μ°λ¦¬λ κΈ°μ‘΄μ λ°©λ²λ€μ΄ νμ΅νκΈ° μ΄μ μ μΆ©λΆν λλ©μΈ μ§μμ μ¬μ©νμ¬ μ€κ³ν λ©νν¨μ€λ λ©νκ·Έλνμ μμ‘΄νλ€λ λ¨μ κ³Ό λ§μ μ΄μ’
κ·Έλνμ μ£μ§κ° λ€λ₯Έ λ
Έλ μ’
λ₯μ¬μ΄μ κ΄κ³μ μ§μ€νκ³ μλ€λ μ μ μ£Όλͺ©νμλ€. μ΄λ₯Ό ν΅ν΄ μ°λ¦¬λ μ¬μ κ³Όμ μ΄ νμμμΌλ©° λ€λ₯Έ μ’
λ₯ μ¬μ΄μ κ΄κ³μ λνμ¬ κ°μ μ’
λ₯ μ¬μ΄μ κ΄κ³λ λμμ ν¨μ¨μ μΌλ‘ νμ΅νκ² νλ λ©νλ
ΈλλΌλ κ°λ
μ μ μνμλ€. λν λ©νλ
Έλλ₯Ό κΈ°λ°μΌλ‘νλ κ·Έλν μ κ²½λ§κ³Ό λμ‘° νμ΅ λͺ¨λΈμ μ μνμλ€. μ°λ¦¬λ μ μν λͺ¨λΈμ λ©νν¨μ€λ₯Ό μ¬μ©νλ μ΄μ’
κ·Έλν νμ΅ λͺ¨λΈκ³Ό λ
Έλ ν΄λ¬μ€ν°λ§ λ±μ μ€ν μ±λ₯μΌλ‘ λΉκ΅ν΄λ³΄μμ λ, λΉλ±νκ±°λ λμ μ±λ₯μ 보μμμ νμΈνμλ€.The goal of unsupervised graph representation learning is extracting useful node-wise or graph-wise vector representation that is aware of the intrinsic structures of the graph and its attributes. These days, designing methodology of unsupervised graph representation learning based on graph neural networks has growing attention due to their powerful representation ability. Many methods are focused on a homogeneous graph that is a network with a single type of node and a single type of edge. However, as many types of relationships exist in this world, graphs can also be classified into various types by structural and semantic properties. For this reason, to learn useful representations from graphs, the unsupervised learning framework must consider the characteristics of the input graph. In this dissertation, we focus on designing unsupervised learning models using graph neural networks for three graph structures that are widely available: homogeneous graphs, tree-like graphs, and heterogeneous graphs.
First, we propose a symmetric graph convolutional autoencoder which produces a low-dimensional latent representation from a homogeneous graph. In contrast to the existing graph autoencoders with asymmetric decoder parts, the proposed autoencoder has a newly designed decoder which builds a completely symmetric autoencoder form. For the reconstruction of node features, the decoder is designed based on Laplacian sharpening as the counterpart of Laplacian smoothing of the encoder, which allows utilizing the graph structure in the whole processes of the proposed autoencoder architecture. In order to prevent the numerical instability of the network caused by the Laplacian sharpening introduction, we further propose a new numerically stable form of the Laplacian sharpening by incorporating the signed graphs. The experimental results of clustering, link prediction and visualization tasks on homogeneous graphs strongly support that the proposed model is stable and outperforms various state-of-the-art algorithms.
Second, we analyze how unsupervised tasks can benefit from learned representations in hyperbolic space. To explore how well the hierarchical structure of unlabeled data can be represented in hyperbolic spaces, we design a novel hyperbolic message passing autoencoder whose overall auto-encoding is performed in hyperbolic space. The proposed model conducts auto-encoding the networks via fully utilizing hyperbolic geometry in message passing. Through extensive quantitative and qualitative analyses, we validate the properties and benefits of the unsupervised hyperbolic representations of tree-like graphs.
Third, we propose the novel concept of metanode for message passing to learn both heterogeneous and homogeneous relationships between any two nodes without meta-paths and meta-graphs. Unlike conventional methods, metanodes do not require a predetermined step to manipulate the given relations between different types to enrich relational information. Going one step further, we propose a metanode-based message passing layer and a contrastive learning model using the proposed layer. In our experiments, we show the competitive performance of the proposed metanode-based message passing method on node clustering and node classification tasks, when compared to state-of-the-art methods for message passing networks for heterogeneous graphs.1 Introduction 1
2 Representation Learning on Graph-Structured Data 4
2.1 Basic Introduction 4
2.1.1 Notations 5
2.2 Traditional Approaches 5
2.2.1 Graph Statistic 5
2.2.2 Neighborhood Overlap 7
2.2.3 Graph Kernel 9
2.2.4 Spectral Approaches 10
2.3 Node Embeddings I: Factorization and Random Walks 15
2.3.1 Factorization-based Methods 15
2.3.2 Random Walk-based Methods 16
2.4 Node Embeddings II: Graph Neural Networks 17
2.4.1 Overview of Framework 17
2.4.2 Representative Models 18
2.5 Learning in Unsupervised Environments 21
2.5.1 Predictive Coding 21
2.5.2 Contrastive Coding 22
2.6 Applications 24
2.6.1 Classifications 24
2.6.2 Link Prediction 26
3 Autoencoder Architecture for Homogeneous Graphs 27
3.1 Overview 27
3.2 Preliminaries 30
3.2.1 Spectral Convolution on Graphs 30
3.2.2 Laplacian Smoothing 32
3.3 Methodology 33
3.3.1 Laplacian Sharpening 33
3.3.2 Numerically Stable Laplacian Sharpening 34
3.3.3 Subspace Clustering Cost for Image Clustering 37
3.3.4 Training 39
3.4 Experiments 40
3.4.1 Datasets 40
3.4.2 Experimental Settings 42
3.4.3 Comparing Methods 42
3.4.4 Node Clustering 43
3.4.5 Image Clustering 45
3.4.6 Ablation Studies 46
3.4.7 Link Prediction 47
3.4.8 Visualization 47
3.5 Summary 49
4 Autoencoder Architecture for Tree-like Graphs 50
4.1 Overview 50
4.2 Preliminaries 52
4.2.1 Hyperbolic Embeddings 52
4.2.2 Hyperbolic Geometry 53
4.3 Methodology 55
4.3.1 Geometry-Aware Message Passing 56
4.3.2 Nonlinear Activation 57
4.3.3 Loss Function 58
4.4 Experiments 58
4.4.1 Datasets 59
4.4.2 Compared Methods 61
4.4.3 Experimental Details 62
4.4.4 Node Clustering and Link Prediction 64
4.4.5 Image Clustering 66
4.4.6 Structure-Aware Unsupervised Embeddings 68
4.4.7 Hyperbolic Distance to Filter Training Samples 71
4.4.8 Ablation Studies 74
4.5 Further Discussions 75
4.5.1 Connection to Contrastive Learning 75
4.5.2 Failure Cases of Hyperbolic Embedding Spaces 75
4.6 Summary 77
5 Contrastive Learning for Heterogeneous Graphs 78
5.1 Overview 78
5.2 Preliminaries 82
5.2.1 Meta-path 82
5.2.2 Representation Learning on Heterogeneous Graphs 82
5.2.3 Contrastive methods for Heterogeneous Graphs 83
5.3 Methodology 84
5.3.1 Definitions 84
5.3.2 Metanode-based Message Passing Layer 86
5.3.3 Contrastive Learning Framework 88
5.4 Experiments 89
5.4.1 Experimental Details 90
5.4.2 Node Classification 94
5.4.3 Node Clustering 96
5.4.4 Visualization 96
5.4.5 Effectiveness of Metanodes 97
5.5 Summary 99
6 Conclusions 101λ°
No Pattern, No Recognition: a Survey about Reproducibility and Distortion Issues of Text Clustering and Topic Modeling
Extracting knowledge from unlabeled texts using machine learning algorithms
can be complex. Document categorization and information retrieval are two
applications that may benefit from unsupervised learning (e.g., text clustering
and topic modeling), including exploratory data analysis. However, the
unsupervised learning paradigm poses reproducibility issues. The initialization
can lead to variability depending on the machine learning algorithm.
Furthermore, the distortions can be misleading when regarding cluster geometry.
Amongst the causes, the presence of outliers and anomalies can be a determining
factor. Despite the relevance of initialization and outlier issues for text
clustering and topic modeling, the authors did not find an in-depth analysis of
them. This survey provides a systematic literature review (2011-2022) of these
subareas and proposes a common terminology since similar procedures have
different terms. The authors describe research opportunities, trends, and open
issues. The appendices summarize the theoretical background of the text
vectorization, the factorization, and the clustering algorithms that are
directly or indirectly related to the reviewed works
Diagnosis of Autism Spectrum Disorder Based on Brain Network Clustering
Developments in magnetic resonance imaging (MRI) provide new non-invasive approachβfunctional MRI (fMRI)βto study functions of brain. With the help of fMRI, I can build functional brain networks (FBN) to model correlations of brain activities between cortical regions. Studies focused on brain diseases, including autism spectrum disorder (ASD), have been conducted based on analyzing alterations in FBNs of patients. New biomarkers are identified, and new theories and assumptions are proposed on pathology of brain diseases.
Considering that traditional clinical ASD diagnosis instruments, which greatly rely on interviews and observations, can yield large variance, recent studies start to combine machine learning methods and FBN to perform auto-classification of ASD. Such studies have achieved relatively good accuracy. However, in most of these studies, features they use are extracted from the whole brain networks thus the dimension of the features can be high. High-dimensional features may yield overfitting issues and increase computational complexity. Therefore, I need a feature selection strategy that effectively reduces feature dimensions while keeping a good classification performance.
In this study, I present a new feature selection strategy that extracting features from functional modules but not the whole brain networks. I will show that my strategy not only reduces feature dimensions, but also improve performances of auto-classifications of ASD. The whole study can be separated into 4 stages: building FBNs, identification of functional modules, statistical analysis of modular alterations and, finally, training classifiers with modular features for auto-classification of ASD. I firstly demonstrate the whole procedure to build FBNs from fMRI images. To identify functional module, I propose a new network clustering algorithm based on joint non-negative matrix factorization. Different from traditional brain network clustering algorithms that mostly perform on an average network of all subjects, I design my algorithm to factorize multiple brain networks simultaneously because the clustering results should be valid not only on the average network but also on each individual network. I show the modules I find are more valid in both views. Then I statistically analyze the alterations in functional modules between ASD and typically developed (TD) group to determine from which modules I extract features from. Several indices based on graph theory are calculated to measure modular properties. I find significant alterations in two modules. With features from these two modules, I train several widely-used classifiers and validate the classifiers on a real-world dataset. The performances of classifiers trained by modular features are better than those with whole-brain features, which demonstrates the effectiveness of my feature selection strategy
Innovative Algorithms and Evaluation Methods for Biological Motif Finding
Biological motifs are defined as overly recurring sub-patterns in biological systems. Sequence motifs and network motifs are the examples of biological motifs. Due to the wide range of applications, many algorithms and computational tools have been developed for efficient search for biological motifs. Therefore, there are more computationally derived motifs than experimentally validated motifs, and how to validate the biological significance of the βcandidate motifsβ becomes an important question. Some of sequence motifs are verified by their structural similarities or their functional roles in DNA or protein sequences, and stored in databases. However, biological role of
network motifs is still invalidated and currently no databases exist for this purpose.
In this thesis, we focus not only on the computational efficiency but also on the biological meanings of the motifs. We provide an efficient way to incorporate biological information with clustering analysis methods: For example, a sparse nonnegative matrix factorization (SNMF) method is used with Chou-Fasman parameters for the protein motif finding. Biological network motifs are searched by various clustering algorithms with Gene ontology (GO) information. Experimental results show that the algorithms perform better than existing algorithms by producing a larger number of high-quality of biological motifs.
In addition, we apply biological network motifs for the discovery of essential proteins. Essential proteins are defined as a minimum set of proteins which are vital for development to a fertile adult and in a cellular life in an organism. We design a new centrality algorithm with biological network motifs, named MCGO, and score proteins in a protein-protein interaction (PPI) network to find essential proteins. MCGO is also combined with other centrality measures to predict essential proteins using machine learning techniques.
We have three contributions to the study of biological motifs through this thesis; 1) Clustering analysis is efficiently used in this work and biological information is easily integrated with the analysis; 2) We focus more on the biological meanings of motifs by adding biological knowledge in the algorithms and by suggesting biologically related evaluation methods. 3) Biological network motifs are successfully applied to a practical application of prediction of essential proteins
Multilayer Networks
In most natural and engineered systems, a set of entities interact with each
other in complicated patterns that can encompass multiple types of
relationships, change in time, and include other types of complications. Such
systems include multiple subsystems and layers of connectivity, and it is
important to take such "multilayer" features into account to try to improve our
understanding of complex systems. Consequently, it is necessary to generalize
"traditional" network theory by developing (and validating) a framework and
associated tools to study multilayer systems in a comprehensive fashion. The
origins of such efforts date back several decades and arose in multiple
disciplines, and now the study of multilayer networks has become one of the
most important directions in network science. In this paper, we discuss the
history of multilayer networks (and related concepts) and review the exploding
body of work on such networks. To unify the disparate terminology in the large
body of recent work, we discuss a general framework for multilayer networks,
construct a dictionary of terminology to relate the numerous existing concepts
to each other, and provide a thorough discussion that compares, contrasts, and
translates between related notions such as multilayer networks, multiplex
networks, interdependent networks, networks of networks, and many others. We
also survey and discuss existing data sets that can be represented as
multilayer networks. We review attempts to generalize single-layer-network
diagnostics to multilayer networks. We also discuss the rapidly expanding
research on multilayer-network models and notions like community structure,
connected components, tensor decompositions, and various types of dynamical
processes on multilayer networks. We conclude with a summary and an outlook.Comment: Working paper; 59 pages, 8 figure
- β¦