1,197 research outputs found
SNE: Signed Network Embedding
Several network embedding models have been developed for unsigned networks.
However, these models based on skip-gram cannot be applied to signed networks
because they can only deal with one type of link. In this paper, we present our
signed network embedding model called SNE. Our SNE adopts the log-bilinear
model, uses node representations of all nodes along a given path, and further
incorporates two signed-type vectors to capture the positive or negative
relationship of each edge along the path. We conduct two experiments, node
classification and link prediction, on both directed and undirected signed
networks and compare with four baselines including a matrix factorization
method and three state-of-the-art unsigned network embedding models. The
experimental results demonstrate the effectiveness of our signed network
embedding.Comment: To appear in PAKDD 201
COIN:Contrastive Identifier Network for Breast Mass Diagnosis in Mammography
Computer-aided breast cancer diagnosis in mammography is a challenging
problem, stemming from mammographical data scarcity and data entanglement. In
particular, data scarcity is attributed to the privacy and expensive
annotation. And data entanglement is due to the high similarity between benign
and malignant masses, of which manifolds reside in lower dimensional space with
very small margin. To address these two challenges, we propose a deep learning
framework, named Contrastive Identifier Network (\textsc{COIN}), which
integrates adversarial augmentation and manifold-based contrastive learning.
Firstly, we employ adversarial learning to create both on- and off-distribution
mass contained ROIs. After that, we propose a novel contrastive loss with a
built Signed graph. Finally, the neural network is optimized in a contrastive
learning manner, with the purpose of improving the deep model's
discriminativity on the extended dataset. In particular, by employing COIN,
data samples from the same category are pulled close whereas those with
different labels are pushed further in the deep latent space. Moreover, COIN
outperforms the state-of-the-art related algorithms for solving breast cancer
diagnosis problem by a considerable margin, achieving 93.4\% accuracy and
95.0\% AUC score. The code will release on ***
IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models
This paper provides a unified account of two schools of thinking in
information retrieval modelling: the generative retrieval focusing on
predicting relevant documents given a query, and the discriminative retrieval
focusing on predicting relevancy given a query-document pair. We propose a game
theoretical minimax game to iteratively optimise both models. On one hand, the
discriminative model, aiming to mine signals from labelled and unlabelled data,
provides guidance to train the generative model towards fitting the underlying
relevance distribution over documents given the query. On the other hand, the
generative model, acting as an attacker to the current discriminative model,
generates difficult examples for the discriminative model in an adversarial way
by minimising its discrimination objective. With the competition between these
two models, we show that the unified framework takes advantage of both schools
of thinking: (i) the generative model learns to fit the relevance distribution
over documents via the signals from the discriminative model, and (ii) the
discriminative model is able to exploit the unlabelled data selected by the
generative model to achieve a better estimation for document ranking. Our
experimental results have demonstrated significant performance gains as much as
23.96% on Precision@5 and 15.50% on MAP over strong baselines in a variety of
applications including web search, item recommendation, and question answering.Comment: 12 pages; appendix adde
Discovering structure without labels
The scarcity of labels combined with an abundance of data makes unsupervised learning more attractive than ever. Without annotations, inductive biases must guide the identification of the most salient structure in the data. This thesis contributes to two aspects of unsupervised learning: clustering and dimensionality reduction.
The thesis falls into two parts. In the first part, we introduce Mod Shift, a clustering method for point data that uses a distance-based notion of attraction and repulsion to determine the number of clusters and the assignment of points to clusters. It iteratively moves points towards crisp clusters like Mean Shift but also has close ties to the Multicut problem via its loss function. As a result, it connects signed graph partitioning to clustering in Euclidean space.
The second part treats dimensionality reduction and, in particular, the prominent neighbor embedding methods UMAP and t-SNE. We analyze the details of UMAP's implementation and find its actual loss function. It differs drastically from the one usually stated. This discrepancy allows us to explain some typical artifacts in UMAP plots, such as the dataset size-dependent tendency to produce overly crisp substructures. Contrary to existing belief, we find that UMAP's high-dimensional similarities are not critical to its success.
Based on UMAP's actual loss, we describe its precise connection to the other state-of-the-art visualization method, t-SNE. The key insight is a new, exact relation between the contrastive loss functions negative sampling, employed by UMAP, and noise-contrastive estimation, which has been used to approximate t-SNE. As a result, we explain that UMAP embeddings appear more compact than t-SNE plots due to increased attraction between neighbors. Varying the attraction strength further, we obtain a spectrum of neighbor embedding methods, encompassing both UMAP- and t-SNE-like versions as special cases. Moving from more attraction to more repulsion shifts the focus of the embedding from continuous, global to more discrete and local structure of the data. Finally, we emphasize the link between contrastive neighbor embeddings and self-supervised contrastive learning. We show that different flavors of contrastive losses can work for both of them with few noise samples
Recommended from our members
End-to-End Quantum-like Language Models with Application to Question Answering
Language Modeling (LM) is a fundamental research topic ina range of areas. Recently, inspired by quantum theory, a novel Quantum Language Model (QLM) has been proposed for Information Retrieval (IR). In this paper, we aim to broaden the theoretical and practical basis of QLM. We develop a Neural Network based Quantum-like Language Model (NNQLM) and apply it to Question Answering. Specifically, based on word embeddings, we design a new density matrix, which represents a sentence (e.g., a question or an answer) and encodes a mixture of semantic subspaces. Such a density matrix, together with a joint representation of the question and the answer, can be integrated into neural network architectures (e.g., 2-dimensional convolutional neural networks). Experiments on the TREC-QA and WIKIQA datasets have verified the effectiveness of our proposed models
Learning Robust Node Representations on Graphs
Graph neural networks (GNN), as a popular methodology for node representation
learning on graphs, currently mainly focus on preserving the smoothness and
identifiability of node representations. A robust node representation on graphs
should further hold the stability property which means a node representation is
resistant to slight perturbations on the input. In this paper, we introduce the
stability of node representations in addition to the smoothness and
identifiability, and develop a novel method called contrastive graph neural
networks (CGNN) that learns robust node representations in an unsupervised
manner. Specifically, CGNN maintains the stability and identifiability by a
contrastive learning objective, while preserving the smoothness with existing
GNN models. Furthermore, the proposed method is a generic framework that can be
equipped with many other backbone models (e.g. GCN, GraphSage and GAT).
Extensive experiments on four benchmarks under both transductive and inductive
learning setups demonstrate the effectiveness of our method in comparison with
recent supervised and unsupervised models.Comment: 16 page
- …