10,490 research outputs found
MCNE: An End-to-End Framework for Learning Multiple Conditional Network Representations of Social Network
Recently, the Network Representation Learning (NRL) techniques, which
represent graph structure via low-dimension vectors to support social-oriented
application, have attracted wide attention. Though large efforts have been
made, they may fail to describe the multiple aspects of similarity between
social users, as only a single vector for one unique aspect has been
represented for each node. To that end, in this paper, we propose a novel
end-to-end framework named MCNE to learn multiple conditional network
representations, so that various preferences for multiple behaviors could be
fully captured. Specifically, we first design a binary mask layer to divide the
single vector as conditional embeddings for multiple behaviors. Then, we
introduce the attention network to model interaction relationship among
multiple preferences, and further utilize the adapted message sending and
receiving operation of graph neural network, so that multi-aspect preference
information from high-order neighbors will be captured. Finally, we utilize
Bayesian Personalized Ranking loss function to learn the preference similarity
on each behavior, and jointly learn multiple conditional node embeddings via
multi-task learning framework. Extensive experiments on public datasets
validate that our MCNE framework could significantly outperform several
state-of-the-art baselines, and further support the visualization and transfer
learning tasks with excellent interpretability and robustness.Comment: Accepted by KDD 2019 Research Track. In Proceedings of the 25th ACM
SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'19
CSNE: Conditional Signed Network Embedding
Signed networks are mathematical structures that encode positive and negative
relations between entities such as friend/foe or trust/distrust. Recently,
several papers studied the construction of useful low-dimensional
representations (embeddings) of these networks for the prediction of missing
relations or signs. Existing embedding methods for sign prediction generally
enforce different notions of status or balance theories in their optimization
function. These theories, however, are often inaccurate or incomplete, which
negatively impacts method performance.
In this context, we introduce conditional signed network embedding (CSNE).
Our probabilistic approach models structural information about the signs in the
network separately from fine-grained detail. Structural information is
represented in the form of a prior, while the embedding itself is used for
capturing fine-grained information. These components are then integrated in a
rigorous manner. CSNE's accuracy depends on the existence of sufficiently
powerful structural priors for modelling signed networks, currently unavailable
in the literature. Thus, as a second main contribution, which we find to be
highly valuable in its own right, we also introduce a novel approach to
construct priors based on the Maximum Entropy (MaxEnt) principle. These priors
can model the \emph{polarity} of nodes (degree to which their links are
positive) as well as signed \emph{triangle counts} (a measure of the degree
structural balance holds to in a network).
Experiments on a variety of real-world networks confirm that CSNE outperforms
the state-of-the-art on the task of sign prediction. Moreover, the MaxEnt
priors on their own, while less accurate than full CSNE, achieve accuracies
competitive with the state-of-the-art at very limited computational cost, thus
providing an excellent runtime-accuracy trade-off in resource-constrained
situations
CSNE : Conditional Signed Network Embedding
Signed networks are mathematical structures that encode positive and negative relations between entities such as friend/foe or trust/distrust. Recently, several papers studied the construction of useful low-dimensional representations (embeddings) of these networks for the prediction of missing relations or signs. Existing embedding methods for sign prediction generally enforce different notions of status or balance theories in their optimization function. These theories, however, are often inaccurate or incomplete, which negatively impacts method performance.
In this context, we introduce conditional signed network embedding (CSNE). Our probabilistic approach models structural information about the signs in the network separately from fine-grained detail. Structural information is represented in the form of a prior, while the embedding itself is used for capturing fine-grained information. These components are then integrated in a rigorous manner. CSNE's accuracy depends on the existence of sufficiently powerful structural priors for modelling signed networks, currently unavailable in the literature. Thus, as a second main contribution, which we find to be highly valuable in its own right, we also introduce a novel approach to construct priors based on the Maximum Entropy (MaxEnt) principle. These priors can model the polarity of nodes (degree to which their links are positive) as well as signed triangle counts (a measure of the degree structural balance holds to in a network).
Experiments on a variety of real-world networks confirm that CSNE outperforms the state-of-the-art on the task of sign prediction. Moreover, the MaxEnt priors on their own, while less accurate than full CSNE, achieve accuracies competitive with the state-of-the-art at very limited computational cost, thus providing an excellent runtime-accuracy trade-off in resource-constrained situations
Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings
We address jointly two important tasks for Question Answering in community
forums: given a new question, (i) find related existing questions, and (ii)
find relevant answers to this new question. We further use an auxiliary task to
complement the previous two, i.e., (iii) find good answers with respect to the
thread question in a question-comment thread. We use deep neural networks
(DNNs) to learn meaningful task-specific embeddings, which we then incorporate
into a conditional random field (CRF) model for the multitask setting,
performing joint learning over a complex graph structure. While DNNs alone
achieve competitive results when trained to produce the embeddings, the CRF,
which makes use of the embeddings and the dependencies between the tasks,
improves the results significantly and consistently across a variety of
evaluation metrics, thus showing the complementarity of DNNs and structured
learning.Comment: community question answering, task-specific embeddings, multi-task
learning, EMNLP-201
Using Embeddings to Correct for Unobserved Confounding in Networks
We consider causal inference in the presence of unobserved confounding. We
study the case where a proxy is available for the unobserved confounding in the
form of a network connecting the units. For example, the link structure of a
social network carries information about its members. We show how to
effectively use the proxy to do causal inference. The main idea is to reduce
the causal estimation problem to a semi-supervised prediction of both the
treatments and outcomes. Networks admit high-quality embedding models that can
be used for this semi-supervised prediction. We show that the method yields
valid inferences under suitable (weak) conditions on the quality of the
predictive model. We validate the method with experiments on a semi-synthetic
social network dataset. Code is available at
github.com/vveitch/causal-network-embeddings.Comment: An earlier version also addressed the use of text embeddings. That
material has been expanded and moved to arxiv:1905.12741, "Using Text
Embeddings for Causal Inference
Agent Embeddings: A Latent Representation for Pole-Balancing Networks
We show that it is possible to reduce a high-dimensional object like a neural
network agent into a low-dimensional vector representation with semantic
meaning that we call agent embeddings, akin to word or face embeddings. This
can be done by collecting examples of existing networks, vectorizing their
weights, and then learning a generative model over the weight space in a
supervised fashion. We investigate a pole-balancing task, Cart-Pole, as a case
study and show that multiple new pole-balancing networks can be generated from
their agent embeddings without direct access to training data from the
Cart-Pole simulator. In general, the learned embedding space is helpful for
mapping out the space of solutions for a given task. We observe in the case of
Cart-Pole the surprising finding that good agents make different decisions
despite learning similar representations, whereas bad agents make similar (bad)
decisions while learning dissimilar representations. Linearly interpolating
between the latent embeddings for a good agent and a bad agent yields an agent
embedding that generates a network with intermediate performance, where the
performance can be tuned according to the coefficient of interpolation. Linear
extrapolation in the latent space also results in performance boosts, up to a
point
TXtract: Taxonomy-Aware Knowledge Extraction for Thousands of Product Categories
Extracting structured knowledge from product profiles is crucial for various
applications in e-Commerce. State-of-the-art approaches for knowledge
extraction were each designed for a single category of product, and thus do not
apply to real-life e-Commerce scenarios, which often contain thousands of
diverse categories. This paper proposes TXtract, a taxonomy-aware knowledge
extraction model that applies to thousands of product categories organized in a
hierarchical taxonomy. Through category conditional self-attention and
multi-task learning, our approach is both scalable, as it trains a single model
for thousands of categories, and effective, as it extracts category-specific
attribute values. Experiments on products from a taxonomy with 4,000 categories
show that TXtract outperforms state-of-the-art approaches by up to 10% in F1
and 15% in coverage across all categories.Comment: Accepted to ACL 2020 (Long Paper
Conditional BERT Contextual Augmentation
We propose a novel data augmentation method for labeled sentences called
conditional BERT contextual augmentation. Data augmentation methods are often
applied to prevent overfitting and improve generalization of deep neural
network models. Recently proposed contextual augmentation augments labeled
sentences by randomly replacing words with more varied substitutions predicted
by language model. BERT demonstrates that a deep bidirectional language model
is more powerful than either an unidirectional language model or the shallow
concatenation of a forward and backward model. We retrofit BERT to conditional
BERT by introducing a new conditional masked language model\footnote{The term
"conditional masked language model" appeared once in original BERT paper, which
indicates context-conditional, is equivalent to term "masked language model".
In our paper, "conditional masked language model" indicates we apply extra
label-conditional constraint to the "masked language model".} task. The well
trained conditional BERT can be applied to enhance contextual augmentation.
Experiments on six various different text classification tasks show that our
method can be easily applied to both convolutional or recurrent neural networks
classifier to obtain obvious improvement.Comment: 9 pages, 1 figur
CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases
How can we enable computers to automatically answer questions like "Who
created the character Harry Potter"? Carefully built knowledge bases provide
rich sources of facts. However, it remains a challenge to answer factoid
questions raised in natural language due to numerous expressions of one
question. In particular, we focus on the most common questions --- ones that
can be answered with a single fact in the knowledge base. We propose CFO, a
Conditional Focused neural-network-based approach to answering factoid
questions with knowledge bases. Our approach first zooms in a question to find
more probable candidate subject mentions, and infers the final answers with a
unified conditional probabilistic framework. Powered by deep recurrent neural
networks and neural embeddings, our proposed CFO achieves an accuracy of 75.7%
on a dataset of 108k questions - the largest public one to date. It outperforms
the current state of the art by an absolute margin of 11.8%.Comment: Accepted by ACL 201
- …