120 research outputs found
Contextual Dictionary Lookup for Knowledge Graph Completion
Knowledge graph completion (KGC) aims to solve the incompleteness of
knowledge graphs (KGs) by predicting missing links from known triples, numbers
of knowledge graph embedding (KGE) models have been proposed to perform KGC by
learning embeddings. Nevertheless, most existing embedding models map each
relation into a unique vector, overlooking the specific fine-grained semantics
of them under different entities. Additionally, the few available fine-grained
semantic models rely on clustering algorithms, resulting in limited performance
and applicability due to the cumbersome two-stage training process. In this
paper, we present a novel method utilizing contextual dictionary lookup,
enabling conventional embedding models to learn fine-grained semantics of
relations in an end-to-end manner. More specifically, we represent each
relation using a dictionary that contains multiple latent semantics. The
composition of a given entity and the dictionary's central semantics serves as
the context for generating a lookup, thus determining the fine-grained
semantics of the relation adaptively. The proposed loss function optimizes both
the central and fine-grained semantics simultaneously to ensure their semantic
consistency. Besides, we introduce two metrics to assess the validity and
accuracy of the dictionary lookup operation. We extend several KGE models with
the method, resulting in substantial performance improvements on widely-used
benchmark datasets
Dynamically Relative Position Encoding-Based Transformer for Automatic Code Edit
Adapting Deep Learning (DL) techniques to automate non-trivial coding
activities, such as code documentation and defect detection, has been
intensively studied recently. Learning to predict code changes is one of the
popular and essential investigations. Prior studies have shown that DL
techniques such as Neural Machine Translation (NMT) can benefit meaningful code
changes, including bug fixing and code refactoring. However, NMT models may
encounter bottleneck when modeling long sequences, thus are limited in
accurately predicting code changes. In this work, we design a Transformer-based
approach, considering that Transformer has proven effective in capturing
long-term dependencies. Specifically, we propose a novel model named DTrans.
For better incorporating the local structure of code, i.e., statement-level
information in this paper, DTrans is designed with dynamically relative
position encoding in the multi-head attention of Transformer. Experiments on
benchmark datasets demonstrate that DTrans can more accurately generate patches
than the state-of-the-art methods, increasing the performance by at least
5.45\%-46.57\% in terms of the exact match metric on different datasets.
Moreover, DTrans can locate the lines to change with 1.75\%-24.21\% higher
accuracy than the existing methods
A Novel Two-Layer DAG-based Reactive Protocol for IoT Data Reliability in Metaverse
Many applications, e.g., digital twins, rely on sensing data from Internet of
Things (IoT) networks, which is used to infer event(s) and initiate actions to
affect an environment. This gives rise to concerns relating to data integrity
and provenance. One possible solution to address these concerns is to employ
blockchain. However, blockchain has high resource requirements, thereby making
it unsuitable for use on resource-constrained IoT devices. To this end, this
paper proposes a novel approach, called two-layer directed acyclic graph
(2LDAG), whereby IoT devices only store a digital fingerprint of data generated
by their neighbors. Further, it proposes a novel proof-of-path (PoP) protocol
that allows an operator or digital twin to verify data in an on-demand manner.
The simulation results show 2LDAG has storage and communication cost that is
respectively two and three orders of magnitude lower than traditional
blockchain and also blockchains that use a DAG structure. Moreover, 2LDAG
achieves consensus even when 49\% of nodes are malicious
Ensembled CTR Prediction via Knowledge Distillation
Recently, deep learning-based models have been widely studied for
click-through rate (CTR) prediction and lead to improved prediction accuracy in
many industrial applications. However, current research focuses primarily on
building complex network architectures to better capture sophisticated feature
interactions and dynamic user behaviors. The increased model complexity may
slow down online inference and hinder its adoption in real-time applications.
Instead, our work targets at a new model training strategy based on knowledge
distillation (KD). KD is a teacher-student learning framework to transfer
knowledge learned from a teacher model to a student model. The KD strategy not
only allows us to simplify the student model as a vanilla DNN model but also
achieves significant accuracy improvements over the state-of-the-art teacher
models. The benefits thus motivate us to further explore the use of a powerful
ensemble of teachers for more accurate student model training. We also propose
some novel techniques to facilitate ensembled CTR prediction, including teacher
gating and early stopping by distillation loss. We conduct comprehensive
experiments against 12 existing models and across three industrial datasets.
Both offline and online A/B testing results show the effectiveness of our
KD-based training strategy.Comment: Published in CIKM'202
- …