115 research outputs found
Rethinking Knowledge Graph Evaluation Under the Open-World Assumption
Most knowledge graphs (KGs) are incomplete, which motivates one important
research topic on automatically complementing knowledge graphs. However,
evaluation of knowledge graph completion (KGC) models often ignores the
incompleteness -- facts in the test set are ranked against all unknown triplets
which may contain a large number of missing facts not included in the KG yet.
Treating all unknown triplets as false is called the closed-world assumption.
This closed-world assumption might negatively affect the fairness and
consistency of the evaluation metrics. In this paper, we study KGC evaluation
under a more realistic setting, namely the open-world assumption, where unknown
triplets are considered to include many missing facts not included in the
training or test sets. For the currently most used metrics such as mean
reciprocal rank (MRR) and Hits@K, we point out that their behavior may be
unexpected under the open-world assumption. Specifically, with not many missing
facts, their numbers show a logarithmic trend with respect to the true strength
of the model, and thus, the metric increase could be insignificant in terms of
reflecting the true model improvement. Further, considering the variance, we
show that the degradation in the reported numbers may result in incorrect
comparisons between different models, where stronger models may have lower
metric numbers. We validate the phenomenon both theoretically and
experimentally. Finally, we suggest possible causes and solutions for this
problem. Our code and data are available at
https://github.com/GraphPKU/Open-World-KG .Comment: Accepted at NeurIPS 202
Weakly-Supervised Dense Action Anticipation
Dense anticipation aims to forecast future actions and their durations for
long horizons. Existing approaches rely on fully-labelled data, i.e. sequences
labelled with all future actions and their durations. We present a (semi-)
weakly supervised method using only a small number of fully-labelled sequences
and predominantly sequences in which only the (one) upcoming action is
labelled. To this end, we propose a framework that generates pseudo-labels for
future actions and their durations and adaptively refines them through a
refinement module. Given only the upcoming action label as input, these
pseudo-labels guide action/duration prediction for the future. We further
design an attention mechanism to predict context-aware durations. Experiments
on the Breakfast and 50Salads benchmarks verify our method's effectiveness; we
are competitive even when compared to fully supervised state-of-the-art models.
We will make our code available at:
https://github.com/zhanghaotong1/WSLVideoDenseAnticipation.Comment: BMVC 202
Neural Common Neighbor with Completion for Link Prediction
Despite its outstanding performance in various graph tasks, vanilla Message
Passing Neural Network (MPNN) usually fails in link prediction tasks, as it
only uses representations of two individual target nodes and ignores the
pairwise relation between them. To capture the pairwise relations, some models
add manual features to the input graph and use the output of MPNN to produce
pairwise representations. In contrast, others directly use manual features as
pairwise representations. Though this simplification avoids applying a GNN to
each link individually and thus improves scalability, these models still have
much room for performance improvement due to the hand-crafted and unlearnable
pairwise features. To upgrade performance while maintaining scalability, we
propose Neural Common Neighbor (NCN), which uses learnable pairwise
representations. To further boost NCN, we study the unobserved link problem.
The incompleteness of the graph is ubiquitous and leads to distribution shifts
between the training and test set, loss of common neighbor information, and
performance degradation of models. Therefore, we propose two intervention
methods: common neighbor completion and target link removal. Combining the two
methods with NCN, we propose Neural Common Neighbor with Completion (NCNC). NCN
and NCNC outperform recent strong baselines by large margins. NCNC achieves
state-of-the-art performance in link prediction tasks. Our code is available at
https://github.com/GraphPKU/NeuralCommonNeighbor
Code Prompting: a Neural Symbolic Method for Complex Reasoning in Large Language Models
Large language models (LLMs) have scaled up to unlock a wide range of complex
reasoning tasks with the aid of various prompting methods. However, current
prompting methods generate natural language intermediate steps to help
reasoning, which can cause imperfect task reduction and confusion. To mitigate
such limitations, we explore code prompting, a neural symbolic prompting method
with both zero-shot and few-shot versions which triggers code as intermediate
steps. We conduct experiments on 7 widely-used benchmarks involving symbolic
reasoning and arithmetic reasoning. Code prompting generally outperforms
chain-of-thought (CoT) prompting. To further understand the performance and
limitations of code prompting, we perform extensive ablation studies and error
analyses, and identify several exclusive advantages of using symbolic
promptings compared to natural language. We also consider the ensemble of code
prompting and CoT prompting to combine the strengths of both. Finally, we show
through experiments how code annotations and their locations affect code
prompting
H chromospheric activity of F-, G-, and K-type stars observed by the LAMOST Medium-Resolution Spectroscopic Survey
Distribution of stellar chromospheric activity with
respect to stellar atmospheric parameters (effective temperature
, surface gravity , and metallicity )
and main-sequence/giant categories is investigated for the F-, G-, and K-type
stars observed by the LAMOST Medium-Resolution Spectroscopic Survey (MRS). A
total of 329,294 MRS spectra from LAMOST DR8 are utilized in the analysis. The
activity index () and the
-index () are evaluated for the
MRS spectra. The chromospheric activity distributions with
individual stellar parameters as well as in the --
and -- parameter spaces are analyzed based
on the index data. It is found that: (1) for the
main-sequence sample, the distribution with
has a bowl-shaped lower envelope with a minimum at about 6200
K, a hill-shaped middle envelope with a maximum at about 5600 K, and an upper
envelope continuing to increase from hotter to cooler stars; (2) for the giant
sample, the middle and upper envelopes of the
distribution first increase with decrease of and then drop to
a lower activity level at about 4300 K, revealing the different activity
characteristics at different stages of stellar evolution; (3) for both the
main-sequence and giant samples, the upper envelope of the
distribution with metallicity is higher for stars with
greater than about , and the lowest-metallicity stars
hardly exhibit high indices. A dataset of
activity indices for the LAMOST MRS spectra analyzed is provided with this
paper.Comment: 32 pages, 12 figures, 1 table, accepted for publication in
Astrophysics and Space Scienc
- …