Virtual Node Tuning for Few-shot Node Classification

Few-shot Node Classification (FSNC) is a challenge in graph representation learning where only a few labeled nodes per class are available for training. To tackle this issue, meta-learning has been proposed to transfer structural knowledge from base classes with abundant labels to target novel classes. However, existing solutions become ineffective or inapplicable when base classes have no or limited labeled nodes. To address this challenge, we propose an innovative method dubbed Virtual Node Tuning (VNT). Our approach utilizes a pretrained graph transformer as the encoder and injects virtual nodes as soft prompts in the embedding space, which can be optimized with few-shot labels in novel classes to modulate node embeddings for each specific FSNC task. A unique feature of VNT is that, by incorporating a Graph-based Pseudo Prompt Evolution (GPPE) module, VNT-GPPE can handle scenarios with sparse labels in base classes. Experimental results on four datasets demonstrate the superiority of the proposed approach in addressing FSNC with unlabeled or sparsely labeled base classes, outperforming existing state-of-the-art methods and even fully supervised baselines.Comment: Accepted to KDD 202

Contextualization Distillation from Large Language Model for Knowledge Graph Completion

While textual information significantly enhances the performance of pre-trained language models (PLMs) in knowledge graph completion (KGC), the static and noisy nature of existing corpora collected from Wikipedia articles or synsets definitions often limits the potential of PLM-based KGC models. To surmount these challenges, we introduce the Contextualization Distillation strategy, a versatile plug-in-and-play approach compatible with both discriminative and generative KGC frameworks. Our method begins by instructing large language models (LLMs) to transform compact, structural triplets into context-rich segments. Subsequently, we introduce two tailored auxiliary tasks, reconstruction and contextualization, allowing smaller KGC models to assimilate insights from these enriched triplets. Comprehensive evaluations across diverse datasets and KGC techniques highlight the efficacy and adaptability of our approach, revealing consistent performance enhancements irrespective of underlying pipelines or architectures. Moreover, our analysis makes our method more explainable and provides insight into generating path selection, as well as the choosing of suitable distillation tasks. All the code and data in this work will be released at https://github.com/David-Li0406/Contextulization-DistillationComment: Accepted by EACL 2024 findings v3: add missing citation

Tuning-Free Accountable Intervention for LLM Deployment -- A Metacognitive Approach

Large Language Models (LLMs) have catalyzed transformative advances across a spectrum of natural language processing tasks through few-shot or zero-shot prompting, bypassing the need for parameter tuning. While convenient, this modus operandi aggravates ``hallucination'' concerns, particularly given the enigmatic ``black-box'' nature behind their gigantic model sizes. Such concerns are exacerbated in high-stakes applications (e.g., healthcare), where unaccountable decision errors can lead to devastating consequences. In contrast, human decision-making relies on nuanced cognitive processes, such as the ability to sense and adaptively correct misjudgments through conceptual understanding. Drawing inspiration from human cognition, we propose an innovative \textit{metacognitive} approach, dubbed \textbf{CLEAR}, to equip LLMs with capabilities for self-aware error identification and correction. Our framework facilitates the construction of concept-specific sparse subnetworks that illuminate transparent decision pathways. This provides a novel interface for model \textit{intervention} after deployment. Our intervention offers compelling advantages: (\textit{i})~at deployment or inference time, our metacognitive LLMs can self-consciously identify potential mispredictions with minimum human involvement, (\textit{ii})~the model has the capability to self-correct its errors efficiently, obviating the need for additional tuning, and (\textit{iii})~the rectification procedure is not only self-explanatory but also user-friendly, enhancing the interpretability and accessibility of the model. By integrating these metacognitive features, our approach pioneers a new path toward engendering greater trustworthiness and accountability in the deployment of LLMs

Contrastive Meta-Learning for Few-shot Node Classification

Few-shot node classification, which aims to predict labels for nodes on graphs with only limited labeled nodes as references, is of great significance in real-world graph mining tasks. Particularly, in this paper, we refer to the task of classifying nodes in classes with a few labeled nodes as the few-shot node classification problem. To tackle such a label shortage issue, existing works generally leverage the meta-learning framework, which utilizes a number of episodes to extract transferable knowledge from classes with abundant labeled nodes and generalizes the knowledge to other classes with limited labeled nodes. In essence, the primary aim of few-shot node classification is to learn node embeddings that are generalizable across different classes. To accomplish this, the GNN encoder must be able to distinguish node embeddings between different classes, while also aligning embeddings for nodes in the same class. Thus, in this work, we propose to consider both the intra-class and inter-class generalizability of the model. We create a novel contrastive meta-learning framework on graphs, named COSMIC, with two key designs. First, we propose to enhance the intra-class generalizability by involving a contrastive two-step optimization in each episode to explicitly align node embeddings in the same classes. Second, we strengthen the inter-class generalizability by generating hard node classes via a novel similarity-sensitive mix-up strategy. Extensive experiments on few-shot node classification datasets verify the superiority of our framework over state-of-the-art baselines. Our code is provided at https://github.com/SongW-SW/COSMIC.Comment: SIGKDD 202

Media Bias Matters: Understanding the Impact of Politically Biased News on Vaccine Attitudes in Social Media

News media has been utilized as a political tool to stray from facts, presenting biased claims without evidence. Amid the COVID-19 pandemic, politically biased news (PBN) has significantly undermined public trust in vaccines, despite strong medical evidence supporting their efficacy. In this paper, we analyze: (i) how inherent vaccine stances subtly influence individuals' selection of news sources and participation in social media discussions; and (ii) the impact of exposure to PBN on users' attitudes toward vaccines. In doing so, we first curate a comprehensive dataset that connects PBN with related social media discourse. Utilizing advanced deep learning and causal inference techniques, we reveal distinct user behaviors between social media groups with various vaccine stances. Moreover, we observe that individuals with moderate stances, particularly the vaccine-hesitant majority, are more vulnerable to the influence of PBN compared to those with extreme views. Our findings provide critical insights to foster this line of research.Comment: 9 pages, 6 figures, 3 table

Transductive Linear Probing: A Novel Framework for Few-Shot Node Classification

Few-shot node classification is tasked to provide accurate predictions for nodes from novel classes with only few representative labeled nodes. This problem has drawn tremendous attention for its projection to prevailing real-world applications, such as product categorization for newly added commodity categories on an E-commerce platform with scarce records or diagnoses for rare diseases on a patient similarity graph. To tackle such challenging label scarcity issues in the non-Euclidean graph domain, meta-learning has become a successful and predominant paradigm. More recently, inspired by the development of graph self-supervised learning, transferring pretrained node embeddings for few-shot node classification could be a promising alternative to meta-learning but remains unexposed. In this work, we empirically demonstrate the potential of an alternative framework, \textit{Transductive Linear Probing}, that transfers pretrained node embeddings, which are learned from graph contrastive learning methods. We further extend the setting of few-shot node classification from standard fully supervised to a more realistic self-supervised setting, where meta-learning methods cannot be easily deployed due to the shortage of supervision from training classes. Surprisingly, even without any ground-truth labels, transductive linear probing with self-supervised graph contrastive pretraining can outperform the state-of-the-art fully supervised meta-learning based methods under the same protocol. We hope this work can shed new light on few-shot node classification problems and foster future research on learning from scarcely labeled instances on graphs.Comment: Accepted to the First Learning on Graph Conference (LoG 2022

Discontinuous Galerkin Immersed Finite Volume Element Method for Anisotropic Flow Models in Porous Medium

By choosing the trial function space to the immersed finite element space and the test function space to be piecewise constant function space, we develop a discontinuous Galerkin immersed finite volume element method to solve numerically a kind of anisotropic diffusion models governed by the elliptic interface problems with discontinuous tensor-conductivity. The existence and uniqueness of the discrete scheme are proved, and an optimal-order energy-norm estimate and L2-norm estimate for the numerical solution are derived

A comparison of posterior lumbar interbody fusion and transforaminal lumbar interbody fusion: a literature review and meta-analysis

BACKGROUND: We compared the perioperative results and complications associated with PLIF and TLIF, and collected evidence for choosing the better fusion method. METHODS: A literature survey of the MEDLINE and EMBASE databases identified 7 comparative observational studies that met our inclusion criteria. Checklists by Cowley were used to evaluate the risk of bias of the included studies. A database including patient demographic information, perioperative results, and complications was established. The summary odds ratio and weighed mean difference with 95% confidence interval were calculated with a random-effects model. RESULTS: We found that PLIF had a higher complication rate (P <0.00001), and TLIF reduced the rate of durotomy (P = 0.01). No statistical difference was found between the two groups with regard to clinical satisfaction (P = 0.54), blood loss (P = 0.14), vertebral root injury (P = 0.08), graft malposition (P = 0.06), infection (P = 0.36), or rate of radiographic fusion (P = 0.27). The evidence indicated that PLIF required longer operative time (P = 0.03). CONCLUSIONS: The evidence indicated that TLIF could reduce the complication rate and durotomy. Neither TLIP nor PLIF was found superior in terms of clinical satisfaction or radiographic fusion rate. PLIF might result in longer time in surgery. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/1471-2474-15-367) contains supplementary material, which is available to authorized users

Evolution characteristics on coal fractures induced with CO2 phase transition fracturing based on CT scanning

This study conducted the CO2-PTF coal experiment to further reveal the fracturing transformation mechanism of CO2 phase transition fracturing(CO2-PTF)coal. According to the CT scanning and 3D fracture reconstruction, we analyzed the fracture structure parameters of coal before and after CO2-PTF, and clarified the evolution characteristics of the three-dimensional fracture structure of coal induced by CO2-PTF. The research results indicated that after CO2-PTF, the total number of fractures in the coal sample decreased, while the total volume and surface area of fractures increased. The CO2-PTF generated fracture expansion and transformation effects where the small-scale fractures were expanded and transformed into larger scale fractures under the CO2-PTF pressure. The number, volume, and surface area of fractures of less than 1 000 μm in length were significantly reduced, while the volume and surface area of fractures of longer than 1 000 μm in length were significantly increased. The expansion and connection between fractures caused a decrease in their quantity. CO2-PTF improves the connectivity of the three-dimensional fracture in coal and is conducive to gas migration and production. This study offers new insights into and evaluation method for the effect of CO2-PTF, and could provide references for the research on fracture evolution characteristics in other unconventional natural gas reservoirs and their modifications