9 research outputs found
Learning Hierarchical Review Graph Representations for Recommendation
The user review data have been demonstrated to be effective in solving
different recommendation problems. Previous review-based recommendation methods
usually employ sophisticated compositional models, such as Recurrent Neural
Networks (RNN) and Convolutional Neural Networks (CNN), to learn semantic
representations from the review data for recommendation. However, these methods
mainly capture the local dependency between neighbouring words in a word
window, and they treat each review equally. Therefore, they may not be
effective in capturing the global dependency between words, and tend to be
easily biased by noise review information. In this paper, we propose a novel
review-based recommendation model, named Review Graph Neural Network (RGNN).
Specifically, RGNN builds a specific review graph for each individual
user/item, which provides a global view about the user/item properties to help
weaken the biases caused by noise review information. A type-aware graph
attention mechanism is developed to learn semantic embeddings of words.
Moreover, a personalized graph pooling operator is proposed to learn
hierarchical representations of the review graph to form the semantic
representation for each user/item. We compared RGNN with state-of-the-art
review-based recommendation approaches on two real-world datasets. The
experimental results indicate that RGNN consistently outperforms baseline
methods, in terms of Mean Square Error (MSE)
ViCGCN: Graph Convolutional Network with Contextualized Language Models for Social Media Mining in Vietnamese
Social media processing is a fundamental task in natural language processing
with numerous applications. As Vietnamese social media and information science
have grown rapidly, the necessity of information-based mining on Vietnamese
social media has become crucial. However, state-of-the-art research faces
several significant drawbacks, including imbalanced data and noisy data on
social media platforms. Imbalanced and noisy are two essential issues that need
to be addressed in Vietnamese social media texts. Graph Convolutional Networks
can address the problems of imbalanced and noisy data in text classification on
social media by taking advantage of the graph structure of the data. This study
presents a novel approach based on contextualized language model (PhoBERT) and
graph-based method (Graph Convolutional Networks). In particular, the proposed
approach, ViCGCN, jointly trained the power of Contextualized embeddings with
the ability of Graph Convolutional Networks, GCN, to capture more syntactic and
semantic dependencies to address those drawbacks. Extensive experiments on
various Vietnamese benchmark datasets were conducted to verify our approach.
The observation shows that applying GCN to BERTology models as the final layer
significantly improves performance. Moreover, the experiments demonstrate that
ViCGCN outperforms 13 powerful baseline models, including BERTology models,
fusion BERTology and GCN models, other baselines, and SOTA on three benchmark
social media datasets. Our proposed ViCGCN approach demonstrates a significant
improvement of up to 6.21%, 4.61%, and 2.63% over the best Contextualized
Language Models, including multilingual and monolingual, on three benchmark
datasets, UIT-VSMEC, UIT-ViCTSD, and UIT-VSFC, respectively. Additionally, our
integrated model ViCGCN achieves the best performance compared to other
BERTology integrated with GCN models
Predicting Viral Rumors and Vulnerable Users for Infodemic Surveillance
In the age of the infodemic, it is crucial to have tools for effectively
monitoring the spread of rampant rumors that can quickly go viral, as well as
identifying vulnerable users who may be more susceptible to spreading such
misinformation. This proactive approach allows for timely preventive measures
to be taken, mitigating the negative impact of false information on society. We
propose a novel approach to predict viral rumors and vulnerable users using a
unified graph neural network model. We pre-train network-based user embeddings
and leverage a cross-attention mechanism between users and posts, together with
a community-enhanced vulnerability propagation (CVP) method to improve user and
propagation graph representations. Furthermore, we employ two multi-task
training strategies to mitigate negative transfer effects among tasks in
different settings, enhancing the overall performance of our approach. We also
construct two datasets with ground-truth annotations on information virality
and user vulnerability in rumor and non-rumor events, which are automatically
derived from existing rumor detection datasets. Extensive evaluation results of
our joint learning model confirm its superiority over strong baselines in all
three tasks: rumor detection, virality prediction, and user vulnerability
scoring. For instance, compared to the best baselines based on the Weibo
dataset, our model makes 3.8\% and 3.0\% improvements on Accuracy and MacF1 for
rumor detection, and reduces mean squared error (MSE) by 23.9\% and 16.5\% for
virality prediction and user vulnerability scoring, respectively. Our findings
suggest that our approach effectively captures the correlation between rumor
virality and user vulnerability, leveraging this information to improve
prediction performance and provide a valuable tool for infodemic surveillance.Comment: Accepted by IP&
Graph Neural Networks for Natural Language Processing: A Survey
Deep learning has become the dominant approach in coping with various tasks
in Natural LanguageProcessing (NLP). Although text inputs are typically
represented as a sequence of tokens, there isa rich variety of NLP problems
that can be best expressed with a graph structure. As a result, thereis a surge
of interests in developing new deep learning techniques on graphs for a large
numberof NLP tasks. In this survey, we present a comprehensive overview onGraph
Neural Networks(GNNs) for Natural Language Processing. We propose a new
taxonomy of GNNs for NLP, whichsystematically organizes existing research of
GNNs for NLP along three axes: graph construction,graph representation
learning, and graph based encoder-decoder models. We further introducea large
number of NLP applications that are exploiting the power of GNNs and summarize
thecorresponding benchmark datasets, evaluation metrics, and open-source codes.
Finally, we discussvarious outstanding challenges for making the full use of
GNNs for NLP as well as future researchdirections. To the best of our
knowledge, this is the first comprehensive overview of Graph NeuralNetworks for
Natural Language Processing.Comment: 127 page
Graph Neural Networks: A Feature and Structure Learning Approach
Deep neural networks (DNNs) have achieved great success on grid-like data such as images, but face tremendous challenges in learning from more generic data such as graphs. In convolutional neural networks (CNNs), for example, the trainable local filters enable the automatic extraction of high-level features. The computation with filters requires a fixed number of ordered units in the receptive fields. However, the number of neighboring units is neither fixed nor are they ordered in generic graphs, thereby hindering the applications of deep learning operations such as convolution, attention, pooling, and unpooling. To address these limitations, we propose several deep learning methods on graph data in this dissertation.
Graph deep learning methods can be categorized into graph feature learning and graph structure learning. In the category of graph feature learning, we propose to learn graph features via learnable graph convolution operations, graph attention operations, and line graph structures. In learnable graph convolution operations, we propose the learnable graph convolutional layer (LGCL). LGCL automatically selects a fixed number of neighboring nodes for each feature based on value ranking in order to transform graph data into grid-like structures in 1-D format, thereby enabling the use of regular convolutional operations on generic graphs. In graph attention operations, we propose novel hard graph attention operator (hGAO) and channel-wise graph attention operator (cGAO). hGAO uses the hard attention mechanism by attending to only important nodes. Compared to GAO, hGAO improves performance and saves computational cost by only attending to important nodes. To further reduce the requirements on computational resources, we propose the cGAO that performs attention operations along channels. cGAO avoids the dependency on the adjacency matrix, leading to dramatic reductions in computational resource requirements. Beside using original graph structures, we investigate feature learning on auxiliary graph structures such as line graph. By using line graph structures, we propose a weighted line graph that corrects biases in line graphs by assigning normalized weights to edges. Based on our weighted line graphs, we develop a weighted line graph convolution layer that takes advantage of line graph structures for better feature learning. In particular, it performs message passing operations on both the original graph and its corresponding weighted line graph. To address efficiency issues in line graph neural networks, we propose to use an incidence matrix to accurately compute the adjacency matrix of the weighted line graph, leading to dramatic reductions in computational resource usage.
In the category of graph structure learning, we propose several deep learning methods to learn new graph structures. Given images are special cases of graphs with nodes lie on 2D lattices, graph embedding tasks have a natural correspondence with image pixel-wise prediction tasks such as segmentation. While encoder-decoder architectures like U-Nets have been successfully applied on many image pixel-wise prediction tasks, similar methods are lacking for graph data. This is due to the fact that pooling and up-sampling operations are not natural on graph data. To address these challenges, we propose novel graph pooling (gPool) and unpooling (gUnpool) operations in this work. The gPool layer adaptively selects some nodes to form a smaller graph based on their scalar projection values on a trainable projection vector. However, gPool uses global ranking methods to sample some of the important nodes, which is not able to incorporate graph topology information in computing ranking scores. To address this issue, we propose the topology-aware pooling (TAP) layer that uses attention operators to generate ranking scores for each node by attending each node to its neighboring nodes. The ranking scores are generated locally while the selection is performed globally, which enables the pooling operation to consider topology information. We further propose the gUnpool layer as the inverse operation of the gPool layer. The gUnpool layer restores the graph into its original structure using the position information of nodes selected in the corresponding gPool layer. Based on our proposed gPool and gUnpool layers, we develop an encoder-decoder model on graph, known as the graph U-Nets.
Our experimental results on node classification graph classification tasks using both real and simulated data demonstrate the effectiveness and efficiency of our methods