5 research outputs found
Graph-incorporated Latent Factor Analysis for High-dimensional and Sparse Matrices
A High-dimensional and sparse (HiDS) matrix is frequently encountered in a
big data-related application like an e-commerce system or a social network
services system. To perform highly accurate representation learning on it is of
great significance owing to the great desire of extracting latent knowledge and
patterns from it. Latent factor analysis (LFA), which represents an HiDS matrix
by learning the low-rank embeddings based on its observed entries only, is one
of the most effective and efficient approaches to this issue. However, most
existing LFA-based models perform such embeddings on a HiDS matrix directly
without exploiting its hidden graph structures, thereby resulting in accuracy
loss. To address this issue, this paper proposes a graph-incorporated latent
factor analysis (GLFA) model. It adopts two-fold ideas: 1) a graph is
constructed for identifying the hidden high-order interaction (HOI) among nodes
described by an HiDS matrix, and 2) a recurrent LFA structure is carefully
designed with the incorporation of HOI, thereby improving the representa-tion
learning ability of a resultant model. Experimental results on three real-world
datasets demonstrate that GLFA outperforms six state-of-the-art models in
predicting the missing data of an HiDS matrix, which evidently supports its
strong representation learning ability to HiDS data
NGAT4Rec: Neighbor-Aware Graph Attention Network For Recommendation
Learning informative representations (aka. embeddings) of users and items is
the core of modern recommender systems. Previous works exploit user-item
relationships of one-hop neighbors in the user-item interaction graph to
improve the quality of representation. Recently, the research of Graph Neural
Network (GNN) for recommendation considers the implicit collaborative
information of multi-hop neighbors to enrich the representation. However, most
works of GNN for recommendation systems do not consider the relational
information which implies the expression differences of different neighbors in
the neighborhood explicitly. The influence of each neighboring item to the
representation of the user's preference can be represented by the correlation
between the item and neighboring items of the user. Symmetrically, for a given
item, the correlation between one neighboring user and neighboring users can
reflect the strength of signal about the item's characteristic. To modeling the
implicit correlations of neighbors in graph embedding aggregating, we propose a
Neighbor-Aware Graph Attention Network for recommendation task, termed
NGAT4Rec. It employs a novel neighbor-aware graph attention layer that assigns
different neighbor-aware attention coefficients to different neighbors of a
given node by computing the attention among these neighbors pairwisely. Then
NGAT4Rec aggregates the embeddings of neighbors according to the corresponding
neighbor-aware attention coefficients to generate next layer embedding for
every node. Furthermore, we combine more neighbor-aware graph attention layer
to gather the influential signals from multi-hop neighbors. We remove feature
transformation and nonlinear activation that proved to be useless on
collaborative filtering. Extensive experiments on three benchmark datasets show
that our model outperforms various state-of-the-art models consistently
Fast Partial Fourier Transform
Given a time series vector, how can we efficiently compute a specified part
of Fourier coefficients? Fast Fourier transform (FFT) is a widely used
algorithm that computes the discrete Fourier transform in many machine learning
applications. Despite its pervasive use, all known FFT algorithms do not
provide a fine-tuning option for the user to specify one's demand, that is, the
output size (the number of Fourier coefficients to be computed) is
algorithmically determined by the input size. This matters because not every
application using FFT requires the whole spectrum of the frequency domain,
resulting in an inefficiency due to extra computation. In this paper, we
propose a fast Partial Fourier Transform (PFT), a careful modification of the
Cooley-Tukey algorithm that enables one to specify an arbitrary consecutive
range where the coefficients should be computed. We derive the asymptotic time
complexity of PFT with respect to input and output sizes, as well as its
numerical accuracy. Experimental results show that our algorithm outperforms
the state-of-the-art FFT algorithms, with an order of magnitude of speedup for
sufficiently small output sizes without sacrificing accuracy.Comment: 15 pages, 3 figure
Curriculum Pre-Training Heterogeneous Subgraph Transformer for Top- Recommendation
Due to the flexibility in modelling data heterogeneity, heterogeneous
information network (HIN) has been adopted to characterize complex and
heterogeneous auxiliary data in top- recommender systems, called
\emph{HIN-based recommendation}. HIN characterizes complex, heterogeneous data
relations, containing a variety of information that may not be related to the
recommendation task. Therefore, it is challenging to effectively leverage
useful information from HINs for improving the recommendation performance. To
address the above issue, we propose a Curriculum pre-training based
HEterogeneous Subgraph Transformer (called \emph{CHEST}) with new \emph{data
characterization}, \emph{representation model} and \emph{learning algorithm}.
Specifically, we consider extracting useful information from HIN to compose
the interaction-specific heterogeneous subgraph, containing both sufficient and
relevant context information for recommendation. Then we capture the rich
semantics (\eg graph structure and path semantics) within the subgraph via a
heterogeneous subgraph Transformer, where we encode the subgraph with
multi-slot sequence representations. Besides, we design a curriculum
pre-training strategy to provide an elementary-to-advanced learning process, by
which we smoothly transfer basic semantics in HIN for modeling user-item
interaction relation.
Extensive experiments conducted on three real-world datasets demonstrate the
superiority of our proposed method over a number of competitive baselines,
especially when only limited training data is available.Comment: 26 page
GraphHINGE: Learning Interaction Models of Structured Neighborhood on Heterogeneous Information Network
Heterogeneous information network (HIN) has been widely used to characterize
entities of various types and their complex relations. Recent attempts either
rely on explicit path reachability to leverage path-based semantic relatedness
or graph neighborhood to learn heterogeneous network representations before
predictions. These weakly coupled manners overlook the rich interactions among
neighbor nodes, which introduces an early summarization issue. In this paper,
we propose GraphHINGE (Heterogeneous INteract and aggreGatE), which captures
and aggregates the interactive patterns between each pair of nodes through
their structured neighborhoods. Specifically, we first introduce
Neighborhood-based Interaction (NI) module to model the interactive patterns
under the same metapaths, and then extend it to Cross Neighborhood-based
Interaction (CNI) module to deal with different metapaths. Next, in order to
address the complexity issue on large-scale networks, we formulate the
interaction modules via a convolutional framework and learn the parameters
efficiently with fast Fourier transform. Furthermore, we design a novel
neighborhood-based selection (NS) mechanism, a sampling strategy, to filter
high-order neighborhood information based on their low-order performance. The
extensive experiments on six different types of heterogeneous graphs
demonstrate the performance gains by comparing with state-of-the-arts in both
click-through rate prediction and top-N recommendation tasks.Comment: TOIS (Special Issue on Graph Technologies for User Modeling and
Recommendation). arXiv admin note: text overlap with arXiv:2007.0021