25 research outputs found
Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks
Recommendations can greatly benefit from good representations of the user
state at recommendation time. Recent approaches that leverage Recurrent Neural
Networks (RNNs) for session-based recommendations have shown that Deep Learning
models can provide useful user representations for recommendation. However,
current RNN modeling approaches summarize the user state by only taking into
account the sequence of items that the user has interacted with in the past,
without taking into account other essential types of context information such
as the associated types of user-item interactions, the time gaps between events
and the time of day for each interaction. To address this, we propose a new
class of Contextual Recurrent Neural Networks for Recommendation (CRNNs) that
can take into account the contextual information both in the input and output
layers and modifying the behavior of the RNN by combining the context embedding
with the item embedding and more explicitly, in the model dynamics, by
parametrizing the hidden unit transitions as a function of context information.
We compare our CRNNs approach with RNNs and non-sequential baselines and show
good improvements on the next event prediction task
Contextual Attention Recurrent Architecture for Context-aware Venue Recommendation
Venue recommendation systems aim to effectively rank a list of interesting venues users should visit based on their historical feedback (e.g. checkins). Such systems are increasingly deployed by Location-based Social Networks (LBSNs) such as Foursquare and Yelp to enhance their usefulness to users. Recently, various RNN architectures have been proposed to incorporate contextual information associated with the users' sequence of checkins (e.g. time of the day, location of venues) to effectively capture the users' dynamic preferences. However, these architectures assume that different types of contexts have an identical impact on the users' preferences, which may not hold in practice. For example, an ordinary context such as the time of the day reflects the user's current contextual preferences, whereas a transition context - such as a time interval from their last visited venue - indicates a transition effect from past behaviour to future behaviour. To address these challenges, we propose a novel Contextual Attention Recurrent Architecture (CARA) that leverages both sequences of feedback and contextual information associated with the sequences to capture the users' dynamic preferences. Our proposed recurrent architecture consists of two types of gating mechanisms, namely 1) a contextual attention gate that controls the influence of the ordinary context on the users' contextual preferences and 2) a time- and geo-based gate that controls the influence of the hidden state from the previous checkin based on the transition context. Thorough experiments on three large checkin and rating datasets from commercial LBSNs demonstrate the effectiveness of our proposed CARA architecture by significantly outperforming many state-of-the-art RNN architectures and factorisation approaches
Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation
Sequential recommender systems aim to model users' evolving interests from
their historical behaviors, and hence make customized time-relevant
recommendations. Compared with traditional models, deep learning approaches
such as CNN and RNN have achieved remarkable advancements in recommendation
tasks. Recently, the BERT framework also emerges as a promising method,
benefited from its self-attention mechanism in processing sequential data.
However, one limitation of the original BERT framework is that it only
considers one input source of the natural language tokens. It is still an open
question to leverage various types of information under the BERT framework.
Nonetheless, it is intuitively appealing to utilize other side information,
such as item category or tag, for more comprehensive depictions and better
recommendations. In our pilot experiments, we found naive approaches, which
directly fuse types of side information into the item embeddings, usually bring
very little or even negative effects. Therefore, in this paper, we propose the
NOninVasive self-attention mechanism (NOVA) to leverage side information
effectively under the BERT framework. NOVA makes use of side information to
generate better attention distribution, rather than directly altering the item
embedding, which may cause information overwhelming. We validate the NOVA-BERT
model on both public and commercial datasets, and our method can stably
outperform the state-of-the-art models with negligible computational overheads.Comment: Accepted at AAAI 202