Next-item recommender systems are often trained using only positive feedback
with randomly-sampled negative feedback. We show the benefits of using real
negative feedback both as inputs into the user sequence and also as negative
targets for training a next-song recommender system for internet radio. In
particular, using explicit negative samples during training helps reduce
training time by ~60% while also improving test accuracy by ~6%; adding user
skips as additional inputs also can considerably increase user coverage
alongside slightly improving accuracy. We test the impact of using a large
number of random negative samples to capture a 'harder' one and find that the
test accuracy increases with more randomly-sampled negatives, but only to a
point. Too many random negatives leads to false negatives that limits the lift,
which is still lower than if using true negative feedback. We also find that
the test accuracy is fairly robust with respect to the proportion of different
feedback types, and compare the learned embeddings for different feedback
types.Comment: 6 pages, 4 figures, accepted to ACM UMAP 202