1,443 research outputs found
Discrete Factorization Machines for Fast Feature-based Recommendation
User and item features of side information are crucial for accurate
recommendation. However, the large number of feature dimensions, e.g., usually
larger than 10^7, results in expensive storage and computational cost. This
prohibits fast recommendation especially on mobile applications where the
computational resource is very limited. In this paper, we develop a generic
feature-based recommendation model, called Discrete Factorization Machine
(DFM), for fast and accurate recommendation. DFM binarizes the real-valued
model parameters (e.g., float32) of every feature embedding into binary codes
(e.g., boolean), and thus supports efficient storage and fast user-item score
computation. To avoid the severe quantization loss of the binarization, we
propose a convergent updating rule that resolves the challenging discrete
optimization of DFM. Through extensive experiments on two real-world datasets,
we show that 1) DFM consistently outperforms state-of-the-art binarized
recommendation models, and 2) DFM shows very competitive performance compared
to its real-valued version (FM), demonstrating the minimized quantization loss.
This work is accepted by IJCAI 2018.Comment: Appeared in IJCAI 201
LightFR: Lightweight Federated Recommendation with Privacy-preserving Matrix Factorization
Federated recommender system (FRS), which enables many local devices to train
a shared model jointly without transmitting local raw data, has become a
prevalent recommendation paradigm with privacy-preserving advantages. However,
previous work on FRS performs similarity search via inner product in continuous
embedding space, which causes an efficiency bottleneck when the scale of items
is extremely large. We argue that such a scheme in federated settings ignores
the limited capacities in resource-constrained user devices (i.e., storage
space, computational overhead, and communication bandwidth), and makes it
harder to be deployed in large-scale recommender systems. Besides, it has been
shown that transmitting local gradients in real-valued form between server and
clients may leak users' private information. To this end, we propose a
lightweight federated recommendation framework with privacy-preserving matrix
factorization, LightFR, that is able to generate high-quality binary codes by
exploiting learning to hash technique under federated settings, and thus enjoys
both fast online inference and economic memory consumption. Moreover, we devise
an efficient federated discrete optimization algorithm to collaboratively train
model parameters between the server and clients, which can effectively prevent
real-valued gradient attacks from malicious parties. Through extensive
experiments on four real-world datasets, we show that our LightFR model
outperforms several state-of-the-art FRS methods in terms of recommendation
accuracy, inference efficiency and data privacy.Comment: Accepted by ACM Transactions on Information Systems (TOIS
Content-aware Neural Hashing for Cold-start Recommendation
Content-aware recommendation approaches are essential for providing
meaningful recommendations for \textit{new} (i.e., \textit{cold-start}) items
in a recommender system. We present a content-aware neural hashing-based
collaborative filtering approach (NeuHash-CF), which generates binary hash
codes for users and items, such that the highly efficient Hamming distance can
be used for estimating user-item relevance. NeuHash-CF is modelled as an
autoencoder architecture, consisting of two joint hashing components for
generating user and item hash codes. Inspired from semantic hashing, the item
hashing component generates a hash code directly from an item's content
information (i.e., it generates cold-start and seen item hash codes in the same
manner). This contrasts existing state-of-the-art models, which treat the two
item cases separately. The user hash codes are generated directly based on user
id, through learning a user embedding matrix. We show experimentally that
NeuHash-CF significantly outperforms state-of-the-art baselines by up to 12\%
NDCG and 13\% MRR in cold-start recommendation settings, and up to 4\% in both
NDCG and MRR in standard settings where all items are present while training.
Our approach uses 2-4x shorter hash codes, while obtaining the same or better
performance compared to the state of the art, thus consequently also enabling a
notable storage reduction.Comment: Accepted to SIGIR 202
Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
Modern deep learning-based recommendation systems exploit hundreds to
thousands of different categorical features, each with millions of different
categories ranging from clicks to posts. To respect the natural diversity
within the categorical data, embeddings map each category to a unique dense
representation within an embedded space. Since each categorical feature could
take on as many as tens of millions of different possible categories, the
embedding tables form the primary memory bottleneck during both training and
inference. We propose a novel approach for reducing the embedding size in an
end-to-end fashion by exploiting complementary partitions of the category set
to produce a unique embedding vector for each category without explicit
definition. By storing multiple smaller embedding tables based on each
complementary partition and combining embeddings from each table, we define a
unique embedding for each category at smaller memory cost. This approach may be
interpreted as using a specific fixed codebook to ensure uniqueness of each
category's representation. Our experimental results demonstrate the
effectiveness of our approach over the hashing trick for reducing the size of
the embedding tables in terms of model loss and accuracy, while retaining a
similar reduction in the number of parameters.Comment: 11 pages, 7 figures, 1 tabl
Improved Practical Matrix Sketching with Guarantees
Matrices have become essential data representations for many large-scale
problems in data analytics, and hence matrix sketching is a critical task.
Although much research has focused on improving the error/size tradeoff under
various sketching paradigms, the many forms of error bounds make these
approaches hard to compare in theory and in practice. This paper attempts to
categorize and compare most known methods under row-wise streaming updates with
provable guarantees, and then to tweak some of these methods to gain practical
improvements while retaining guarantees.
For instance, we observe that a simple heuristic iSVD, with no guarantees,
tends to outperform all known approaches in terms of size/error trade-off. We
modify the best performing method with guarantees FrequentDirections under the
size/error trade-off to match the performance of iSVD and retain its
guarantees. We also demonstrate some adversarial datasets where iSVD performs
quite poorly. In comparing techniques in the time/error trade-off, techniques
based on hashing or sampling tend to perform better. In this setting we modify
the most studied sampling regime to retain error guarantee but obtain dramatic
improvements in the time/error trade-off.
Finally, we provide easy replication of our studies on APT, a new testbed
which makes available not only code and datasets, but also a computing platform
with fixed environmental settings.Comment: 27 page
- …