2 research outputs found
A Boosting Framework of Factorization Machine
Recently, Factorization Machines (FM) has become more and more popular for
recommendation systems, due to its effectiveness in finding informative
interactions between features. Usually, the weights for the interactions is
learnt as a low rank weight matrix, which is formulated as an inner product of
two low rank matrices. This low rank can help improve the generalization
ability of Factorization Machines. However, to choose the rank properly, it
usually needs to run the algorithm for many times using different ranks, which
clearly is inefficient for some large-scale datasets. To alleviate this issue,
we propose an Adaptive Boosting framework of Factorization Machines (AdaFM),
which can adaptively search for proper ranks for different datasets without
re-training. Instead of using a fixed rank for FM, the proposed algorithm will
adaptively gradually increases its rank according to its performance until the
performance does not grow, using boosting strategy. To verify the performance
of our proposed framework, we conduct an extensive set of experiments on many
real-world datasets. Encouraging empirical results shows that the proposed
algorithms are generally more effective than state-of-the-art other
Factorization Machines
RaFM: Rank-Aware Factorization Machines
Factorization machines (FM) are a popular model class to learn pairwise
interactions by a low-rank approximation. Different from existing FM-based
approaches which use a fixed rank for all features, this paper proposes a
Rank-Aware FM (RaFM) model which adopts pairwise interactions from embeddings
with different ranks. The proposed model achieves a better performance on
real-world datasets where different features have significantly varying
frequencies of occurrences. Moreover, we prove that the RaFM model can be
stored, evaluated, and trained as efficiently as one single FM, and under some
reasonable conditions it can be even significantly more efficient than FM. RaFM
improves the performance of FMs in both regression tasks and classification
tasks while incurring less computational burden, therefore also has attractive
potential in industrial applications.Comment: 9 pages, 4 figures, accepted by ICML 201