15 research outputs found
Strongly Hierarchical Factorization Machines and ANOVA Kernel Regression
High-order parametric models that include terms for feature interactions are
applied to various data mining tasks, where ground truth depends on
interactions of features. However, with sparse data, the high- dimensional
parameters for feature interactions often face three issues: expensive
computation, difficulty in parameter estimation and lack of structure. Previous
work has proposed approaches which can partially re- solve the three issues. In
particular, models with factorized parameters (e.g. Factorization Machines) and
sparse learning algorithms (e.g. FTRL-Proximal) can tackle the first two issues
but fail to address the third. Regarding to unstructured parameters,
constraints or complicated regularization terms are applied such that
hierarchical structures can be imposed. However, these methods make the
optimization problem more challenging. In this work, we propose Strongly
Hierarchical Factorization Machines and ANOVA kernel regression where all the
three issues can be addressed without making the optimization problem more
difficult. Experimental results show the proposed models significantly
outperform the state-of-the-art in two data mining tasks: cold-start user
response time prediction and stock volatility prediction.Comment: 9 pages, to appear in SDM'1
Projective Quadratic Regression for Online Learning
This paper considers online convex optimization (OCO) problems - the
paramount framework for online learning algorithm design. The loss function of
learning task in OCO setting is based on streaming data so that OCO is a
powerful tool to model large scale applications such as online recommender
systems. Meanwhile, real-world data are usually of extreme high-dimensional due
to modern feature engineering techniques so that the quadratic regression is
impractical. Factorization Machine as well as its variants are efficient models
for capturing feature interactions with low-rank matrix model but they can't
fulfill the OCO setting due to their non-convexity. In this paper, We propose a
projective quadratic regression (PQR) model. First, it can capture the import
second-order feature information. Second, it is a convex model, so the
requirements of OCO are fulfilled and the global optimal solution can be
achieved. Moreover, existing modern online optimization methods such as Online
Gradient Descent (OGD) or Follow-The-Regularized-Leader (FTRL) can be applied
directly. In addition, by choosing a proper hyper-parameter, we show that it
has the same order of space and time complexity as the linear model and thus
can handle high-dimensional data. Experimental results demonstrate the
performance of the proposed PQR model in terms of accuracy and efficiency by
comparing with the state-of-the-art methods.Comment: AAAI 202
Learning Inconsistent Preferences with Kernel Methods
We propose a probabilistic kernel approach for preferential learning from
pairwise duelling data using Gaussian Processes. Different from previous
methods, we do not impose a total order on the item space, hence can capture
more expressive latent preferential structures such as inconsistent preferences
and clusters of comparable items. Furthermore, we prove the universality of the
proposed kernels, i.e. that the corresponding reproducing kernel Hilbert Space
(RKHS) is dense in the space of skew-symmetric preference functions. To
conclude the paper, we provide an extensive set of numerical experiments on
simulated and real-world datasets showcasing the competitiveness of our
proposed method with state-of-the-art