22 research outputs found
Network On Network for Tabular Data Classification in Real-world Applications
Tabular data is the most common data format adopted by our customers ranging
from retail, finance to E-commerce, and tabular data classification plays an
essential role to their businesses. In this paper, we present Network On
Network (NON), a practical tabular data classification model based on deep
neural network to provide accurate predictions. Various deep methods have been
proposed and promising progress has been made. However, most of them use
operations like neural network and factorization machines to fuse the
embeddings of different features directly, and linearly combine the outputs of
those operations to get the final prediction. As a result, the intra-field
information and the non-linear interactions between those operations (e.g.
neural network and factorization machines) are ignored. Intra-field information
is the information that features inside each field belong to the same field.
NON is proposed to take full advantage of intra-field information and
non-linear interactions. It consists of three components: field-wise network at
the bottom to capture the intra-field information, across field network in the
middle to choose suitable operations data-drivenly, and operation fusion
network on the top to fuse outputs of the chosen operations deeply. Extensive
experiments on six real-world datasets demonstrate NON can outperform the
state-of-the-art models significantly. Furthermore, both qualitative and
quantitative study of the features in the embedding space show NON can capture
intra-field information effectively
Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction
Click-Through Rate prediction is an important task in recommender systems,
which aims to estimate the probability of a user to click on a given item.
Recently, many deep models have been proposed to learn low-order and high-order
feature interactions from original features. However, since useful interactions
are always sparse, it is difficult for DNN to learn them effectively under a
large number of parameters. In real scenarios, artificial features are able to
improve the performance of deep models (such as Wide & Deep Learning), but
feature engineering is expensive and requires domain knowledge, making it
impractical in different scenarios. Therefore, it is necessary to augment
feature space automatically. In this paper, We propose a novel Feature
Generation by Convolutional Neural Network (FGCNN) model with two components:
Feature Generation and Deep Classifier. Feature Generation leverages the
strength of CNN to generate local patterns and recombine them to generate new
features. Deep Classifier adopts the structure of IPNN to learn interactions
from the augmented feature space. Experimental results on three large-scale
datasets show that FGCNN significantly outperforms nine state-of-the-art
models. Moreover, when applying some state-of-the-art models as Deep
Classifier, better performance is always achieved, showing the great
compatibility of our FGCNN model. This work explores a novel direction for CTR
predictions: it is quite useful to reduce the learning difficulties of DNN by
automatically identifying important features