14,401 research outputs found
Deep Character-Level Click-Through Rate Prediction for Sponsored Search
Predicting the click-through rate of an advertisement is a critical component
of online advertising platforms. In sponsored search, the click-through rate
estimates the probability that a displayed advertisement is clicked by a user
after she submits a query to the search engine. Commercial search engines
typically rely on machine learning models trained with a large number of
features to make such predictions. This is inevitably requires a lot of
engineering efforts to define, compute, and select the appropriate features. In
this paper, we propose two novel approaches (one working at character level and
the other working at word level) that use deep convolutional neural networks to
predict the click-through rate of a query-advertisement pair. Specially, the
proposed architectures only consider the textual content appearing in a
query-advertisement pair as input, and produce as output a click-through rate
prediction. By comparing the character-level model with the word-level model,
we show that language representation can be learnt from scratch at character
level when trained on enough data. Through extensive experiments using billions
of query-advertisement pairs of a popular commercial search engine, we
demonstrate that both approaches significantly outperform a baseline model
built on well-selected text features and a state-of-the-art word2vec-based
approach. Finally, by combining the predictions of the deep models introduced
in this study with the prediction of the model in production of the same
commercial search engine, we significantly improve the accuracy and the
calibration of the click-through rate prediction of the production system.Comment: SIGIR2017, 10 page
Conversion Prediction Using Multi-task Conditional Attention Networks to Support the Creation of Effective Ad Creative
Accurately predicting conversions in advertisements is generally a
challenging task, because such conversions do not occur frequently. In this
paper, we propose a new framework to support creating high-performing ad
creatives, including the accurate prediction of ad creative text conversions
before delivering to the consumer. The proposed framework includes three key
ideas: multi-task learning, conditional attention, and attention highlighting.
Multi-task learning is an idea for improving the prediction accuracy of
conversion, which predicts clicks and conversions simultaneously, to solve the
difficulty of data imbalance. Furthermore, conditional attention focuses
attention of each ad creative with the consideration of its genre and target
gender, thus improving conversion prediction accuracy. Attention highlighting
visualizes important words and/or phrases based on conditional attention. We
evaluated the proposed framework with actual delivery history data (14,000
creatives displayed more than a certain number of times from Gunosy Inc.), and
confirmed that these ideas improve the prediction performance of conversions,
and visualize noteworthy words according to the creatives' attributes.Comment: 9 pages, 6 figures. Accepted at The 25th ACM SIGKDD Conference on
Knowledge Discovery and Data Mining (KDD 2019) as an applied data science
pape
Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction
Click-Through Rate prediction is an important task in recommender systems,
which aims to estimate the probability of a user to click on a given item.
Recently, many deep models have been proposed to learn low-order and high-order
feature interactions from original features. However, since useful interactions
are always sparse, it is difficult for DNN to learn them effectively under a
large number of parameters. In real scenarios, artificial features are able to
improve the performance of deep models (such as Wide & Deep Learning), but
feature engineering is expensive and requires domain knowledge, making it
impractical in different scenarios. Therefore, it is necessary to augment
feature space automatically. In this paper, We propose a novel Feature
Generation by Convolutional Neural Network (FGCNN) model with two components:
Feature Generation and Deep Classifier. Feature Generation leverages the
strength of CNN to generate local patterns and recombine them to generate new
features. Deep Classifier adopts the structure of IPNN to learn interactions
from the augmented feature space. Experimental results on three large-scale
datasets show that FGCNN significantly outperforms nine state-of-the-art
models. Moreover, when applying some state-of-the-art models as Deep
Classifier, better performance is always achieved, showing the great
compatibility of our FGCNN model. This work explores a novel direction for CTR
predictions: it is quite useful to reduce the learning difficulties of DNN by
automatically identifying important features
Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking
Machine-learned models are often described as "black boxes". In many
real-world applications however, models may have to sacrifice predictive power
in favour of human-interpretability. When this is the case, feature engineering
becomes a crucial task, which requires significant and time-consuming human
effort. Whilst some features are inherently static, representing properties
that cannot be influenced (e.g., the age of an individual), others capture
characteristics that could be adjusted (e.g., the daily amount of carbohydrates
taken). Nonetheless, once a model is learned from the data, each prediction it
makes on new instances is irreversible - assuming every instance to be a static
point located in the chosen feature space. There are many circumstances however
where it is important to understand (i) why a model outputs a certain
prediction on a given instance, (ii) which adjustable features of that instance
should be modified, and finally (iii) how to alter such a prediction when the
mutated instance is input back to the model. In this paper, we present a
technique that exploits the internals of a tree-based ensemble classifier to
offer recommendations for transforming true negative instances into positively
predicted ones. We demonstrate the validity of our approach using an online
advertising application. First, we design a Random Forest classifier that
effectively separates between two types of ads: low (negative) and high
(positive) quality ads (instances). Then, we introduce an algorithm that
provides recommendations that aim to transform a low quality ad (negative
instance) into a high quality one (positive instance). Finally, we evaluate our
approach on a subset of the active inventory of a large ad network, Yahoo
Gemini.Comment: 10 pages, KDD 201
A convolutional neural-network model of human cochlear mechanics and filter tuning for real-time applications
Auditory models are commonly used as feature extractors for automatic
speech-recognition systems or as front-ends for robotics, machine-hearing and
hearing-aid applications. Although auditory models can capture the biophysical
and nonlinear properties of human hearing in great detail, these biophysical
models are computationally expensive and cannot be used in real-time
applications. We present a hybrid approach where convolutional neural networks
are combined with computational neuroscience to yield a real-time end-to-end
model for human cochlear mechanics, including level-dependent filter tuning
(CoNNear). The CoNNear model was trained on acoustic speech material and its
performance and applicability were evaluated using (unseen) sound stimuli
commonly employed in cochlear mechanics research. The CoNNear model accurately
simulates human cochlear frequency selectivity and its dependence on sound
intensity, an essential quality for robust speech intelligibility at negative
speech-to-background-noise ratios. The CoNNear architecture is based on
parallel and differentiable computations and has the power to achieve real-time
human performance. These unique CoNNear features will enable the next
generation of human-like machine-hearing applications
Product-based Neural Networks for User Response Prediction
Predicting user responses, such as clicks and conversions, is of great
importance and has found its usage in many Web applications including
recommender systems, web search and online advertising. The data in those
applications is mostly categorical and contains multiple fields; a typical
representation is to transform it into a high-dimensional sparse binary feature
representation via one-hot encoding. Facing with the extreme sparsity,
traditional models may limit their capacity of mining shallow patterns from the
data, i.e. low-order feature combinations. Deep models like deep neural
networks, on the other hand, cannot be directly applied for the
high-dimensional input because of the huge feature space. In this paper, we
propose a Product-based Neural Networks (PNN) with an embedding layer to learn
a distributed representation of the categorical data, a product layer to
capture interactive patterns between inter-field categories, and further fully
connected layers to explore high-order feature interactions. Our experimental
results on two large-scale real-world ad click datasets demonstrate that PNNs
consistently outperform the state-of-the-art models on various metrics.Comment: 6 pages, 5 figures, ICDM201
- …