3,090 research outputs found
Applying Deep Learning To Airbnb Search
The application to search ranking is one of the biggest machine learning
success stories at Airbnb. Much of the initial gains were driven by a gradient
boosted decision tree model. The gains, however, plateaued over time. This
paper discusses the work done in applying neural networks in an attempt to
break out of that plateau. We present our perspective not with the intention of
pushing the frontier of new modeling techniques. Instead, ours is a story of
the elements we found useful in applying neural networks to a real life
product. Deep learning was steep learning for us. To other teams embarking on
similar journeys, we hope an account of our struggles and triumphs will provide
some useful pointers. Bon voyage!Comment: 8 page
CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks
Data quality affects machine learning (ML) model performances, and data
scientists spend considerable amount of time on data cleaning before model
training. However, to date, there does not exist a rigorous study on how
exactly cleaning affects ML -- ML community usually focuses on developing ML
algorithms that are robust to some particular noise types of certain
distributions, while database (DB) community has been mostly studying the
problem of data cleaning alone without considering how data is consumed by
downstream ML analytics. We propose a CleanML study that systematically
investigates the impact of data cleaning on ML classification tasks. The
open-source and extensible CleanML study currently includes 14 real-world
datasets with real errors, five common error types, seven different ML models,
and multiple cleaning algorithms for each error type (including both commonly
used algorithms in practice as well as state-of-the-art solutions in academic
literature). We control the randomness in ML experiments using statistical
hypothesis testing, and we also control false discovery rate in our experiments
using the Benjamini-Yekutieli (BY) procedure. We analyze the results in a
systematic way to derive many interesting and nontrivial observations. We also
put forward multiple research directions for researchers.Comment: published in ICDE 202
Assessing and Remedying Coverage for a Given Dataset
Data analysis impacts virtually every aspect of our society today. Often,
this analysis is performed on an existing dataset, possibly collected through a
process that the data scientists had limited control over. The existing data
analyzed may not include the complete universe, but it is expected to cover the
diversity of items in the universe. Lack of adequate coverage in the dataset
can result in undesirable outcomes such as biased decisions and algorithmic
racism, as well as creating vulnerabilities such as opening up room for
adversarial attacks.
In this paper, we assess the coverage of a given dataset over multiple
categorical attributes. We first provide efficient techniques for traversing
the combinatorial explosion of value combinations to identify any regions of
attribute space not adequately covered by the data. Then, we determine the
least amount of additional data that must be obtained to resolve this lack of
adequate coverage. We confirm the value of our proposal through both
theoretical analyses and comprehensive experiments on real data.Comment: in ICDE 201
Optimizing Airbnb Search Journey with Multi-task Learning
At Airbnb, an online marketplace for stays and experiences, guests often
spend weeks exploring and comparing multiple items before making a final
reservation request. Each reservation request may then potentially be rejected
or cancelled by the host prior to check-in. The long and exploratory nature of
the search journey, as well as the need to balance both guest and host
preferences, present unique challenges for Airbnb search ranking. In this
paper, we present Journey Ranker, a new multi-task deep learning model
architecture that addresses these challenges. Journey Ranker leverages
intermediate guest actions as milestones, both positive and negative, to better
progress the guest towards a successful booking. It also uses contextual
information such as guest state and search query to balance guest and host
preferences. Its modular and extensible design, consisting of four modules with
clear separation of concerns, allows for easy application to use cases beyond
the Airbnb search ranking context. We conducted offline and online testing of
the Journey Ranker and successfully deployed it in production to four different
Airbnb products with significant business metrics improvements.Comment: Search Ranking, Recommender Systems, User Search Journey, Multi-task
learning, Two-sided marketplac
Sparse tree-based initialization for neural networks
Dedicated neural network (NN) architectures have been designed to handle
specific data types (such as CNN for images or RNN for text), which ranks them
among state-of-the-art methods for dealing with these data. Unfortunately, no
architecture has been found for dealing with tabular data yet, for which tree
ensemble methods (tree boosting, random forests) usually show the best
predictive performances. In this work, we propose a new sparse initialization
technique for (potentially deep) multilayer perceptrons (MLP): we first train a
tree-based procedure to detect feature interactions and use the resulting
information to initialize the network, which is subsequently trained via
standard stochastic gradient strategies. Numerical experiments on several
tabular data sets show that this new, simple and easy-to-use method is a solid
concurrent, both in terms of generalization capacity and computation time, to
default MLP initialization and even to existing complex deep learning
solutions. In fact, this wise MLP initialization raises the resulting NN
methods to the level of a valid competitor to gradient boosting when dealing
with tabular data. Besides, such initializations are able to preserve the
sparsity of weights introduced in the first layers of the network through
training. This fact suggests that this new initializer operates an implicit
regularization during the NN training, and emphasizes that the first layers act
as a sparse feature extractor (as for convolutional layers in CNN)
- …