Unbiased Learning to Rank with Unbiased Propensity Estimation

Learning to rank with biased click data is a well-known challenge. A variety of methods has been explored to debias click data for learning to rank such as click models, result interleaving and, more recently, the unbiased learning-to-rank framework based on inverse propensity weighting. Despite their differences, most existing studies separate the estimation of click bias (namely the \textit{propensity model}) from the learning of ranking algorithms. To estimate click propensities, they either conduct online result randomization, which can negatively affect the user experience, or offline parameter estimation, which has special requirements for click data and is optimized for objectives (e.g. click likelihood) that are not directly related to the ranking performance of the system. In this work, we address those problems by unifying the learning of propensity models and ranking models. We find that the problem of estimating a propensity model from click data is a dual problem of unbiased learning to rank. Based on this observation, we propose a Dual Learning Algorithm (DLA) that jointly learns an unbiased ranker and an \textit{unbiased propensity model}. DLA is an automatic unbiased learning-to-rank framework as it directly learns unbiased ranking models from biased click data without any preprocessing. It can adapt to the change of bias distributions and is applicable to online learning. Our empirical experiments with synthetic and real-world data show that the models trained with DLA significantly outperformed the unbiased learning-to-rank algorithms based on result randomization and the models trained with relevance signals extracted by click models

Do Organic Results Help or Hurt Sponsored Search Performance

We study the impact of changes in the competitors’ listings in organic search results on the performance of sponsored search advertisements. Using data from an online retailer’s keyword advertising campaign, we measure the impact of organic competition on both click-through rate and conversion rate of sponsored search advertisements. We find that an increase in organic competition leads to a decrease in the click performance of sponsored advertisements. However, organic competition helps the conversion performance of sponsored ads and leads to higher revenue. We also find that organic competition has a higher negative effect on click performance than does sponsored competition. Our results inform advertisers on how the presence of organic results influences the performance of their sponsored advertisements. Specifically, we show that organic competition acts as a substitute for clicks, but has a complementary effect on the conversion performance

Essays in Modeling the Consumer Choice Process

In this dissertation, I utilize and develop empirical tools to help academics and practitioners model the consumer\u27s choice process. This collection of three essays strives to answer three main research questions in this theme. In the first paper, I ask: how is the consumer\u27s purchase decision impacted by the search for general product-category information prior to search for their match with a retailer or manufacturer ( sellers )? This paper studies the impact of informational organic keyword search results on the performance of sponsored search advertising. We show that, even though advertisers can target consumers who have specific needs and preferences, for many consumers this is not a sufficient condition for search advertising to work. By allowing consumers to access content that satisfies their information requirements, informational organic results can allow consumers to learn about the product category prior to making their purchase decision. We develop a model characterize the situation in which consumers can search for general information about the product category as well as for information about the individual sellers\u27 offerings. We estimate this model using a unique dataset of search advertising in which commercial websites are restricted in the organic listing, allowing us to identify consumer clicks as informational (from organic links) or purchase oriented (from sponsored links). With the estimation results, we show that consumer welfare is improved by 29%, while advertisers generate 19% more sales, and search engines obtain 18% more paid clicks, as compared to the scenario without informational links. We conduct counterfactuals and find that consumers, advertisers, and the search engine are significantly better off when the search engine provides free general information about the product. When the search engine provides information about the advertisers\u27 specific offerings, however, there are fewer paid clicks and advertisers at high ad positions will obtain lower sales. We further investigate the implications on the equilibrium advertiser bidding strategy. Results show that advertiser bids will remain constant in the former scenario. When the search engine provides advertiser information, advertisers will increase their bids because of the increased conversion rate; however, the search engine still loses revenue due to the decreased paid clicks. The findings shed important managerial insights on how to improve the effectiveness of search advertising. In the second paper, I ask: how is the consumer\u27s search for information, during their choice process and in an advertising context, influenced by the signaling theory of advertising? Using a dataset of travel-related keywords obtained from a search engine, we test to what extent consumers are searching and advertisers are bidding in accordance with the signaling theory of advertising in literature. We find significant evidence that consumers are more likely to click on advertisers at higher positions because they infer that such advertisers are more likely to match with their needs. Further, consumers are more likely to find a match with advertisers who have paid more for higher positions. We also find strong evidence that advertisers increase their bids when there is an improvement in the likelihood that their offerings match with consumers\u27 needs, and the improvement cannot be readily observed by consumers prior to searching advertisers\u27 websites. These results are consistent with the predictions from the signaling theory. We test several alternative explanations and show that they cannot fully explain the results. Furthermore, through an extension we find that advertisers can generate more clicks when competing against advertisers with higher match value. We offer an explanation for this finding based on the signaling theory. In the third paper, I ask: can we model the consumer\u27s choice of brand as a sequential elimination of alternatives based on shared or unique aspects while incorporating continuous variables, such as price? With aggregate scanner data, marketing researchers typically estimate the mixed logit model, which accounts for non-IIA substitution patterns among brands, which arise due to similarity and dominance effects in demand. Using numerical examples and analytical illustrations, this research shows that the mixed logit model, which is widely believed to be a highly flexible characterization of brand switching behavior, is not well designed to handle non-IIA substitution patterns. The probit allows only for pair-wise inter-brand similarities, and ignores third-order or higher dependencies. In the presence of similarity and dominance effects, the mixed logit model and the probit model yield systematically distorted marketing mix elasticities. This limits the usefulness of mixed logit and probit for marketing decision-making. We propose a more flexible demand model that is an extension of the elimination-by-aspects (EBA) model (Tversky 1972a, 1972b) to handle marketing variables. The model vastly expands the domain of applicability of the EBA model to aggregate scanner data. Using an analytical closed-form that nests the traditional logit model as a special case, the EBA demand model is estimated with marketing variables from aggregate scanner data in 9 different product categories. It is compared to the mixed logit and probit models on the same datasets. In terms of multiple fit and predictive metrics (LL, BIC, MSE, MAD), the EBA model outperforms the mixed logit and the probit in a majority of categories in terms of both in-sample fit and holdout predictions. The results show significant differences in the estimated price elasticity matrices between the EBA model and the comparison models. In addition, a simulation shows that the retailer can improve gross profits up to 34.4% from pricing based on the EBA model rather than the mixed logit model. Finally, the results suggest that empirical IO researchers, who routinely use mixed logit models as inputs to oligopolistic pricing models, should consider the EBA demand model as the appropriate model of demand for differentiated products

Media, Aggregators, and the Link Economy: Strategic Hyperlink Formation in Content Networks

A defining property of the World Wide Web is a content site's ability to place virtually costless hyperlinks to third-party content as a substitute or complement to its own content. Costless hyperlinking has enabled new types of players, usually referred to as content aggregators, to successfully enter content ecosystems, attracting traffic and revenue by hosting links to the content of others. This, in turn, has sparked a heated controversy between content creators and aggregators regarding the legitimacy and costs/benefits of uninhibited free linking. To our knowledge, this work is the first to model the complex interplay between content and links in settings where a set of sites compete for traffic. We develop a series of analytical models that distill how hyperlinking affects the (a) incentives of content nodes to produce quality content versus link to third-party content, (b) profits of the various stakeholders, (c) average quality of content that becomes available to consumers, and (d) impact of content aggregators. Our results provide a nuanced view of the so-called link economy, highlighting both the beneficial consequences and the drawbacks of free hyperlinks for content creators and consumers.This paper was accepted by Lorin Hitt, information systems