Counterfactual Estimation and Optimization of Click Metrics for Search Engines

Optimizing an interactive system against a predefined online metric is particularly challenging, when the metric is computed from user feedback such as clicks and payments. The key challenge is the counterfactual nature: in the case of Web search, any change to a component of the search engine may result in a different search result page for the same query, but we normally cannot infer reliably from search log how users would react to the new result page. Consequently, it appears impossible to accurately estimate online metrics that depend on user feedback, unless the new engine is run to serve users and compared with a baseline in an A/B test. This approach, while valid and successful, is unfortunately expensive and time-consuming. In this paper, we propose to address this problem using causal inference techniques, under the contextual-bandit framework. This approach effectively allows one to run (potentially infinitely) many A/B tests offline from search log, making it possible to estimate and optimize online metrics quickly and inexpensively. Focusing on an important component in a commercial search engine, we show how these ideas can be instantiated and applied, and obtain very promising results that suggest the wide applicability of these techniques

Measures to Evaluate the Superiority of a Search Engine

Main objective of a search engine is to return relevant results according to user query in less time. Evaluation metrics are used to measure the superiority of a search engine in terms of quality. This is a review paper presenting a summary of different metrics used for evaluation of a search engine in terms of effectiveness, efficiency and relevancy

How Will Online Affiliate Marketing Networks Impact Search Engine Rankings?

In online affiliate marketing networks advertising web sites offer their affiliates revenues based on provided web site traffic and associated leads and sales. Advertising web sites can have a network of thousands of affiliates providing them with web site traffic through hyperlinks on their web sites. Search engines such as Google, MSN, and Yahoo, consider hyperlinks as a proof of quality and/or reliability of the linked web sites, and therefore use them to determine the relevance of web sites with regard to search queries. In this research we investigate the potential impact of online affiliate marketing networks on the ranking of advertisers’ web sites in search results. This article empirically explores how seven different affiliate marketing networks affect the rankings of the advertising web sites within web search engines. The field study followed intensively seven online affiliate marketing networks for twelve weeks after their launch. The results indicate that newly started affiliate networks effectively improve the rankings of advertising web sites in search engine results. Also, it was found that the effects of affiliate marketing networks on search engine rankings were smaller for advertising web sites operating in highly competitive markets. Another finding was that a growth in visitors coming from search engines was present as a result of the improvement of search engine rankings. Finally, the results indicate that cost-benefit metrics associated with affiliate marketing programs, such as the average marketing cost will decrease when the positive effects of affiliate marketing on search engine rankings are taken into account.Advertising;Hyperlinks;Online Affiliate Marketing Networks;Search Engine Rankings

A Review of Elastic Search: Performance Metrics and challenges

The most important aspect of a search engine is the search. Elastic search is a highly scalable search engine that stores data in a structure, optimized for language based searches. When it comes to using Elastic search, there are lots of metrics engendered. By using Elastic search to index millions of code repositories as well as indexing critical event data, you can satisfy the search needs of millions of users while instantaneously providing strategic operational visions that help you iteratively improve customer service. In this paper we are going to study about Elastic searchperformance metrics to watch, important Elastic search challenges, and how to deal with them. This should be helpful to anyone new to Elastic search, and also to experienced users who want a quick start into performance monitoring of Elastic search

On automatic testing of web search engines

Web search engines are very important because they are the means by which people retrieve information from the World Wide Web. However, testing these web search engines is difficult because there are no test oracles, so this research proposes seven new metrics based on the idea of metamorphic relations to alleviate the oracle problem in search engine testing. Using these metrics, our method can test search engines automatically in the absence of an ideal oracle. Using this method, we further conduct large-scale empirical studies to investigate and compare the qualities of four major search engines, namely, Google (www.google.com), Baidu (www.baidu.com), Bing (www.bing.com), and Chinese Bing (www.bing.com.cn). Our empirical studies involve more than 50 million queries sent to the search engines across 9 months, and about 300 GB data collected from the search engine responses. It is found that different search engines have significantly different performance and that the nature of the query terms can have a significant impact on the performance of the search engines. These empirical study results demonstrate that our method can effectively alleviate the oracle problem in search engine testing, and can help both developers and users to obtain a better understanding of the search engine behaviour under different operational profiles