An experimental comparison of a genetic algorithm and a hill-climber for term selection

Purpose – The term selection problem for selecting query terms in information filtering and routing has been investigated using hill-climbers of various kinds, largely through the Okapi experiments in the TREC series of conferences. Although these are simple deterministic approaches which examine the effect of changing the weight of one term at a time, they have been shown to improve the retrieval effectiveness of filtering queries in these TREC experiments. Hill-climbers are, however, likely to get trapped in local optima, and the use of more sophisticated local search techniques for this problem that attempt to break out of these optima are worth investigating. To this end, we apply a genetic algorithm (GA) to the same problem. Design/Methodology/Approach – We use a standard TREC test collection from the TREC-8 filtering track, recording mean average precision and recall measures to allow comparison between the hillclimber and GA algorithms. We also vary elements of the GA, such as probability of a word being included, probability of mutation and population size in order to measure the effect of these variables. Different strategies such as Elitist and Non-Elitist methods are used, as well as Roulette Wheel and Rank selection GA algorithms. Findings – The results of tests suggest that both techniques are, on average, better than the baseline, but the implemented GA does not match the overall performance of a hill-climber. The Rank selection algorithm does better on average than the Roulette Wheel algorithm. There is no evidence in this study that varying word inclusion probability, mutation probability or Elitist method make much difference to the overall results. Small population sizes do not appear to be as effective as larger population sizes. Research limitations/implications – The evidence provided here would suggest that being stuck in a local optima for the term selection optimization problem does not appear to be detrimental to the overall success of the hill-climber. The evidence from term rank order would appear to provide extra useful evidence which hill-climbers can use efficiently and effectively to narrow the search space. Originality/Value – The paper represents the first attempt to compare hill-climbers with GAs on a problem of this type

Promoting Diversity in Academic Research Communities Through Multivariate Expert Recommendation

Expert recommendation is the process of identifying individuals who have the appropriate knowledge and skills to achieve a specific task. It has been widely used in the educational environment mainly in the hiring process, paper-reviewer assignment, and assembling conference program committees. In this research, we highlight the problem of diversity and fair representation of underrepresented groups in expertise recommendation, factors that current expertise recommendation systems rarely consider. We introduce a novel way to model experts in academia by considering demographic attributes in addition to skills. We use the h-index score to quantify skills for a researcher and we identify five demographic features with which to represent a researcher\u27s demographic profile. We highlight the importance of these features and their role in bias within the academic environment. We utilize these demographic features within an expert recommender system in academia to achieve demographic diversity and increase the exposure of the underrepresented groups using two approaches. In the first approach, we present three different algorithms for scholar recommendation: expertise-based, diversity-based, and a hybrid algorithm that uses a tuning parameter to calibrate the balance between expertise loss and diversity gain. To evaluate the ranking produced by these algorithms, we introduce a modified normalized Discounted Cumulative Gain (nDCG) version that supports multi-dimensional features, and we report diversity gain from each method. Our results show that we can achieve the best possible balance between diversity gain and expertise loss when the tuning parameter value is set around 0.4, giving nearly equal weight to both expertise and diversity. Finally, we explore diversity from the lens of the demographic parity and develop two algorithms to achieve a representative group that reflects the demographics of the recommendation pool. One is inspired by Hill Climbing, a mathematical optimization technique, wherein a solution is built gradually to the problem, and the other one is inspired by the problem of seat allocation in electoral voting systems. We evaluated these algorithms by comparing them to the hybrid algorithm from the previous approach. Our evaluation shows that both approaches provide a better diversity gain as compared to the hybrid algorithm. However, Hill Climbing Diversity is more effective when it comes to expertise savings with a statistically significant result, making it the preferred algorithm to achieve the goal of promoting diversity while maintaining expertise in an expert recommendation process

An Efficient Bandit Algorithm for Realtime Multivariate Optimization

Optimization is commonly employed to determine the content of web pages, such as to maximize conversions on landing pages or click-through rates on search engine result pages. Often the layout of these pages can be decoupled into several separate decisions. For example, the composition of a landing page may involve deciding which image to show, which wording to use, what color background to display, etc. Such optimization is a combinatorial problem over an exponentially large decision space. Randomized experiments do not scale well to this setting, and therefore, in practice, one is typically limited to optimizing a single aspect of a web page at a time. This represents a missed opportunity in both the speed of experimentation and the exploitation of possible interactions between layout decisions. Here we focus on multivariate optimization of interactive web pages. We formulate an approach where the possible interactions between different components of the page are modeled explicitly. We apply bandit methodology to explore the layout space efficiently and use hill-climbing to select optimal content in realtime. Our algorithm also extends to contextualization and personalization of layout selection. Simulation results show the suitability of our approach to large decision spaces with strong interactions between content. We further apply our algorithm to optimize a message that promotes adoption of an Amazon service. After only a single week of online optimization, we saw a 21% conversion increase compared to the median layout. Our technique is currently being deployed to optimize content across several locations at Amazon.com.Comment: KDD'17 Audience Appreciation Awar