It Is Different When Items Are Older: Debiasing Recommendations When Selection Bias and User Preferences Are Dynamic

User interactions with recommender systems (RSs) are affected by user selection bias, e.g., users are more likely to rate popular items (popularity bias) or items that they expect to enjoy beforehand (positivity bias). Methods exist for mitigating the effects of selection bias in user ratings on the evaluation and optimization of RSs. However, these methods treat selection bias as static, despite the fact that the popularity of an item may change drastically over time and the fact that user preferences may also change over time. We focus on the age of an item and its effect on selection bias and user preferences. Our experimental analysis reveals that the rating behavior of users on the MovieLens dataset is better captured by methods that consider effects from the age of item on bias and preferences. We theoretically show that in a dynamic scenario in which both the selection bias and user preferences are dynamic, existing debiasing methods are no longer unbiased. To address this limitation, we introduce DebiAsing in the dyNamiC scEnaRio (DANCER), a novel debiasing method that extends the inverse propensity scoring debiasing method to account for dynamic selection bias and user preferences. Our experimental results indicate that DANCER improves rating prediction performance compared to debiasing methods that incorrectly assume that selection bias is static in a dynamic scenario. To the best of our knowledge, DANCER is the first debiasing method that accounts for dynamic selection bias and user preferences in RSs.Comment: WSDM 202

An IPW-based Unbiased Ranking Metric in Two-sided Markets

In modern recommendation systems, unbiased learning-to-rank (LTR) is crucial for prioritizing items from biased implicit user feedback, such as click data. Several techniques, such as Inverse Propensity Weighting (IPW), have been proposed for single-sided markets. However, less attention has been paid to two-sided markets, such as job platforms or dating services, where successful conversions require matching preferences from both users. This paper addresses the complex interaction of biases between users in two-sided markets and proposes a tailored LTR approach. We first present a formulation of feedback mechanisms in two-sided matching platforms and point out that their implicit feedback may include position bias from both user groups. On the basis of this observation, we extend the IPW estimator and propose a new estimator, named two-sided IPW, to address the position bases in two-sided markets. We prove that the proposed estimator satisfies the unbiasedness for the ground-truth ranking metric. We conducted numerical experiments on real-world two-sided platforms and demonstrated the effectiveness of our proposed method in terms of both precision and robustness. Our experiments showed that our method outperformed baselines especially when handling rare items, which are less frequently observed in the training data

Ground truth deficiencies in software engineering: when codifying the past can be counterproductive

Many software engineering tools build and evaluate their models based on historical data to support development and process decisions. These models help us answer numerous interesting questions, but have their own caveats. In a real-life setting, the objective function of human decision-makers for a given task might be influenced by a whole host of factors that stem from their cognitive biases, subverting the ideal objective function required for an optimally functioning system. Relying on this data as ground truth may give rise to systems that end up automating software engineering decisions by mimicking past sub-optimal behaviour. We illustrate this phenomenon and suggest mitigation strategies to raise awareness

Automation, Decision Making and Business to Business Pricing

In a world going towards automation, I ask whether salespeople making pricing decisions in a high human interaction environment such as business to business (B2B) retail, could be automated, and under what conditions it would be most beneficial. I propose a hybrid approach to automation that combines the expert salesperson and an artificial intelligence model of the salesperson in making pricing decisions in B2B. The hybrid approach preserves individual and organizational knowledge both by learning the expert's decision making behavior and by keeping the expert in the decision making process for decisions that require human judgment. Using sales transactions data from a B2B aluminum retailer, I create an automated version of each salesperson, that learns the salesperson's pricing policy based on her past pricing decisions. In a field experiment, I provide salespeople in the B2B retailer with their own model's price recommendations through their CRM system in real-time, and allow them to adjust their original pricing accordingly. I find that despite the loss of non-codeable information that is available to the salesperson but not to the model, providing the model's price increases profits for treated quotes by as much as 10% relative to a control condition, which translates to approximately $1.3 million in yearly profits. Using a counterfactual analysis, I also find that a hybrid pricing approach, that follows the model's pricing most of time, but defers to the salesperson's pricing when the model is missing important information is more profitable than pure automation or pure reliance on the salesperson's pricing. I find that in most cases the model's scalability and consistency lead to better pricing decisions that translate to higher profits, but when pricing uncommon products or pricing for unfamiliar clients it is best to use human judgment. I investigate different ways, including machine learning methods, to model the salesperson's behavior and to combine salespeople's expertise as reflected by their automated representations, and discuss implications for automation of tasks that involve soft skills

Relation between client opinion (Net Promoter Score) and transactional data: A Pratical Example in Retail at WORTEN

Internship Report presented as the partial requirement for obtaining a Master's degree in Statistics and Information Management, specialization in Marketing Research and CRMThis professional internship took place at Worten, in Lisbon, with a duration of 9 months in the year 2021/2022 in order to apply and consolidate, in a practical context, the theoretical knowledge acquired in the 1st and 2nd semester of the Master with guidance and supervision, with the to complete the master's degree and gain experience in the area. The main objective of this study was to try to understand customer behaviour considering their opinion given in the NPS (Net Promoter Score) process, trying to measure, classify and predict the customer's transactional behaviour in the company. Although this metric has been criticized by the academic community due to its poor predictive sales performance, NPS remains the most notorious metric in the market adopted by managers as a metric of consumer mindset. This internship report validates that NPS is a bad predictor of Sales in the long term, but a good predictor of frequency of purchase in the short term. This report also emphasizes the significance of conducting a segmented and in-depth analysis of each business area in order to identify the areas that are harming the company the most and those that may have potential churners. Finally, this report offers a comprehensive view of the company and its relationship with the NPS metric