Fair Ranking under Disparate Uncertainty

Ranking is a ubiquitous method for focusing the attention of human evaluators on a manageable subset of options. Its use ranges from surfacing potentially relevant products on an e-commerce site to prioritizing college applications for human review. While ranking can make human evaluation far more effective by focusing attention on the most promising options, we argue that it can introduce unfairness if the uncertainty of the underlying relevance model differs between groups of options. Unfortunately, such disparity in uncertainty appears widespread, since the relevance estimates for minority groups tend to have higher uncertainty due to a lack of data or appropriate features. To overcome this fairness issue, we propose Equal-Opportunity Ranking (EOR) as a new fairness criterion for ranking that provably corrects for the disparity in uncertainty between groups. Furthermore, we present a practical algorithm for computing EOR rankings in time $O(n \log(n))$ and prove its close approximation guarantee to the globally optimal solution. In a comprehensive empirical evaluation on synthetic data, a US Census dataset, and a real-world case study of Amazon search queries, we find that the algorithm reliably guarantees EOR fairness while providing effective rankings.Comment: A version of this paper was accepted as Spotlight (Oral) at UAI workshop on Epistemic in AI, 202

Ranking and Selection under Input Uncertainty: Fixed Confidence and Fixed Budget

In stochastic simulation, input uncertainty (IU) is caused by the error in estimating the input distributions using finite real-world data. When it comes to simulation-based Ranking and Selection (R&S), ignoring IU could lead to the failure of many existing selection procedures. In this paper, we study R&S under IU by allowing the possibility of acquiring additional data. Two classical R&S formulations are extended to account for IU: (i) for fixed confidence, we consider when data arrive sequentially so that IU can be reduced over time; (ii) for fixed budget, a joint budget is assumed to be available for both collecting input data and running simulations. New procedures are proposed for each formulation using the frameworks of Sequential Elimination and Optimal Computing Budget Allocation, with theoretical guarantees provided accordingly (e.g., upper bound on the expected running time and finite-sample bound on the probability of false selection). Numerical results demonstrate the effectiveness of our procedures through a multi-stage production-inventory problem

The Performance of Compliance Measures and Instruments for Nitrate Nonpoint Pollution Control Under Uncertainty and Alternative Agricultural Commodity Policy Regimes

Following Weitzman (1974), there is ample theoretical literature indicating that choice of pollution control instruments under conditions of uncertainty will affect the expected net benefits that can be realized from environmental protection. However, there is little empirical research on the ex ante efficiency of alternative instruments for controlling water, or other types of pollution, under uncertainty about costs and benefits. Using a simulation model that incorporates various sources of uncertainty, the ex ante efficiency of price and quantity controls applied to two alternative policy targets, fertilizer application rates and estimated excess nitrogen applications, are examined under varying assumptions about agricultural income support policies. Results indicate price instruments outperform quantity instruments. A tax on excess nitrogen substantially outperforms a fertilizer tax in the scenario with support programs, while the ranking is reversed in the scenario without support programs.Environmental Economics and Policy,

A Generalized Hybrid Approach to Controlling Emissions

This paper examines the optimal instrument choice to control emissions under uncertainty. A hybrid regulation mechanism is developed that contains cap-and-trade, emissions taxes and socalled safety valves as special cases. This makes it possible to examine optimal policy choice and the resulting efficiency losses for each instrument. It is shown that the hybrid regulation mechanism in efficiency terms is weakly superior to the other instruments. The model is also used to study optimal response to non-optimal policy implementations.Emissions tax; Emissions trading; Safety valve; Ranking; Uncertainty

PREDICTING CONSUMER INFORMATION SEARCH BENEFITS FOR PERSONALIZED ONLINE PRODUCT RANKING: A CONFIDENCE-BASED APPROACH

Product ranking mechanism is an important service for e-commerce that facilitates consumers’ decision-making process. This paper studies online product ranking under uncertainty. Different from previous studies that generally rank products merely based on predicted ratings, a new personalized product ranking method is proposed based on estimating consumer information search benefits and taking prediction uncertainty and confidence into consideration. Experiments using real data of movie ratings illustrate that the proposed method is advantageous over traditional point estimation methods, thus may help enhance customers’ satisfaction with the decision-making process and choices through saving their time and efforts