Improving weighted least squares inference

These days, it is common practice to base inference about the coefficients in a hetoskedastic linear model on the ordinary least squares estimator in conjunction with using heteroskedasticity consistent standard errors. Even when the true form of heteroskedasticity is unknown, heteroskedasticity consistent standard errors can also used to base valid inference on a weighted least squares estimator and using such an estimator can provide large gains in efficiency over the ordinary least squares estimator. However, intervals based on asymptotic approximations with plug-in standard errors often have coverage that is below the nominal level, especially for small sample sizes. Similarly, tests can have null rejection probabilities that are above the nominal level. It is shown that under unknown hereroskedasticy, a bootstrap approximation to the sampling distribution of the weighted least squares estimator is valid, which allows for inference with improved finite-sample properties. For testing linear constraints, permutations tests are proposed which are exact when the error distribution is symmetric and is asymptotically valid otherwise. Another concern that has discouraged the use of weighting is that the weighted least squares estimator may be less efficient than the ordinary least squares estimator when the model used to estimate the unknown form of the heteroskedasticity is misspecified. To address this problem, a new estimator is proposed that is asymptotically at least as efficient as both the ordinary and the weighted least squares estimator. Simulation studies demonstrate the attractive finite-sample properties of this new estimator as well as the improvements in performance realized by bootstrap confidence intervals

Detection and Mitigation of Algorithmic Bias via Predictive Rate Parity

Recently, numerous studies have demonstrated the presence of bias in machine learning powered decision-making systems. Although most definitions of algorithmic bias have solid mathematical foundations, the corresponding bias detection techniques often lack statistical rigor, especially for non-iid data. We fill this gap in the literature by presenting a rigorous non-parametric testing procedure for bias according to Predictive Rate Parity, a commonly considered notion of algorithmic bias. We adapt traditional asymptotic results for non-parametric estimators to test for bias in the presence of dependence commonly seen in user-level data generated by technology industry applications and illustrate how these approaches can be leveraged for mitigation. We further propose modifications of this methodology to address bias measured through marginal outcome disparities in classification settings and extend notions of predictive rate parity to multi-objective models. Experimental results on real data show the efficacy of the proposed detection and mitigation methods

Improving weighted least squares inference

These days, it is common practice to base inference about the coefficients in a hetoskedastic linear model on the ordinary least squares estimator in conjunction with using heteroskedasticity consistent standard errors. Even when the true form of heteroskedasticity is unknown, heteroskedasticity consistent standard errors can also used to base valid inference on a weighted least squares estimator and using such an estimator can provide large gains in efficiency over the ordinary least squares estimator. However, intervals based on asymptotic approximations with plug-in standard errors often have coverage that is below the nominal level, especially for small sample sizes. Similarly, tests can have null rejection probabilities that are above the nominal level. In this paper, it is shown that under unknown hereroskedasticy, a bootstrap approximation to the sampling distribution of the weighted least squares estimator is valid, which allows for inference with improved finite-sample properties. For testing linear constraints, permutations tests are proposed which are exact when the error distribution is symmetric and is asymptotically valid otherwise. Another concern that has discouraged the use of weighting is that the weighted least squares estimator may be less efficient than the ordinary least squares estimator when the model used to estimate the unknown form of the heteroskedasticity is misspecified. To address this problem, a new estimator is proposed that is asymptotically at least as efficient as both the ordinary and the weighted least squares estimator. Simulation studies demonstrate the attractive finite-sample properties of this new estimator as well as the improvements in performance realized by bootstrap confidence intervals

Long-term Dynamics of Fairness Intervention in Connection Recommender Systems

Recommender system fairness has been studied from the perspectives of a variety of stakeholders including content producers, the content itself and recipients of recommendations. Regardless of which type of stakeholders are considered, most works in this area assess the efficacy of fairness intervention by evaluating a single fixed fairness criterion through the lens of a one-shot, static setting. Yet recommender systems constitute dynamical systems with feedback loops from the recommendations to the underlying population distributions which could lead to unforeseen and adverse consequences if not taken into account. In this paper, we study a connection recommender system patterned after the systems employed by web-scale social networks and analyze the long-term effects of intervening on fairness in the recommendations. We find that, although seemingly fair in aggregate, common exposure and utility parity interventions fail to mitigate amplification of biases in the long term. We theoretically characterize how certain fairness interventions impact the bias amplification dynamics in a stylized P\'{o}lya urn model.Comment: Conference on Artificial Intelligence, Ethics, and Society (AIES 2022