Rank Aggregation via Heterogeneous Thurstone Preference Models

We propose the Heterogeneous Thurstone Model (HTM) for aggregating ranked data, which can take the accuracy levels of different users into account. By allowing different noise distributions, the proposed HTM model maintains the generality of Thurstone's original framework, and as such, also extends the Bradley-Terry-Luce (BTL) model for pairwise comparisons to heterogeneous populations of users. Under this framework, we also propose a rank aggregation algorithm based on alternating gradient descent to estimate the underlying item scores and accuracy levels of different users simultaneously from noisy pairwise comparisons. We theoretically prove that the proposed algorithm converges linearly up to a statistical error which matches that of the state-of-the-art method for the single-user BTL model. We evaluate the proposed HTM model and algorithm on both synthetic and real data, demonstrating that it outperforms existing methods.Comment: 36 pages, 2 figures, 8 tables. In AAAI 202

A Margin-based MLE for Crowdsourced Partial Ranking

A preference order or ranking aggregated from pairwise comparison data is commonly understood as a strict total order. However, in real-world scenarios, some items are intrinsically ambiguous in comparisons, which may very well be an inherent uncertainty of the data. In this case, the conventional total order ranking can not capture such uncertainty with mere global ranking or utility scores. In this paper, we are specifically interested in the recent surge in crowdsourcing applications to predict partial but more accurate (i.e., making less incorrect statements) orders rather than complete ones. To do so, we propose a novel framework to learn some probabilistic models of partial orders as a \emph{margin-based Maximum Likelihood Estimate} (MLE) method. We prove that the induced MLE is a joint convex optimization problem with respect to all the parameters, including the global ranking scores and margin parameter. Moreover, three kinds of generalized linear models are studied, including the basic uniform model, Bradley-Terry model, and Thurstone-Mosteller model, equipped with some theoretical analysis on FDR and Power control for the proposed methods. The validity of these models are supported by experiments with both simulated and real-world datasets, which shows that the proposed models exhibit improvements compared with traditional state-of-the-art algorithms.Comment: 9 pages, Accepted by ACM Multimedia 2018 as a full pape

Static and Dynamic BART for Rank-Order Data

Ranking lists are often provided at regular time intervals by one or multiple rankers in a range of applications, including sports, marketing, and politics. Most popular methods for rank-order data postulate a linear specification for the latent scores, which determine the observed ranks, and ignore the temporal dependence of the ranking lists. To address these issues, novel nonparametric static (ROBART) and autoregressive (ARROBART) models are introduced, with latent scores defined as nonlinear Bayesian additive regression tree functions of covariates. To make inferences in the dynamic ARROBART model, closed-form filtering, predictive, and smoothing distributions for the latent time-varying scores are derived. These results are applied in a Gibbs sampler with data augmentation for posterior inference. The proposed methods are shown to outperform existing competitors in simulation studies, and the advantages of the dynamic model are demonstrated by forecasts of weekly pollster rankings of NCAA football teams.Comment: The Supplementary Material is available upon request to the author

Prioritizing Offshore Vendor Selection Criteria for the North American Geospatial Industry

The U.S. market for geospatial services totaled US $2.2 billion in 2010, representing 50% of the global market. Data-processing firms subcontract labor-intensive portions of data services to offshore providers in South and East Asia and Eastern Europe. In general, half of all offshore contracts fail within the first 5 years because one or more parties consider the relationship unsuccessful. Despite the high failure rates, no study has examined the offshore vendor selection process in the geospatial industry. The purpose of this study was to determine the list of key offshore vendor selection criteria and the efficacy of the analytic hierarchy process (AHP) for ranking the criteria that North American geospatial companies consider in the offshore vendor selection process. After the selection of the initial list of factors from the literature and their validation in a pilot study, a final survey instrument was developed and administered to 15 subject matter experts (SMEs) in North America. The SMEs expressed their preferences for one criterion over another by pairwise comparisons, which served as input to the AHP procedure. The results showed that the quality of deliverables was the top ranked (out of 26) factors, instead of the price, which ranked third. Similarly, SMEs considered social and environmental consciousness on the vendor side as irrelevant. More importantly, the findings indicated that the structured AHP process provides a useful and effective methodology whose application may considerably improve the quality of the overall vendor selection process. Last, improved and stabilized business relationships leading to predictable budgets might catalyze social change, supporting stable employment. Consumers could benefit from derivative improvements in product quality and pricing

Valuing agricultural externalities in Canterbury rivers and streams

Water quality and quantity concerns in Canterbury are intrinsically related to agriculture. Monetary values for impacts on streams and rivers is lacking in policy debate. This paper employs choice modelling to estimate values of three impacts on rivers and streams in Canterbury associated with agriculture: health risks of E coli from animal waste, ecological effects of excess nutrients, and low-flow impacts of irrigation. This study provides a valuation of outcomes for public policy implemented in Canterbury such as The Dairy and Clean Streams Accord, Living Streams, and The Restorative Programme for Lowland Streams.non-market-valuation, choice experiment, agricultural externalities, New Zealand, Agricultural and Food Policy, Environmental Economics and Policy, Food Consumption/Nutrition/Food Safety, Health Economics and Policy,