887 research outputs found
A Margin-based MLE for Crowdsourced Partial Ranking
A preference order or ranking aggregated from pairwise comparison data is
commonly understood as a strict total order. However, in real-world scenarios,
some items are intrinsically ambiguous in comparisons, which may very well be
an inherent uncertainty of the data. In this case, the conventional total order
ranking can not capture such uncertainty with mere global ranking or utility
scores. In this paper, we are specifically interested in the recent surge in
crowdsourcing applications to predict partial but more accurate (i.e., making
less incorrect statements) orders rather than complete ones. To do so, we
propose a novel framework to learn some probabilistic models of partial orders
as a \emph{margin-based Maximum Likelihood Estimate} (MLE) method. We prove
that the induced MLE is a joint convex optimization problem with respect to all
the parameters, including the global ranking scores and margin parameter.
Moreover, three kinds of generalized linear models are studied, including the
basic uniform model, Bradley-Terry model, and Thurstone-Mosteller model,
equipped with some theoretical analysis on FDR and Power control for the
proposed methods. The validity of these models are supported by experiments
with both simulated and real-world datasets, which shows that the proposed
models exhibit improvements compared with traditional state-of-the-art
algorithms.Comment: 9 pages, Accepted by ACM Multimedia 2018 as a full pape
Towards autonomous decision-making: A probabilistic model for learning multi-user preferences
Information systems have revolutionized the provisioning of decision-relevant information, and decision support tools have improved human decisions in many domains. Autonomous decision- making, on the other hand, remains hampered by systems’ inability to faithfully capture human preferences. We present a computational preference model that learns unobtrusively from lim- ited data by pooling observations across like-minded users. Our model quantifies the certainty of its own predictions as input to autonomous decision-making tasks, and it infers probabilistic segments based on user choices in the process. We evaluate our model on real-world preference data collected on a commercial crowdsourcing platform, and we find that it outperforms both individual and population-level estimates in terms of predictive accuracy and the informative- ness of its certainty estimates. Our work takes an important step toward systems that act autonomously on their users’ behalf
- …