The wisdom of collective grading and the effects of epistemic and semantic diversity

A computer simulation is used to study collective judgements that an expert panel reaches on the basis of qualitative probability judgements contributed by individual members. The simulated panel displays a strong and robust crowd wisdom effect. The panel's performance is better when members contribute precise probability estimates instead of qualitative judgements, but not by much. Surprisingly, it doesn't always hurt for panel members to interpret the probability expressions differently. Indeed, coordinating their understandings can be much worse

Beyond the worst case: Distortion in impartial culture electorate

{\em Distortion} is a well-established notion for quantifying the loss of social welfare that may occur in voting. As voting rules take as input only ordinal information, they are essentially forced to neglect the exact values the agents have for the alternatives. Thus, in worst-case electorates, voting rules may return low social welfare alternatives and have high distortion. Accompanying voting rules with a small number of cardinal queries per agent may reduce distortion considerably. To explore distortion beyond worst-case conditions, we introduce a simple stochastic model, according to which the values the agents have for the alternatives are drawn independently from a common probability distribution. This gives rise to so-called {\em impartial culture electorates}. We refine the definition of distortion so that it is suitable for this stochastic setting and show that, rather surprisingly, all voting rules have high distortion {\em on average}. On the positive side, for the fundamental case where the agents have random {\em binary} values for the alternatives, we present a mechanism that achieves approximately optimal average distortion by making a {\em single} cardinal query per agent. This enables us to obtain slightly suboptimal average distortion bounds for general distributions using a simple randomized mechanism that makes one query per agent. We complement these results by presenting new tradeoffs between the distortion and the number of queries per agent in the traditional worst-case setting.Comment: 27 pages, 2 figure

Interpreting the will of the people: a positive analysis of ordinal preference aggregation

We investigate how individuals think groups should aggregate members’ ordinal preferences – that is, how they interpret “the will of the people.” In an experiment, we elicit revealed attitudes toward ordinal preference aggregation and classify subjects according to the rules they apparently deploy. Majoritarianism is rare. Instead, people employ rules that place greater weight on compromise options. The classification’s fit is excellent, and clustering analysis reveals that it does not omit important rules. We ask whether rules are stable across domains, whether people impute cardinal utility from ordinal ranks, and whether attitudes toward aggregation differ across countries with divergent traditions