Scoring rules for judgment aggregation

This paper studies a class of judgment aggregation rules, to be called 'scoring rules' after their famous counterpart in preference aggregation theory. A scoring rule delivers the collective judgments which reach the highest total 'score' across the individuals, subject to the judgments having to be rational. Depending on how we define 'scores', we obtain several (old and new) solutions to the judgment aggregation problem, such as distance-based aggregation, premise- and conclusion-based aggregation, truth-tracking rules, and a generalization of Borda rule to judgment aggregation. Scoring rules are shown to generalize the classical scoring rules of preference aggregation theory

Scoring rules for judgment aggregation

This paper studies a class of judgment aggregation rules, to be called `scoring rules' after their famous counterpart in preference aggregation theory. A scoring rule delivers the collective judgments which reach the highest total `score' across the individuals, subject to the judgments having to be rational. Depending on how we define `scores', we obtain several (old and new) solutions to the judgment aggregation problem,such as distance-based aggregation, premise- and conclusion-based aggregation, truth-tracking rules, and a Borda-type rule. Scoring rules are shown to generalize the classical scoring rules of preference aggregation theory

Scoring rules for judgment aggregation

This paper studies a class of judgment aggregation rules, to be called `scoring rules' after their famous counterpart in preference aggregation theory. A scoring rule delivers the collective judgments which reach the highest total `score' across the individuals, subject to the judgments having to be rational. Depending on how we define `scores', we obtain several (old and new) solutions to the judgment aggregation problem,such as distance-based aggregation, premise- and conclusion-based aggregation, truth-tracking rules, and a Borda-type rule. Scoring rules are shown to generalize the classical scoring rules of preference aggregation theory

Scoring rules for judgment aggregation

This paper studies a class of judgment aggregation rules, to be called `scoring rules' after their famous counterpart in preference aggregation theory. A scoring rule delivers the collective judgments which reach the highest total `score' across the individuals, subject to the judgments having to be rational. Depending on how we define `scores', we obtain several (old and new) solutions to the judgment aggregation problem,such as distance-based aggregation, premise- and conclusion-based aggregation, truth-tracking rules, and a Borda-type rule. Scoring rules are shown to generalize the classical scoring rules of preference aggregation theory

Judgment aggregation in search for the truth

We analyze the problem of aggregating judgments over multiple issues from the perspective of whether aggregate judgments manage to efficiently use all voters' private information. While new in judgment aggregation theory, this perspective is familiar in a different body of literature about voting between two alternatives where voters' disagreements stem from conflicts of information rather than of interest. Combining the two bodies of literature, we consider a simple judgment aggregation problem and model the private information underlying voters' judgments. Assuming that voters share a preference for true collective judgments, we analyze the resulting strategic incentives and determine which voting rules efficiently use all private information. We find that in certain, but not all cases a quota rule should be used, which decides on each issue according to whether the proportion of ‘yes’ votes exceeds a particular quota

Challenging the Majority Rule in Matters of Truth

The majority rule has caught much attention in recent debate about the aggregation of judgments. But its role in finding the truth is limited. A majority of expert judgments is not necessarily authoritative, even if all experts are equally competent, if they make their judgments independently of each other, and if all the judgments are based on the same source of (good) evidence. In this paper I demonstrate this limitation by presenting a simple counterexample and a related general result. I pave the way for this argument by introducing a Bayesian model of evidence and expert judgment in order to give a precise account of the basic problem

Complex Collective Decisions and the Probability of Collective Inconsistencies

Many groups are required to make collective decisions over multiple interconnected propositions. The "doctrinal paradox" or "discursive dilemma" shows that propostionwise majority voting can lead to inconsistent collective outcomes, even when the judgments of individual group members are consistent. How likely is the occurence of this paradox? This paper develops a simple model for determining the probability of the paradox's occurrence, given various assumptions about the probability of different individual judgments. Several convergence results will be proved, identifying conditions under which the probability of the paradox's occurrence converges to certainty as the number of individuals increases, and conditions under which that probability vanishes. The present model will also be used for assessing the "truth-tracking" performance of two escape-routes from the paradox, the premise- and conclusion-based procedures. Finally, the results on the probability of the doctrinal paradox will be compared with existing results on the probability of Condorcet's paradox of cyclical preferences. It will be suggested that the doctrinal paradox is more likely to occur than Condorcet's paradox.

Experimental philosophy and moral responsibility

Can experimental philosophy help us answer central questions about the nature of moral responsibility, such as the question of whether moral responsibility is compatible with determinism? Specifically, can folk judgments in line with a particular answer to that question provide support for that answer. Based on reasoning familiar from Condorcet’s Jury Theorem, such support could be had if individual judges track the truth of the matter independently and with some modest reliability: such reliability quickly aggregates as the number of judges goes up. In this chapter, however, I argue, partly based on empirical evidence, that although non-specialist judgments might on average be more likely than not to get things right, their individual likelihoods fail to aggregate because they do not track truth with sufficient independence

The problem of evaluating automated large-scale evidence aggregators

In the biomedical context, policy makers face a large amount of potentially discordant evidence from different sources. This prompts the question of how this evidence should be aggregated in the interests of best-informed policy recommendations. The starting point of our discussion is Hunter and Williams’ recent work on an automated aggregation method for medical evidence. Our negative claim is that it is far from clear what the relevant criteria for evaluating an evidence aggregator of this sort are. What is the appropriate balance between explicitly coded algorithms and implicit reasoning involved, for instance, in the packaging of input evidence? In short: What is the optimal degree of ‘automation’? On the positive side: We propose the ability to perform an adequate robustness analysis as the focal criterion, primarily because it directs efforts to what is most important, namely, the structure of the algorithm and the appropriate extent of automation. Moreover, where there are resource constraints on the aggregation process, one must also consider what balance between volume of evidence and accuracy in the treatment of individual evidence best facilitates inference. There is no prerogative to aggregate the total evidence available if this would in fact reduce overall accuracy