The wisdom of collective grading and the effects of epistemic and semantic diversity

A computer simulation is used to study collective judgements that an expert panel reaches on the basis of qualitative probability judgements contributed by individual members. The simulated panel displays a strong and robust crowd wisdom effect. The panel's performance is better when members contribute precise probability estimates instead of qualitative judgements, but not by much. Surprisingly, it doesn't always hurt for panel members to interpret the probability expressions differently. Indeed, coordinating their understandings can be much worse

Grading in Groups

Juries, committees and experts panels commonly appraise things of one kind or another on the basis of grades awarded by several people. When everybody's grading thresholds are known to be the same, the results sometimes can be counted on to reflect the graders’ opinion. Otherwise, they often cannot. Under certain conditions, Arrow's ‘impossibility’ theorem entails that judgements reached by aggregating grades do not reliably track any collective sense of better and worse at all. These claims are made by adapting the Arrow–Sen framework for social choice to study grading in groups

Safety in numbers: how social choice theory can inform avalanche risk management

Avalanche studies have undergone a transition in recent years. Early research focused mainly on environmental factors. More recently, attention has turned to human factors in decision making, such as behavioural and cognitive biases. This article adds a social component to this human turn in avalanche studies. It identifies lessons for decision making by groups of skiers from the perspective of social choice theory, a sub-field of economics, decision theory, philosophy and political science that investigates voting methods and other forms of collective decision making. In the first part, we outline the phenomenon of wisdom of crowds, where groups make better decisions than their individual members. Drawing on the conceptual apparatus of social choice theory and using idealised scenarios, we identify conditions under which wisdom of crowds arises and also explain how and when deciding together can instead result in worse decisions than may be expected from individual group members. In the second part, we use this theoretical understanding to offer practical suggestions for decision making in avalanche terrain. Finally, we make several suggestions for risk management in other outdoor and adventure sports and for outdoor sports education

High-resolution analysis of HLA class I alterations in colorectal cancer

BACKGROUND: Previous studies indicate that alterations in Human Leukocyte Antigen (HLA) class I expression are frequent in colorectal tumors. This would suggest serious limitations for immunotherapy-based strategies involving T-cell recognition. Distinct patterns of HLA surface expression might conceal different immune escape mechanisms employed by the tumors and are worth further study. METHOD: We applied four-color multiparameter flow cytometry (FCM), using a large panel of alloantigen-specific anti-HLA-A and -B monoclonal antibodies, to study membranous expression of individual HLA alleles in freshly isolated colorectal cancer cell suspensions from 21 patients. RESULTS: Alterations in HLA class I phenotype were observed in 8 (38%) of the 21 tumors and comprised loss of a single A or B alleles in 4 cases, and loss of all four A and B alleles in the other 4 cases. Seven of these 8 tumors were located on the right side of the colon, and those showing loss of both HLA-A and -B membranous expression were all of the MSI-H phenotype. CONCLUSION: FCM allows the discrimination of complex phenotypes related to the expression of HLA class I. The different patterns of HLA class I expression might underlie different tumor behavior and influence the success rate of immunotherapy

Democracy without Enlightenment: A Jury Theorem for Evaluative Voting

Say a jury is going to decide who wins a competition. First, each member evaluates all the competitors by grading them; then, for each competitor, a collective grade is derived from all the judgments of all the members; finally, the jury chooses as the winner the competitor with the highest collective grade. This is collective grading. The grades that are used might typically be numerical scores, or evaluative expressions of a natural language, such as “good,” “fair,” and “bad.” They could be any signs at all, though, that come in a “top” to “bottom” order: thumbs up and down; happy, neutral, and sad emojis; or cheering, clapping, booing, and angry hissing at public events. Panels, boards, and committees throughout society evaluate all manner of things by grading them. Thus risks are prioritized, research proposals are funded, and candidates are shortlisted for jobs. Apart from acclamation in special cases, collective grading is not a usual way to pick winners in political elections. This article takes up a question about the quality of judgments and decisions made by collective grading: under which conditions are outcomes likely to be right? An answer comes in the form of a jury theorem for median grading. Here, the collective grade for a thing is the median of its individually assigned grades—the one in the middle, when all of them are listed from top to bottom. Section III prepares the ground for this theorem by discussing different senses in which grades can be the right ones for things, or the wrong ones as the case may be, independently of which grades are assigned in the end. These notions of right and wrong are relevant to judgments of different kinds of things: risks, research proposals, job candidates, options in referendums and elections. The grading‐jury theorem in Section V identifies conditions on the grading competence of individual people under which median grades, and decisions that follow them, are likely to be, independently, right