Save the last dance for me: Unwanted serial position effects in jury evaluations

Whenever competing options are considered in sequence, their evaluations may be affected by order of appearance. Such serial position effects would threaten the fairness of competitions using jury evaluations. Randomization cannot reduce potential order effects, but it does give candidates an equal chance of being assigned to preferred serial positions. Whether, or what, serial position effects emerge may depend on the cognitive demands of the judgment task. In end-of-sequence procedures, final scores are not given until all candidates have performed, possibly burdening judges’ memory. If judges’ evaluations are based on how well they remember performances, serial position effects may resemble those found with free recall. Candidates may also be evaluated step-by-step, immediately after each performance. This procedure should not burden memory, though it may produce different serial position effects. Yet, this paper reports similar serial position effects with end-of-sequence and step-by-step procedures used for the Eurovision Song Contest: Ratings increased with serial position. The linear order effect was replicated in the step-by-step judgments of World and European Figure Skating Contests. It is proposed that, independent of the evaluation procedure, judges’ initial impressions of sequentially appearing candidates may be formed step-by-step, yielding serial position effects

Save the last dance for me: Unwanted serial position effects in jury evaluations

Whenever competing options are considered in sequence, their evaluations may be affected by order of appearance. Such serial position effects would threaten the fairness of competitions using jury evaluations. Randomization cannot reduce potential order effects, but it does give candidates an equal chance of being assigned to preferred serial positions. Whether, or what, serial position effects emerge may depend on the cognitive demands of the judgment task. In end-of-sequence procedures, final scores are not given until all candidates have performed, possibly burdening judges’ memory. If judges’ evaluations are based on how well they remember performances, serial position effects may resemble those found with free recall. Candidates may also be evaluated step-by-step, immediately after each performance. This procedure should not burden memory, though it may produce different serial position effects. Yet, this paper reports similar serial position effects with end-of-sequence and step-by-step procedures used for the Eurovision Song Contest: Ratings increased with serial position. The linear order effect was replicated in the step-by-step judgments of World and European Figure Skating Contests. It is proposed that, independent of the evaluation procedure, judges’ initial impressions of sequentially appearing candidates may be formed step-by-step, yielding serial position effects

Save the last dance for me: Unwanted serial position effects in jury evaluations

Whenever competing options are considered in sequence, their evaluations may be affected by order of appearance. Such serial position effects would threaten the fairness of competitions using jury evaluations. Randomization cannot reduce potential order effects, but it does give candidates an equal chance of being assigned to preferred serial positions. Whether, or what, serial position effects emerge may depend on the cognitive demands of the judgment task. In end-of-sequence procedures, final scores are not given until all candidates have performed, possibly burdening judges’ memory. If judges’ evaluations are based on how well they remember performances, serial position effects may resemble those found with free recall. Candidates may also be evaluated step-by-step, immediately after each performance. This procedure should not burden memory, though it may produce different serial position effects. Yet, this paper reports similar serial position effects with end-of-sequence and step-by-step procedures used for the Eurovision Song Contest: Ratings increased with serial position. The linear order effect was replicated in the step-by-step judgments of World and European Figure Skating Contests. It is proposed that, independent of the evaluation procedure, judges’ initial impressions of sequentially appearing candidates may be formed step-by-step, yielding serial position effects

Fifty-fifty=50%?

Several recent surveys have asked respondents to estimate the probabilities of relatively unlikely events, such as dying from breast cancer and smoking. Examination of their response distributions reveals a seemingly inappropriate ‘blip’ at 50. The two studies reported here indicate that it reflects a response artifact associated with open-ended probability scales. The blip vanishes when a response scale with explicit response options is offered. Apparently, the open-ended format leads some people to use the 50% option as ‘fifty–fifty’, an expression of having no idea as to the answer. As a result, the accuracy of people's reported beliefs depends on the response scale used, as well as on how it evokes and channels such feelings of epistemic uncertainty

Fifty-fifty=50%?

Several recent surveys have asked respondents to estimate the probabilities of relatively unlikely events, such as dying from breast cancer and smoking. Examination of their response distributions reveals a seemingly inappropriate ‘blip’ at 50. The two studies reported here indicate that it reflects a response artifact associated with open-ended probability scales. The blip vanishes when a response scale with explicit response options is offered. Apparently, the open-ended format leads some people to use the 50% option as ‘fifty–fifty’, an expression of having no idea as to the answer. As a result, the accuracy of people's reported beliefs depends on the response scale used, as well as on how it evokes and channels such feelings of epistemic uncertainty