Trialling meta-research in comparative cognition: claims and statistical inference in animal physical cognition

Scientific disciplines face concerns about replicability and statistical inference, and these concerns are also relevant in animal cognition research. This paper presents a first attempt to assess how researchers make and publish claims about animal physical cognition, and the statistical inferences they use to support them. We surveyed 116 published experiments from 63 papers on physical cognition, covering 43 different species. The most common tasks in our sample were trap-tube tasks (14 papers), other tool use tasks (13 papers), means-end understanding and string-pulling tasks (11 papers), object choice and object permanence tasks (9 papers) and access tasks (5 papers). This sample is not representative of the full scope of physical cognition research; however, it does provide data on the types of statistical design and publication decisions researchers have adopted. Across the 116 experiments, the median sample size was 7. Depending on the definitions we used, we estimated that between 44% and 59% of our sample of papers made positive claims about animals’ physical cognitive abilities, between 24% and 46% made inconclusive claims, and between 10% and 17% made negative claims. Several failures of animals to pass physical cognition tasks were reported. Although our measures had low inter-observer reliability, these findings show that negative results can and have been published in the field. However, publication bias is still present, and consistent with this, we observed a drop in the frequency of p-values above .05. This suggests that some non-significant results have not been published. More promisingly, we found that researchers are likely making many correct statistical inferences at the individual-level. The strength of evidence of statistical effects at the group-level was weaker, and its p-value distribution was consistent with some effect sizes being overestimated. Studies such as ours can form part of a wider investigation into statistical reliability in comparative cognition. However, future work should focus on developing the validity and reliability of the measurements they use, and we offer some starting points

Long-tailed macaques extract statistical information from repeated types of events to make rational decisions under uncertainty

Human children and apes seem to be intuitive statisticians when making predictions from populations of objects to randomly drawn samples, whereas monkeys seem not to be. Statistical reasoning can also be investigated in tasks in which the probabilities of different possibilities must be inferred from relative frequencies of events, but little is known about the performance of nonhuman primates in such tasks. In the current study, we investigated whether long-tailed macaques extract statistical information from repeated types of events to make predictions under uncertainty. In each experiment, monkeys first experienced the probability of rewards associated with different factors separately. In a subsequent test trial, monkeys could then choose between the different factors presented simultaneously. In Experiment 1, we tested whether long-tailed macaques relied on probabilities and not on a comparison of absolute quantities to make predictions. In Experiment 2 and 3 we varied the nature of the predictive factors and the complexity of the covariation structure between rewards and factors. Results indicate that long-tailed macaques extract statistical information from repeated types of events to make predictions and rational decisions under uncertainty, in more or less complex scenarios. These findings suggest that the presentation format affects the monkeys’ statistical reasoning abilities

Stochasticity in economic losses increases the value of reputation in indirect reciprocity.

Recent theory predicts harsh and stochastic conditions to generally promote the evolution of cooperation. Here, we test experimentally whether stochasticity in economic losses also affects the value of reputation in indirect reciprocity, a type of cooperation that is very typical for humans. We used a repeated helping game with observers. One subject (the "Unlucky") lost some money, another one (the "Passer-by") could reduce this loss by accepting a cost to herself, thereby building up a reputation that could be used by others in later interactions. The losses were either stable or stochastic, but the average loss over time and the average efficiency gains of helping were kept constant in both treatments. We found that players with a reputation of being generous were generally more likely to receive help by others, such that investing into a good reputation generated long-term benefits that compensated for the immediate costs of helping. Helping frequencies were similar in both treatments, but players with a reputation to be selfish lost more resources under stochastic conditions. Hence, returns on investment were steeper when losses varied than when they did not. We conclude that this type of stochasticity increases the value of reputation in indirect reciprocity

Trialling Meta-Research in Comparative Cognition: Claims and Statistical Inference in Animal Physical Cognition.

Scientific disciplines face concerns about replicability and statistical inference, and these concerns are also relevant in animal cognition research. This paper presents a first attempt to assess how researchers make and publish claims about animal physical cognition, and the statistical inferences they use to support them. We surveyed 116 published experiments from 63 papers on physical cognition, covering 43 different species. The most common tasks in our sample were trap-tube tasks (14 papers), other tool use tasks (13 papers), means-end understanding and string-pulling tasks (11 papers), object choice and object permanence tasks (9 papers) and access tasks (5 papers). This sample is not representative of the full scope of physical cognition research; however, it does provide data on the types of statistical design and publication decisions researchers have adopted. Across the 116 experiments, the median sample size was 7. Depending on the definitions we used, we estimated that between 44% and 59% of our sample of papers made positive claims about animals' physical cognitive abilities, between 24% and 46% made inconclusive claims, and between 10% and 17% made negative claims. Several failures of animals to pass physical cognition tasks were reported. Although our measures had low inter-observer reliability, these findings show that negative results can and have been published in the field. However, publication bias is still present, and consistent with this, we observed a drop in the frequency of p-values above .05. This suggests that some non-significant results have not been published. More promisingly, we found that researchers are likely making many correct statistical inferences at the individual-level. The strength of evidence of statistical effects at the group-level was weaker, and its p-value distribution was consistent with some effect sizes being overestimated. Studies such as ours can form part of a wider investigation into statistical reliability in comparative cognition. However, future work should focus on developing the validity and reliability of the measurements they use, and we offer some starting points.BB/M011194/