We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing

In behavioral, cognitive, and social sciences, reaction time measures are an important source of information. However, analyses on reaction time data are affected by researchers' analytical choices and the order in which these choices are applied. The results of a systematic literature review, presented in this paper, revealed that the justification for and order in which analytical choices are conducted are rarely reported, leading to difficulty in reproducing results and interpreting mixed findings. To address this methodological shortcoming, we created a checklist on reporting reaction time pre-processing to make these decisions more explicit, improve transparency, and thus, promote best practices within the field. The importance of the pre-processing checklist was additionally supported by an expert consensus survey and a multiverse analysis. Consequently, we appeal for maximal transparency on all methods applied and offer a checklist to improve replicability and reproducibility of studies that use reaction time measures

Cognitive effort investment: Does disposition become action?

Contrary to the law of less work, individuals with high levels of need for cognition and self-control tend to choose harder tasks more often. While both traits can be integrated into a core construct of dispositional cognitive effort investment, its relation to actual cognitive effort investment remains unclear. As individuals with high levels of cognitive effort investment are characterized by a high intrinsic motivation towards effortful cognition, they would be less likely to increase their effort based on expected payoff, but rather based on increasing demand. In the present study, we measured actual effort investment on multiple dimensions, i.e., subjective load, reaction time, accuracy, early and late frontal midline theta power, N2 and P3 amplitude, and pupil dilation. In a sample of N = 148 participants, we examined the relationship of dispositional cognitive effort investment and effort indices during a flanker and an n-back task with varying demand and payoff. Exploratorily, we examined this relationship for the two subdimensions cognitive motivation and effortful-self-control as well. In both tasks, effort indices were sensitive to demand and partly to payoff. The analyses revealed a main effect of cognitive effort investment for accuracy (n-back task), interaction effects with payoff for reaction time (n-back and flanker task) and P3 amplitude (n-back task) and demand for early frontal midline theta power (flanker task). Taken together, our results partly support the notion that individuals with high levels of cognitive effort investment exert effort more efficiently. Moreover, the notion that these individuals exert effort regardless of payoff is partly supported, too. This may further our understanding of the conditions under which person-situation interactions occur, i.e. the conditions under which situations determine effort investment in goal-directed behavior more than personality, and vice versa

We Don’t Know What You Did Last Summer. On the Importance of Transparent Reporting of Reaction Time Data Pre-processing

  In behavioral, cognitive, and social sciences, reaction time measures are an important source of information. However, analyses on reaction time data are affected by researchers' analytical choices and the order in which these choices are applied. The results of a systematic literature review, presented in this paper, revealed that the justification for and order in which analytical choices are conducted are rarely reported, leading to difficulty in reproducing results and interpreting mixed findings. To address this methodological shortcoming, we created a checklist on reporting reaction time pre-processing to make these decisions more explicit, improve transparency, and thus, promote best practices within the field. The importance of the pre-processing checklist was additionally supported by an expert consensus survey and a multiverse analysis. Consequently, we appeal for maximal transparency on all methods applied and offer a checklist to improve replicability and reproducibility of studies that use reaction time measures.  </p