Holistic and mechanical combination in psychological assessment:Why algorithms are underutilized and what is needed to increase their use

Although mechanical combination results in more valid judgments and decisions than holistic combination, existing publications suggest that mechanical combination is rarely used in practice. Yet, these publications are either descriptions of anecdotal experiences or outdated surveys. Therefore, in several Western countries, we conducted two surveys (total N = 323) and two focus groups to investigate (1) how decision makers in psychological and HR practice combine information, (2) why they do (not) use mechanical combination, and (3) what may be needed to increase its use in practice. Many participants reported mostly using holistic combination, usually in teams. The most common reasons for not using mechanical combination were that algorithms are unavailable in practice and that stakeholders do not accept their use. Furthermore, decision makers do not quantify information, do not believe in research findings on evidence-based decision making, and think that combining holistic and mechanical combination results in the best decisions. The most important reason why mechanical combination is used was to increase predictive validity. To stimulate the use of mechanical combination in practice, our results suggest that decision makers should receive more training on evidence-based decision making, and decision aids supporting the use of mechanical combination should be developed

A Study on the Validity of a Computer-Based Game to Assess Cognitive Processes, Reward Mechanisms, and Time Perception in Children Aged 4-8 Years

BACKGROUND: A computer-based game, named Timo's Adventure, was developed to assess specific cognitive functions (eg, attention, planning, and working memory), time perception, and reward mechanisms in young school-aged children. The game consists of 6 mini-games embedded in a story line and includes fantasy elements to enhance motivation. OBJECTIVE: The aim of this study was to investigate the validity of Timo's Adventure in normally developing children and in children with attention-deficit/hyperactivity disorder (ADHD). METHODS: A total of 96 normally developing children aged 4-8 years and 40 children with ADHD were assessed using the game. Clinical validity was investigated by examining the effects of age on performances within the normally developing children, as well as performance differences between the healthy controls and the ADHD group. RESULTS: Our analyses in the normally developing children showed developmental effects; that is, older children made fewer inhibition mistakes (r=-.33, P=.001), had faster (and therefore better) reaction times (r=-.49, P<.001), and were able to produce time intervals more accurately than younger children (ρ=.35, P<.001). Discriminant analysis showed that Timo's Adventure was accurate in most classifications whether a child belonged to the ADHD group or the normally developing group: 78% (76/97) of the children were correctly classified as having ADHD or as being in the normally developing group. The classification results showed that 72% (41/57) children in the control group were correctly classified, and 88% (35/40) of the children in the ADHD group were correctly classified as having ADHD. Sensitivity (0.89) and specificity (0.69) of Timo's Adventure were satisfying. CONCLUSIONS: Computer-based games seem to be a valid tool to assess specific strengths and weaknesses in young children with ADHD