Optimizing the usage of pupillary based indicators for cognitive workload

The Index of Cognitive Activity (ICA) and its open-source alternative, the Index of Pupillary Activity (IPA), are pupillary-based indicators for cognitive workload and are independent of light changes. Both indicators were investigated regarding influences of cognitive demand, fatigue and inter-individual differences. In addition, the variability of pupil changes between both eyes (difference values) were compared with the usually calculated pupillary changes averaged over both eyes (mean values). Fifty-five participants performed a spatial thinking test, the R-Cube-Vis Test, with six distinct difficulty levels and a simple fixation task before and after the R-Cube-Vis Test. The distributions of the ICA and IPA were comparable. The ICA/IPA values were lower during the simple fixation tasks than during the cognitively demanding R-Cube-Vis Test. A fatigue effect was found only for the mean ICA values. The effects of both indicators were larger between difficulty levels of the test when inter-individual differences were controlled using z-standardization. The difference values seemed to control for fatigue and appeared to differentiate better between more demanding cognitive tasks than the mean values. The derived recommendations for the ICA/IPA values are beneficial to gain more insights in individual performance and behavior during, e.g., training and testing scenarios

Unifying the ability-as-compensator and ability-as-enhancer hypotheses

Spatial abilities have been found to interact with the design of visualizations in educational materials in different forms: (1) spatial abilities enhanced learning with optimized visual design (ability-as-enhancer) or (2) spatial abilities compensated for suboptimal visual design (ability-as-compensator). A brief review of pertinent studies suggests that these two forms are viewed as mutually exclusive. We propose a novel unifying conceptualization. This conceptualization suggests that the ability-as enhancer interaction will be found in the low-medium range of a broad ability continuum whereas the ability-as-compensator interaction will be found in the medium-high range. The largest difference in learning outcomes between visual design variations is expected for medium ability. A corresponding analytical approach is suggested that includes nonlinear quadratic interactions. The unifying conceptualization was confirmed in an experiment with a consistent visual-spatial task. In addition, the conceptualization was investigated with a reanalysis of pooled data from four multimedia learning experiments. Consistent with the conceptualization, quadratic interactions were found, meaning that interactions depended on ability range. The largest difference between visual design variations was obtained for medium ability, as expected. It is concluded that the unifying conceptualization is a useful theoretical and methodological approach to analyze and interpret aptitude-treatment interactions that go beyond linear interactions

The diagnostic potential of eye tracking and pupilometry in the context of spatial thinking

Eye tracking and pupillometry are promising techniques in the context of psychological diagnostics. Standard tests are usually based on accuracy and/or reaction times as critical measures for ability estimation. Deeper insights in how participants solve certain tasks or why they fail can mostly be detected insufficiently by those measures. In contrast, gaze fixation patterns are able to reflect cognitive processes while a participant is solving a specific task. Furthermore, pupillary-based measures can indicate the cognitive workload. Hence, eye tracking and pupillometry have the potential to improve psychological tests by providing information to establish sophisticated cognitive processing models that allow obtaining a differential profile of a participant. The goal of the present studies was to reveal this potential in the context of spatial thinking by demonstrating how and which prerequisites has to be performed to be able to establish such a cognitive processing model. In the first four studies, the R-Cube-Vis test was developed and validated to overcome the drawbacks of standard spatial thinking test with respect to the usage of eye tracking. The R-Cube-Vis test measures the first factor of spatial thinking, visualization, and was developed as short and long version. For comparison reasons, the R-CubeSR test was developed and validated in addition, measuring the second factor of spatial thinking, spatial relations. The studies’ results indicate high evidence of discriminant and convergent validity of each test for different investigated measures and tasks. Additionally, the studies delivered strong evidence for conformity of the R-Cube-Vis test with the linear logistic test model with six difficulty levels. In the fifth study, various gaze-fixation-based indicators were analyzed to identify item-independent patterns indicating the cognitive processes that are presumably necessary to solve an item versus item-dependent patterns that only reflect random characteristics of the item’s visual appearance that, however, are not connected with the demanded cognitive processes. In the second part of the fifth study, these indicators were analyzed regarding their potential to indicate test performance and the correctness of the answer of a single item. The sixth study investigated the Index of Cognitive Activity (ICA), a pupillarybased measure of cognitive workload, for its usage with the R-Cube-Vis test. In addition to the expected ordering (i.e., low ICA values for easier items and higher values for more difficult items), further aspects such as the effect of fatigue, hemispheric lateralization, and controlling of inter-individual differences were analyzed to derive recommendations for an appropriate usage. In the final application study, changes from the first to the last item within each of the six difficulty levels were analyzed for behavioral measures, self-assessment as well as eye-trackingbased and pupillary-based measures. All measures were also investigated with respect to differences between participants regarding their ability in visualization. The results showed the gain of information by gaze-related and pupillary-based measures that goes beyond the information from item accuracy and reaction times. Although no cognitive processing model was established, the studies demonstrate the high potential of gaze-related and pupillary-based measures for psychological testing but also show the necessary requirements for the stimulus materials as well as the need of preliminary work to be able to interpret the respective indicators in an appropriate way

Spatial thinking from a different view: Disentangling top-down and bottom-up processes using eye tracking

The goal of the present study was to investigate the potential of gaze fixation patterns to reflect cognitive processing steps during test performance. Gaze movements, however, can reflect top-down and bottom-up processes. Top-down processes are the cognitive processing steps that are necessary to solve a certain test item. In contrast, bottom-up processes may be provoked by varying visual features that are not related to the item solution. To disentangle top-down and bottom-up processes in the context of spatial thinking, a new test (R-Cube-Vis Test) was developed and validated explicitly for the usage of eye tracking in three studies as long and short version. The R-Cube-Vis Test measures visualization and is conform to the linear logistic test model with six difficulty levels. All items of one level demand the same transformation steps to solve an item. The R-Cube-Vis Test was then utilized to investigate different gaze-fixation-based indicators to identify top-down and bottom-up processes. Some of the indicators were also able to predict the correctness of the answer of a single item. Gaze-related measures have a high potential to reveal cognitive processing steps during solving an item of a given difficulty level, if top-down and bottom-up processes can be segregated