Physiological signals can potentially be applied as objective measures to
understand the behavior and engagement of users interacting with information
access systems. However, the signals are highly sensitive, and many controls
are required in laboratory user studies. To investigate the extent to which
controlled or uncontrolled (i.e., confounding) variables such as task sequence
or duration influence the observed signals, we conducted a pilot study where
each participant completed four types of information-processing activities
(READ, LISTEN, SPEAK, and WRITE). Meanwhile, we collected data on blood volume
pulse, electrodermal activity, and pupil responses. We then used machine
learning approaches as a mechanism to examine the influence of controlled and
uncontrolled variables that commonly arise in user studies. Task duration was
found to have a substantial effect on the model performance, suggesting it
represents individual differences rather than giving insight into the target
variables. This work contributes to our understanding of such variables in
using physiological signals in information retrieval user studies.Comment: Accepted to the 46th International ACM SIGIR Conference on Research
and Development in Information Retrieval (SIGIR '23