Human computer interaction (HCI) groups are more and more often exploring the utility of new, lower cost electroencephalography (EEG) interfaces for assessing user engagement and experience as well as for directly controlling computers. While the potential benefits of using EEG are considerable, we argue that research is easily driven by what we term naïve neurorealism. That is, data obtained with psychophysiological devices have poor reliability and uncertain validity, making inferences on mental states difficult. This means that unless sufficient care is taken to address the inherent shortcomings, the contributions of psychophysiological human computer interaction are limited to their novelty value rather than bringing scientific advance. Here, we outline the nature and severity of the reliability and validity problems and give practical suggestions for HCI researchers and reviewers on the way forward, and which obstacles to avoid. We hope that this critical perspective helps to promote good practice in the emerging field of psychophysiology in HCI
