A test of a computer-adaptive survey using online reviews

Abstract

© 26th European Conference on Information Systems: Beyond Digitization - Facets of Socio-Technical Change, ECIS 2018. All Rights Reserved. Traditional surveys are excellent instruments for establishing the correlational relationship between two constructs. However, they are unable to identify reasons why such correlations exist. Computer-Adaptive Surveys (CAS) are multi-dimensional instruments where questions asked of respondents depend on the previous questions asked. Assessing the validity of CAS is an underexplored research area as CAS differs from traditional surveys. Therefore, validating a CAS requires different techniques. This study attempts to validate the conclusion validity of a CAS about café customer satisfaction using online customer reviews. For our CAS to have conclusion validity, there should be a high correspondence where most respondents in CAS and online reviewers both agree that certain constructs are the cause of their dissatisfaction. We created a Computer-Adaptive Survey (CAS) of café satisfaction and used online customer reviews to assess its conclusion validity. Our research thus contributes to the measurement literature in two ways, one, we demonstrate that CAS captures the same criticisms of cafes as that in online reviews, and two, CAS captures problems about customer satisfaction at a deeper level than that found in online reviews