Abstract Conducting scientific studies is an often frustrating and tedious task. To minimize the usual problems, such as lack of concentration or willingness to participate, and instead promote interest in the study, a smart mobile device assessment tool was developed. The tablet-based assessment tool offers a wide range of visual tasks that can be employed when conducting studies utilizing the European Framework of Visual Literacy (ENViL). Furthermore, the assessment tool is highly configurable in the field using a centralized server and spreadsheet-based configuration files, thereby ensuring that no programming language is required to adapt the tasks on the mobile devices participating in the study. Finally, the presented framework and architecture are completely cross-platform and cross-device and can be re-used and extended for any number of similar studies and tasks