Evaluation is essential to understanding the value that digital creativity
brings to people's experience, for example in terms of their enjoyment,
creativity, and engagement. There is a substantial body of research on how to
design and evaluate interactive arts and digital creativity applications. There
is also extensive Human-Computer Interaction (HCI) literature on how to
evaluate user interfaces and user experiences. However, it can be difficult for
artists, practitioners, and researchers to navigate such a broad and disparate
collection of materials when considering how to evaluate technology they create
that is at the intersection of art and interaction. This chapter provides a
guide to designing robust user studies of creative applications at the
intersection of art, technology and interaction, which we refer to as Media and
Arts Technology (MAT). We break MAT studies down into two main kinds:
proof-of-concept and comparative studies. As MAT studies are exploratory in
nature, their evaluation requires the collection and analysis of both
qualitative data such as free text questionnaire responses, interviews, and
observations, and also quantitative data such as questionnaires, number of
interactions, and length of time spent interacting. This chapter draws on over
15 years of experience of designing and evaluating novel interactive systems to
provide a concrete template on how to structure a study to evaluate MATs that
is both rigorous and repeatable, and how to report study results that are
publishable and accessible to a wide readership in art and science communities
alike.Comment: Preprint. Chapter to appear in "Creating Digitally. Shifting
Boundaries: Arts and Technologies - Contemporary Applications and Concepts",
Anthony L. Brooks (Editor), Springer.
https://link.springer.com/book/978303131359