Anatomy of a failure: how we knew when our design went wrong, and what we learned from it

Abstract

In this paper, we describe the failure of a novel sensor-based system intended to evoke user interpretation and appropriation in domestic settings. We contrast participants' interactions in this case study with those observed during more successful deployments to identify 'symptoms of failure' under four themes: engagement, reference, accommodation, and surprise and insight. These themes provide a set of sensitivities or orientations that may complement traditional task-based approaches to evaluation as well as the more open-ended ones we describe here. Our system showed symptoms of failure under each of these themes. We examine the reasons for this at three levels: problems particular to the specific design hypothesis; problems relevant for input-output mapping more generally; and problems in the design process we used. We conclude by noting that, although interpretive systems such as the one we describe here may succeed in a myriad of different ways, it is reassuring to know that they can also fail, and fail incontrovertibly, yet instructively