Ubiquitous computing is becoming an increasingly prevalent part of our everyday lives. The reliance of society upon such devices as mobile phones, coupled with the increasing complexity of those devices is an example of how our everyday human-human interaction is affected by this phenomenon. Social scientists studying human-human interaction must now take into account the effects of these technologies not just on the interaction itself, but also on the approach required to study it. User evaluation is a challenging topic in ubiquitous computing. It is generally considered to be difficult, certainly more so than in previous computational settings. Heterogeneity in design, distributed and mobile users, invisible sensing systems and so on, all add up to render traditional methods of observation and evaluation insufficient to construct a complete view of interactional activity. These challenges necessitate the development of new observational technologies. This thesis explores some of those challenges and demonstrates that system logs, with suitable methods of synchronising, filtering and visualising them for use in conjunction with more traditional observational approaches such as video, can be used to overcome many of these issues.
Through a review of both the literature of the field, and the state of the art of computer aided qualitative data analysis software (CAQDAS), a series of guidelines are constructed showing what would be required of a software toolkit to meet the challenges of studying ubiquitous computing systems. It outlines the design and implementation of two such software packages, \textit{Replayer} and \textit{Digital Replay System}, which approach the problem from different angles, the former being focussed on visualising and exploring the data in system logs and the latter focussing on supporting the methods used by social scientists to perform qualitative analyses.
The thesis shows through case studies how this technique can be applied to add significant value to the qualitative analysis of ubiquitous computing systems: how the coordination of system logs and other media can help us find information in the data that would otherwise be inaccessible; an ability to perform studies in locations/settings that would otherwise be impossible, or at least very difficult; and how creating accessible qualitative data analysis tools allows people to study particular settings or technologies who could not have studied them before. This software aims to demonstrate the direction in which other CAQDAS packages may have to move in order to support the study of the characteristics of human-computer and human-human interaction in a world increasingly reliant upon ubiquitous computing technology
