The colour of life: interacting with SenseCam images on large multi-touch display walls

Abstract

A SenseCam can provide a detailed visual archive of a person’s life, activities and experiences. However, as the number of images captured per year can extend beyond one million, gaining an insight into an individual’s lifestyle in a fast, effective and intuitive manner is a challenging prospect. In this work, we develop an interactive image browsing tool, which incorporates visualisation techniques that can capture not only a snapshot of an individual’s lifestyle over long periods of time, but also how that lifestyle varies with changing days, weeks, or years. The image retrieval tool incorporates the Colour of Life algorithms [1], which can represent an overview of millions of images with a single visualisation. The Colour of Life algorithms focus on the relationship between lifestyle and colour, by capturing the colours to which we are exposed in our lives (and therefore captured by SenseCam images), collating similar colours for specific time periods and depicting how those colours change over time with a flowing time-line – see Figure 1 which depicts the life of a SenseCam user over the period of 8 days. In this figure, time is orientated along the horizontal axis and larger vertical peaks indicate higher user activity for a given period of time. In Figure 1, the normal working week consists of the rhythmical blue, pink (work) and yellow (home) peaks and troughs for each day (with less activity at the start and end of the days), whereas time outdoors increases at the weekend, especially during the night (and hence the darker colours on the left hand side of the figure). The Colour of Life visualisation, while providing information on changes in lifestyle, does not provide sufficient context to understand the exact activities of a user for a given time period. For example, on the left of Figure 1 there is a peak of purple, that does not occur anywhere else during the 8 days of activities images – where was the user at this point in time and what was he doing? In this work, we build an interactive image browsing tool based around the Colour of Life visualisation. We exploit the use of high resolution multi-touch display walls, where we extend the Colour of Life algorithms to produce an intuitive visualisation, which incorporates image mosaicing (see Figure 2). Through this we incorporate coarse lifestyle data with more fine detailed contextual information on human activities into one interactive visualisation tool. As an additional feature, we have investigated the use of image classification within the framework of the Colour of Life. One such example is the categorisation of images as being as social (i.e. interacting with other people) or non-social. Using such a classification, we can depict a person’s social lifestyle, and how that varies over time