Combining face detection and novelty to identify important events in a visual lifelog

The SenseCam is a passively capturing wearable camera, worn around the neck and takes an average of almost 2,000 images per day, which equates to over 650,000 images per year. It is used to create a personal lifelog or visual recording of the wearer’s life and generates information which can be helpful as a human memory aid. For such a large amount of visual information to be of any use, it is accepted that it should be structured into “events”, of which there are about 8,000 in a wearer’s average year. In automatically segmenting SenseCam images into events, it is desirable to automatically emphasise more important events and decrease the emphasis on mundane/routine events. This paper introduces the concept of novelty to help determine the importance of events in a lifelog. By combining novelty with face-to-face conversation detection, our system improves on previous approaches. In our experiments we use a large set of lifelog images, a total of 288,479 images collected by 6 users over a time period of one month each

Structuring and augmenting a visual personal diary

This paper refers to research in the domain of visual lifelogging, whereby individuals capture much of their lives using digital cameras. The potential benefits of lifelogging include: applications to review tourist trips, memory aid applications, learning assistants, etc. The SenseCam, developed by Microsoft Research in Cambridge, UK, is a small wearable device which incorporates a digital camera and onboard sensors (motion, ambient temperature, light level, and passive infrared to detect presence of people). There exists a number of challenges in managing the vast quantities of data generated by lifelogging devices such as the SenseCam. Our work concentrates on the following areas withing visual lifelogging: Segmenting sequences of images into events (e.g. breakfast, at meeting); retrieving similar events (what other times was I at the park?); determining most important events (meeting an old friend is more important than breakfast); selection of ideal keyframe to provide an event summary; and augmenting lifeLog events with images taken by millions of users from "Web 2.0" websites (show me other pictures of the Statue of Liberty to augment my own lifelog images)

Intelligent image processing techniques for structuring a visual diary

The SenseCam is a small wearable personal device which automatically captures up to 3,500 images per day. This yields a very large personal collection of images or in a sense, a diary of a person's day. Over one million images will need to be stored each year, therefore intelligent techniques are necessary for the effective searching and browsing of this image collection for important or significant events in a person's life, and one of the issues is how to detect and then relate similar events in a lifetime. This is necessary in order to detect unusual or once-off events, as well as determining routine activities. This poster will present the various sources of data that can be collected with a SenseCam device, and also other sources that can be collected to compliment the SenseCam data sources. Different forms of image processing that can be carried out on this large set of images will be detailed, specifically how to detect what images belong to individual events, and also how similar various events are to each other. There will be hundreds of thousands of images of everyday routines; as a result more memorable events are quite likely to be significantly different to other normal reoccurring events

Organising a large quantity of lifelog images

Preliminary research indicates that a visual recording of one’s activities may be beneficial for sufferers of neurodegenerative diseases. However there exists a number of challenges in managing the vast quantities of data generated by lifelogging devices such as the SenseCam. Our work concentrates on the following areas within visual lifelogging: Segmenting sequences of images into events (e.g. breakfast, at meeting); retrieving similar events (“what other times was I at the park?”); determining most important events (meeting an old friend is more important than breakfast); selection of ideal keyframe to provide an event summary; and augmenting lifeLog events with images taken by millions of users from ‘Web 2.0’ websites (“show me other pictures of the Statue of Liberty to augment my own lifelog images”)

Automatically detecting important moments from everyday life using a mobile device

This paper proposes a new method to detect important moments in our lives. Our work is motivated by the increase in the quantity of multimedia data, such as videos and photos, which are capturing life experiences into personal archives. Even though such media-rich data suggests visual processing to identify important moments, the oft-mentioned problem of the semantic gap means that users cannot automatically identify or retrieve important moments using visual processing techniques alone. Our approach utilises on-board sensors from mobile devices to automatically identify important moments, as they are happening

Summarisation & Visualisation of Large Volumes of Time-Series Sensor Data

a number of sensors, including an electricity usage sensor supplied by Episensor. This poses our second With the increasing ubiquity of sensor data, challenge, how to summarise an extended period of presenting this data in a meaningful way to electrictiy usage data for a home user. users is a challenge that must be addressed before we can easily deploy real-world sensor network interfaces in the home or workplace. In this paper, we will present one solution to the visualisation of large quantities of sensor data that is easy to understand and yet provides meaningful and intuitive information to a user, even when examining many weeks or months of historical data. We will illustrate this visulalisation technique with two real-world deployments of sensing the person and sensing the home

Mining user activity as a context source for search and retrieval

Nowadays in information retrieval it is generally accepted that if we can better understand the context of users then this could help the search process, either at indexing time by including more metadata or at retrieval time by better modelling the user context. In this work we explore how activity recognition from tri-axial accelerometers can be employed to model a user's activity as a means of enabling context-aware information retrieval. In this paper we discuss how we can gather user activity automatically as a context source from a wearable mobile device and we evaluate the accuracy of our proposed user activity recognition algorithm. Our technique can recognise four kinds of activities which can be used to model part of an individual's current context. We discuss promising experimental results, possible approaches to improve our algorithms, and the impact of this work in modelling user context toward enhanced search and retrieval

Video shot boundary detection: seven years of TRECVid activity

Shot boundary detection (SBD) is the process of automatically detecting the boundaries between shots in video. It is a problem which has attracted much attention since video became available in digital form as it is an essential pre-processing step to almost all video analysis, indexing, summarisation, search, and other content-based operations. Automatic SBD was one of the tracks of activity within the annual TRECVid benchmarking exercise, each year from 2001 to 2007 inclusive. Over those seven years we have seen 57 different research groups from across the world work to determine the best approaches to SBD while using a common dataset and common scoring metrics. In this paper we present an overview of the TRECVid shot boundary detection task, a high-level overview of the most significant of the approaches taken, and a comparison of performances, focussing on one year (2005) as an example

Retrieval of similar travel routes using GPS tracklog place names

GPS tracklogs provide a valuable record of routes travelled. In this paper we describe initial experiments exploring the use of text information retrieval techniques for the location of similar trips from within a GPS tracklog. We performed the experiment on a dataset of 528 individual trips gathered over a seven month time period from a single user. The results of our preliminary study suggest that traditional text-based information retrieval techniques can indeed be used to locate similar and related tracklogs

Keyframe detection in visual lifelogs

The SenseCam is a wearable camera that passively captures images. Therefore, it requires no conscious effort by a user in taking a photo. A Visual Diary from such a source could prove to be a valuable tool in assisting the elderly, individuals with neurodegenerative diseases, or other traumas. One issue with Visual Lifelogs is the large volume of image data generated. In previous work we spit a day's worth of images into more manageable segments, i.e. into distinct events or activities. However, each event coud stil consist of 80-100 images. thus, in this paper we propose a novel approach to selecting the key images within an event using a combination of MPEG-7 and Scale Invariant Feature Transform (SIFT) features