Jones, Gareth J.F.

Lee, Hyowon

O'Connor, Noel E.

Smeaton, Alan F.

English

O\u27Connor, Noel E.

Irish Universities

Adaptive visual summary of lifeLog photos for
personal information management

DCU Online Research Access Service

Adaptive Visual Summary of LifeLog Photos forPersonal Information ManagementHyowon Lee, Alan F. Smeaton, Noel E. O’Connor and Gareth J.F. JonesAdaptive Information Cluster & Centre for Digital Video ProcessingDublin City University, Glasnevin, Dublin 9, Irelandhlee@computing.dcu.ieAs the cost of taking a photo drops, due to the use of digital cameras, andthe incentive to take photos increases, due to the ubiquity of camera phones andeasy ways to share them with a large number of viewers, the average number ofpersonal photos produced has increased dramatically. The need for supportingaccess to a large number of photos by efficient browsing and searching has be-come more crucial than ever, and the use of context information such as GPSlocation and time of capture are currently being researched to reduce the user’sannotation burden and to aid retrieval.Enter passive capture - the user attaches a camera the size of a button onher chest. The camera automatically and regularly takes photos whenever aninteresting event happens throughout the day while the user goes about herdaily activity or holiday trip. The SenseCam, developed by Microsoft Research,is a small digital camera that a user wears around her neck. It contains a numberof sensors including infra-red and motion sensors to automatically trigger photocapture in such a way that the photos taken are not blurred. On a typicalday, the SenseCam will take 3,000 - 4,000 photos throughout the day capturingmeaningful and significant images of the wearer’s activity, in effect chroniclingmost of the day’s events.At the end of the day, the wearer can download all the photos from theSenseCam to their computer as a detailed visual record of the day. The factthat everything is captured for reviewing or searching is comforting on the onehand, yet going through over 3,000 photos for each day can take a long time andmuch effort, and when multiple days are captured it becomes prohibitive to tryto extract anything of use from this number of photos.We use various content-based image analysis techniques on SenseCam photocollections spanning multiple days to automatically detect visual events. Foreach event we detect a landmark photo as a kind of “keyframe”. We then auto-matically compose an interactive browser that summarises, emphasises and canreplay thousands of SenseCam photos on a single page in an efficient and com-fortable way so as to not overload the viewer (see Figure 1). Significant eventsare detected among each day’s photos and their uniqueness or importance scoresare calculated by examining how frequently and for how long similar events haveoccurred during the previous 1-week period. For example, the wearer workingin front of a computer for 2 hours in the morning would appear almost everyday, and thus such a visual event scores as less important; whereas a 15-minuteunexpected meeting with a colleague on a corridor which happened only once inthe whole week, is given a higher importance score. The first day’s desk work in adifferent university lab for a research visit would be determined as an importantevent as this is unique among the visual events of the preceding week, but asthe days pass the desk work at the same lab will bear less and less importanceas it becomes a common activity. Thus, the system adaptively re-ranks the im-portance of each event as the day’s photos come into the database using a 7-daywindow. The current day’s photos are presented as a comic book style interfacewith different size photos according to their ranked importance.At the poster session, the overall information flow and processing of imageswill be presented with the SenseCam device and a few sample collections ofits photos. The prototype of the interactive browser under development will bepresented which dynamically summarises a large number of photos in a highlyinviting, simple, and enjoyable way.Figure 1. Interactive browser for reviewing a day’s SenseCam imagesAcknowledgmentsThis work is supported by Science Foundation Ireland under grant number03/IN.3/I361, by Microsoft Research and by the EC under contract FP6-027026(K-Space). We are grateful to the aceMedia Project for use of the aceToolbox.

Adaptive visual summary of lifeLog photos for

personal information management

Adaptive visual summary of lifeLog photos for personal information management

Abstract

Similar works

Full text

Available Versions

Irish Universities

DCU Online Research Access Service