SenseCam™ is a wearable, automatic camera with support for memory recall used as a lifelogging device. Recent and continuing work in Dublin City University to apply sophisticated time series analysis methods to the multiple time series generated on a Microsoft SenseCam™ have proved useful in detecting ‘‘Significant Events’’

Crane, Martin

Li, Na

Ruskin, Heather J.

Irish Universities

Automatically detecting "significant events" on SenseCam

LI, Na

Name not available

rising distance between the object andthe sensor the object is marked darker(from green to blue). The person walksaway from the sensor and falls after 9sec in the same direction of hiswalking. Therefore the color of the datapoints representing the persondecreases with time from bright to darkblue. CARE mainly targets single indi-vidual elderly people living in theirown private homes (Figure 4) with ahuge potential. The AAL market ischanging and is expected to boom inthe next few years as a result of demo-graphic developments and large invest-ments of industries and stakeholders.Actual products, which are on themarket for several years, mainly consistof wearable systems and intelligentfloor technologies; however their suc-cess is restricted to limited markets likenursing homes rather than the broaderaged communities. In the context ofindependent living, wearable systemsdo not have the best acceptance for pri-mary end-users (elderly), especiallythose who are not impaired. The CARE system for fall detectionservices is integrated with the Everoncaring system (see Figure 1, right) forwider deployment in Europe, strength-ening the independent living market andsociety.Link: http://www.care-aal.euPlease contact:Ahmed Nabil Belbachir, ErwinSchoitsch, AIT  Austrian Institute ofTechnology / AARIT, Austria  E-mail: {nabil.belbachir,erwin.schoitsch}@ait.ac.atFerenc VajdaBudapest University of Technologyand Economics, HungaryE-mail: vajda@iit.bme.hu Tommi LundenOy Everon Ab, FinlandE-mail: tommi.lunden@everon.fiERCIM NEWS 87   October 201130Special Theme: Ambient Assisted LivingFigure 3: CARE system and service chain.Automatically Detecting ‘‘Significant Events’’ on SenseCam™by Na Li, Martin Crane and Heather J. RuskinSenseCam™isawearable,automaticcamerawithsupportformemoryrecallusedasalifeloggingdevice.RecentandcontinuingworkinDublinCityUniversitytoapplysophisticatedtimeseriesanalysismethodstothemultipletimeseriesgeneratedonaMicrosoftSenseCam™haveprovedusefulindetecting‘‘SignificantEvents’’.A SenseCam™ is a small, wearablecamera that takes images, automati-cally, in order to document the events ofwearer’s day. It can be periodicallyreviewed to refresh and strengthen thewearer’s memory of an event. With apicture, taken typically every 30 sec-onds, thousands of images are capturedper day. Although experience showsthat the SenseCam™ can be an effectivememory-aiding device, as it helpswearers to improve retention of anexperience, wearers seldom wish toreview life events by browsing largecollections of images manually. Thechallenge is to manage, organise andanalyse these large image collections inorder to automatically highlight keyepisodes.Researchers within the Centre forScientific Computing and ComplexSystem Modelling, (Sci-Sym) atDublin City University have addressedthis problem by applying sophisticatedtime series methods to analysis of themultiple time series recorded by theSenseCam™, in order to automaticallycapture important or significant eventsof the wearer’s day. Detrended Fluctuation Analysis (DFA)was used initially to analyse imagetime series, recorded by theSenseCam™ and exposed strong long-range correlation in these collections.It implies that continuous low levels ofbackground information picked up allthe time by the device. Consequently,ERCIM NEWS 87   October 2011 31DFA provides a useful backgroundsummary. The equal-time cross-correlationmatrix with different sliding windowsize was also used to investigate thelargest eigenvalue and the changes inthe sub-dominant eigenvalue ratiospectrums. The smoothness of thelargest eigenvalue series was improvedby choosing a larger window size, tofilter small changes, with peaks evidentacross all scales, and large changeshighlighted. We detail the scenario forthe peaks in Figure 2 as an example:peaks a3 and a4 correspond to thewearer based in the office and sitting infront of the laptop. The laptop is pre-dominantly white, with the screen thelargest object in these images. TheSenseCam™ captures the image basisfor the two peaks, including lights onthe ceiling. A small dip between the twopeaks implies that the camera wasblocked for a while. Clearly, imagescaptured when objects are maintainedwithout change are of higher qualitythan when movement is increased.Thus, the laptop, lights and unchanged‘seated position’ contribute higherpixel values in these sequences ofimages. These consistently occurringpeaks in Figure 2 aided identification oflight intensity as a major event delin-eator during static periods of imagesequence. An eigenvalue ratio analysisconfirmed that the largest eigenvaluecarries most of the major event infor-mation, whereas sub-eigenvalues carryinformation on supporting or lead in/lead out events.The dynamics of the largest eigenvalueand changes of ratios of eigenvalueswere examined, using the MaximumOverlap Discrete Wavelet Transform(MODWT) method. This techniquegives a clear picture of the movementsin the image time series by recon-structing them using each wavelet com-ponent. Some peaks were visible acrossall scales, as expected, with specificevent information markedly apparent atlarger scales. By studying the largesteigenvalue across all wavelet scales, itwas possible to identify the time seriesfluctuation caused by change of asingle type, eg with environment con-stant, but with wearer movementincreased or more people joining ascene. It was also possible to validatethe analysis approach, by confirmingthat the largest time series effects aredue to grouped or combined changes,eg when the wearer's environmentchanges completely; (involving lightintensity, temperature, additionalpeople etc.). The eigenvalue ratiosanalyses are in good agreement withfindings for the largest eigenvalue,indicating that wavelet methods,(MODWT), provide a powerful tool forexamination of the nature of the cap-tured SenseCam™ data.Future work includes extension of theMaximum Overlap Discrete WaveletTransform to the investigation ofchanges in the cross-correlation struc-ture across all scales. Particular featuresshould be marked at different scales,enabling identification of key compo-nents of major or related events, andeven potential classification of eventtype in the SenseCam™ data.AcknowledgementsWe would like to thank our colleaguesfrom the Centre for Sensor WebTechnologies, (CLARITY), at DublinCity University for research assistance.Links:http://research.microsoft.com/en-us/um/cambridge/projects/sensecam/http://sci-sym.dcu.ie/http://www.clarity-centre.org/Please contact:Na Li, University of Limerick andDublin City University, IrelandE-mail: Na.Li@ul.ieMartin CraneDublin City University, IrelandE-mail: mcrane@computing.dcu.ieHeather RuskinDublin City University, IrelandE-mail: hruskin@computing.dcu.ieFigure 2: Largest Eigenvalue Distribution using a sliding window of 400 Images.Figure 1: SenseCam.

Automatically detecting "significant events" on SenseCam

Abstract

Similar works

Full text

Available Versions

Irish Universities

Name not available

DCU Online Research Access Service