This presentation will be of interest to 1) physical activity researchers interested in better capturing participant physical activity and sedentary behaviour, and 2) computer science researchers interested in wearable sensors and image processing challenges in a new application area. In this lecture, Dr Doherty will discuss use of the Microsoft Research SenseCam, a wearable camera sensor which automatically captures up to 5,000 first-person point-of-view images. The images strongly help identify the type of activity a participant is involved in, and can also help determine the environment and situation surrounding the given activity. Because up to 35,000 images can be captured from a participant each week, there exists a substantial information management challenge in storing, annotating, and retrieving image content. This talk will discuss current state-of-art computational approaches applied to SenseCam images, and their application in the field of physical activity

Doherty, Aiden R.

English

Name not available

British Heart Foundation Health Promotion Research Group 3rd March 2011 – EPARC & CWPHS, San Diego  Using wearable image sensing to measure physical activity & sedentary behavior Aiden Doherty BHF HPRG Department of Public Health  University of Oxford International consensus on health benefits of physical activity •Physical activity can reduce the risk of: •Cardiovascular disease •Hypertension •Obesity •Some forms of cancers •Non insulin-dependent diabetes mellitus •Strokes  •Osteoarthritis, by maintaining normal muscle strength, joint structure and joint function •Osteoporosis   • Cognitive function • Crime reduction and community safety • Economic impact and regeneration of communities • Education and lifelong learning • Psychological well-being • Self esteem • Management of anxiety and depression • Social capital and community cohesion • Drug misuse • Carbon use  (US Dept Health & Human Sciences, 1996; U.K. CMO, 2004; Sport England, 2009) Physical activity Work Leisure & Play Exercise & Sport Household Active Travel % active Age 61% of men and 71% of women do not meet the U.K. Chief Medical  Officer‟s minimum recommendations for physical activity in adults Sedentary Behaviour Sitting (or lying down), involving < 2 MET (metabolic equivalent) MET 1 2 3 4 0.9: Sleeping 1.0: Sitting quietly (TV viewing) 1.5: Sitting (talking) 1.8: Sitting (desk work) 2.5: Slow walking 3.8: Brisk walking Sedentary Light Moderate Ainsworth BE, et al. Med Sci Sport Exer. 2000;32:S498–S516 2.0: Standing Public Health Physical Activity Guidelines: time spent in moderate-vigorous activity Our modern „sitting-oriented‟ society Sleep 11pm  Awake 7 am     Work on  computer 4 hrs Transport to work 45 mins Lunch 30 mins Evening meal 30 mins Breakfast 15 mins Work on  computer 3.5 hrs Transport  From work 45 mins Watch TV 4 hrs Sitting Opportunities 15.5 hrs  Walk – 30 min AusDiab: are 5-year changes in TV viewing time associated with 5-year changes in: • Overweight (waist circumference) and other metabolic syndrome variables   •     independently of physical activity,                                                      diet quality, and other confounding  factors  •     in population-based sample of  healthy Australian adults (AusDiab)  2000 2005 Daily Sitting Time and All-cause Mortality in 17,013 Canadian Men and Women  Canada Fitness Survey 12-year Mortality Follow-up, 1981-1993 Almost None of the Time ¼ of the Time ½ of the Time ¾ of the Time Almost All of the Time Katzmarzyk PT et al. (2009) Sitting time and mortality from all causes, cardiovascular disease, and cancer.  Med Sci Sports Exerc 41: 998-1005 From: Pucher & Buehler. Transport Reviews, 2008.  OECD (age 15 and over). Data from various sources. Obesity & Active Travel Obesity and active travel • Each additional kilometre walked per day is associated with a 4.8% reduction in likelihood of obesity • Each additional hour spent in a car per day associated with a 6% increase in likelihood of obesity. • Active travel interventions must contain environmental supports to sustain individual choice (i.e. public transport) Frank, L., et al (2004) Obesity relationships with community design, physical activity, and time spent in cars. American Journal of Preventive Medicine, 27(2): 87-96.  NICE review – physical activity and environment   32% risk reduction  all cause mortality  (Hamer and Chida, 2008) 28% risk reduction  all cause mortality  (Anderson et al, 2000) Pressure on transport systems Sedentary behaviour Carbon emissions  Aims Why  research active travel? SenseCam Study results Other applications Sedentary behaviour  Establish links  between physical  activity & health   Measure physical activity Test  interventions Identify correlates Translate into practice Behavioural epidemiology framework Sallis and Owen (1999)  Current tools and technologies Pedometer Accelerometer Travel Diary GPS tracker British Heart Foundation Health Promotion Research Group Percentage of adults from same study meeting physical activity recommendations:  NHANES (self report): 50% Accelerometer: 5% (Troiano et al, 2009)  Self-report questionnaire: 38% Accelerometer: 5% (HSE, 2009)   The gold standard is direct observation 17 17 Visual Lifelogging Devices •Much past research focus on miniaturising hardware and increasing battery-life + storage e.g. visual lifelogging domain Tano et. al. University of Electro-Communications, Tokyo, Japan Microsoft Research SenseCam Steve Mann. Wearable computing: a first step toward personal imaging. Computer, 30:25–32, Feb 1997. TIMELINE Human Digital Memory (HDM) Why do HDM? HDM Software Future Opportunities 33 Daily Browser Overview Event Segmentation SenseCam Images of a day (about 3,000) Using MOTION sensors – very quick & accurate EVENT SEGMENTATION 34 Best: Compare Event Averages   (from middle n images) Visual Search Facilities Event Segmentation Day -1 Day -2 Day -5 Day -3 Day -4 Day -6 Event-Event Comparison within the Multi-day Event database Event database containing last 7 days’ Events SenseCam Images of a day (about 3,000) Better: Compare Event Averages ? … … Cross compare -Too slow 35 Selecting Event “Keyframe” Event Segmentation Day -1 Day -2 Day -5 Day -3 Day -4 Day -6 Event-Event Comparison within the Multi-day Event database Event database containing last 7 days’ Events Landmark Image Selection SenseCam Images of a day (about 3,000) Best QUALITY image around MIDDLE of event 36 Suggest Interesting Events Event Segmentation CALCULATE INTERESTINGNESS OF EVENTS 2 Sept 06 Interactive Browser Day -1 Day -2 Day -5 Day -3 Day -4 Day -6 Event-Event Comparison within the Multi-day Event database Event database containing last 7 days’ Events Landmark Image Selection SenseCam Images of a day (about 3,000) Mon Tue Wed Thr Fri Sat Sun Unique Events Mon Similar Events - Aiden waiting for bus Similar Events - Aiden at the office corridor Similar Events - Aiden working on the desk VISUAL NOVELTY + FACE DETECTION 37 So what can the SenseCam be used for?  Case study:  - Quantifying active travel self report error UK National Travel Survey 1. Quantifying error on self-report   Widely used, important for trends, used with other devices  Errors potentially come from recall, perception, human factors and social desirability  We intend to investigate the size of any error on self-reported journey behaviour  Error = a + b + c + d +?  a – systematic error b – intra-person variability c – inter-person variability d – modal effects ? – regular vs. irregular   Research questions  1. Will people wear it?  2. How does SenseCam and Self-report compare?  3. What are the sources of any error?    Study     Protocol: Wear SenseCam and complete travel diary for one day   Participants: 20 volunteers    Structured interviews about burden and experience       Will people wear SenseCam? 91% 94% 105 journeys (car, walk, bike, bus) 96 journeys 99 journeys How do self report and SenseCam data compare? Journey time = 20 minutes Journey time = 12 min 48 sec  How did they compare?  y = 0.9601x + 190.09 R² = 0.8425 0500100015002000250030003500-500 0 500 1000 1500 2000 2500 3000 3500Correlation NTS (sec) SenseCam (sec) Systematic over report =  190 sec +/- 47 sec Average over report  = 154 sec  +/- 30 sec All journeys  +2 min 30 sec  (S.E. 32 sec) Car +2 min 08 sec (S.E. 60 sec) Walk +1 min 41 sec (S.E. 45 sec) Bike +4 min 33 sec (S.E. 64 sec) So what…? 154 sec per journey  =  6 min 42 sec per day*  =  54 min per week  =  36% of recommended amount**   *3 ‘Active transportation’ journeys per participant per day  **Physical activity recommendations; 30 min per day, 5 days per week…or 150 minutes per week     (Chief Medical Officer, Department of Health)  Why are people over-reporting travel time?  Retrospective interviews:   Example A;   “I said 25 minutes because it took 10 minutes to get the kids in the car”  Example B;   “I think about the time I leave the house and the time I walk into the office, not the time spent cycling” OK it’s promising to investigate inherent error in active travel self-report … what else can it be useful for with respect to physical activity? 2. Combination with GPS  Location important for many reasons   Limitations include cold start, signal loss and estimation of mode from speed or self-report  16:01:48 16:24:03 16:25:28 18:33:53 (QStarz BT Q1000X)  3. Combination with accelerometer  Intensity important  Challenge to verify mode or behaviour from trace    MIS-CLASSIFYING SEDENTARY BEHAVIOUR AS NON-WEAR TIME… 5. Environmental audit or determinants   Cycle lane use Automated activity detection  Identifying Activities Sitting/Standing = 75% accurate Using a range of classifiers: Logistic Regression, Naïve Bayes, J48, SVM, Etc. Identifying Activities Walking = 77% Accurate Identifying Activities Driving = 88% Accurate 73  Activity Recognition using Images •27 “activities”  •Validated on 95k annotated images   74 Concept detection process Classifier Fusion SVM SVM SVM Lifelog images Feature Fusion Colour Layout Scalable Colour Visual features Concept probability Labeled examples 75 Event accuracy is better 76 steeringWheeleatinginsideVehiclevehiclesExternalreading holdingPhone-1.5-0.50.51.52.53.5user 1 user 2 user 3 user 4 user 5standard deviations away from sample meansteeringWheeleatinginsideVehiclevehiclesExternalreadingholdingPhoneComparison of Lifestyle Within Social Groups But let‟s use more people (34x)... Participant Group and (#) Median # of Days of SenceCam data Median # of Events per Day Median # SenseCam Images per Day Median SenseCam wear per Day Office Workers (6) 7 19.5 1,599 6h 55m Researchers (15) 8 20 1,640 7h 15m Retired (5) 3 23 1,886 7h 45m Regular lifeloggers (8) 42 18.5 1,517 10h 21m Overall Averages 15.1 20.9 1,712 8h 45m Differences between groups... face insideVehicle meeting screen -1-0.500.511.5lifelogger (8x) office (6x) researcher(15x)retired (5x)Num Standard Deviations From Mean faceinsideVehiclemeetingscreenWhen do people eat? 0%5%10%15%20%25%30%8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23Probability of Eating Hour in Day Eating  Patterns During Average Day lifelogger office researcher retiredWhen do people look at screens? 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23Hour in Day "Screen" Patterns During Average Day lifelogger office researcher retiredIn Conclusion:  Computer Scientists: Measuring health-related behaviour offers many opportunities   Physical Activity Researchers: SenseCam offers potential as a powerful context reinstatement tool  Thanks to Dr Charlie Foster Paul Kelly Prof. Alan Smeaton Dr Steve Hodges Sensors and Devices Group Microsoft Research Cambridge 3rd March 2011 – EPARC & CWPHS, San Diego  Using wearable image sensing to measure physical activity & sedentary behavior Aiden Doherty 

Using wearable image sensing to measure physical activity & sedentary behavior

DCU Online Research Access Service

Irish Universities

http://doras.dcu.ie/16294/1/Aiden_Doherty_SenseCam_seminar.pdf

Using wearable image sensing to measure physical activity & sedentary behavior

Abstract

Similar works

Full text

Available Versions

Name not available

DCU Online Research Access Service

Irish Universities