Real-time food intake classification and energy expenditure estimation on a mobile device

© 2015 IEEE.Assessment of food intake has a wide range of applications in public health and life-style related chronic disease management. In this paper, we propose a real-time food recognition platform combined with daily activity and energy expenditure estimation. In the proposed method, food recognition is based on hierarchical classification using multiple visual cues, supported by efficient software implementation suitable for realtime mobile device execution. A Fischer Vector representation together with a set of linear classifiers are used to categorize food intake. Daily energy expenditure estimation is achieved by using the built-in inertial motion sensors of the mobile device. The performance of the vision-based food recognition algorithm is compared to the current state-of-the-art, showing improved accuracy and high computational efficiency suitable for realtime feedback. Detailed user studies have also been performed to demonstrate the practical value of the software environment

Technology-assisted dietary assessment

Dietary intake provides valuable insights for mounting intervention programs for prevention of disease. With growing concern for adolescent obesity, the need to accurately measure diet becomes imperative. Assessment among adolescents is problematic as this group has irregular eating patterns and have less enthusiasm for recording food intake. Preliminary studies among adolescents suggest that innovative use of technology may improve the accuracy of diet information from young people. In this paper, we propose a novel food record method using a mobile device that will provide an accurate account of daily food and nutrient intake among adolescents. Our approach includes the use of image analysis tools for identification and quantification of food consumption. Images obtained before and after food is consumed can be used to estimate the diet of an individual. In this paper we describe our initial results and indicate the potential of the proposed system

Validation of a recommender system for prompting omitted foods in online dietary assessment surveys

Recall assistance methods are among the key aspects that improve the accuracy of online dietary assessment surveys. These methods still mainly rely on experience of trained interviewers with nutritional background, but data driven approaches could improve cost-efficiency and scalability of automated dietary assessment. We evaluated the effectiveness of a recommender algorithm developed for an online dietary assessment system called Intake24, that automates the multiple-pass 24-hour recall method. The recommender builds a model of eating behavior from recalls collected in past surveys. Based on foods they have already selected, the model is used to remind respondents of associated foods that they may have omitted to report. The performance of prompts generated by the model was compared to that of prompts hand-coded by nutritionists in two dietary studies. The results of our studies demonstrate that the recommender system is able to capture a higher number of foods omitted by respondents of online dietary surveys than prompts hand-coded by nutritionists. However, the considerably lower precision of generated prompts indicates an opportunity for further improvement of the system

Validation Study of a Passive Image-Assisted Dietary Assessment Method with Automated Image Analysis Process

Background: Image-assisted dietary assessment is being developed to enhance accuracy of dietary assessment. This study validated a passive image-assisted dietary assessment method, with an emphasis on examining if food shape and complexity influenced results.Methods: A 2x2x2x2x3 mixed factorial design was used, with a between-subject factor of meal orders, and within-subject factors of food shapes, food complexities, meals, and methods of measurement, to validate the passive image-assisted dietary assessment method. Thirty men and women (22.7 ± 1.6 kg/m2, 25.1 ± 6.6 years, 46.7% White) wore the Sony Smarteyeglass that automatically took images while two meals containing four foods representing four food categories were consumed. Images from the first 5 minutes of each meal were coded and then compared to DietCam for food identification. The comparison produced four outcomes: DietCam identifying food correctly in image (True Positive), DietCam incorrectly identifying food in image (False Positive), DietCam not identifying food in image (False Negative), or DietCam correctly identifying that the food is not in the image (True Negative). Participants’ feedback about the Sony Smarteyeglass was obtained by a survey.Results: A total of 36,412 images were coded by raters and analyzed by DietCam, with raters coding that 92.4% of images contained foods and DietCam coding that 76.3% of images contained foods. Mixed factorial analysis of covariance revealed a significant main effect of percent agreement between DietCam and rater’s coded images [(F (3,48) = 8.5, p \u3c 0.0001]. The overall mean of True Positive was 22.2 ± 3.6 %, False Positive was 1.2 ± 0.4%, False Negative was 19.6 ± 5.0%, and True Negative was 56.8 ± 7.2%. True Negative was significantly (p \u3c 0.0001) different from all other percent agreement categories. No main effects of food shape or complexity were found. Participants reported that they were not willing to wear the Sony Smarteyeglass under different types of dining experiences.Conclusion: DietCam is most accurate in identifying images that do not contain food. The platform from which the images are collected needs to be modified to enhance consumer acceptance

Systems and WBANs for Controlling Obesity

According to World Health Organization (WHO) estimations, one out of five adults worldwide will be obese by 2025. Worldwide obesity has doubled since 1980. In fact, more than 1.9 billion adults (39%) of 18 years and older were overweight and over 600 million (13%) of these were obese in 2014. 42 million children under the age of five were overweight or obese in 2014. Obesity is a top public health problem due to its associated morbidity and mortality. This paper reviews the main techniques to measure the level of obesity and body fat percentage, and explains the complications that can carry to the individual's quality of life, longevity and the significant cost of healthcare systems. Researchers and developers are adapting the existing technology, as intelligent phones or some wearable gadgets to be used for controlling obesity. They include the promoting of healthy eating culture and adopting the physical activity lifestyle. The paper also shows a comprehensive study of the most used mobile applications and Wireless Body Area Networks focused on controlling the obesity and overweight. Finally, this paper proposes an intelligent architecture that takes into account both, physiological and cognitive aspects to reduce the degree of obesity and overweight

Quantification of energy intake using food image analysis

Obtaining real-time and accurate estimates of energy intake while people reside in their natural environment is technically and methodologically challenging. The goal of this project is to estimate energy intake accurately in real-time and free-living conditions. In this study, we propose a computer vision based system to estimate energy intake based on food pictures taken and emailed by subjects participating in the experiment. The system introduces a reference card inclusion procedure, which is used for geometric and photometric corrections. Image classification and segmentation methods are also incorporated into the system to have fully-automated decision making

A Pilot Study Of The Effectiveness And Usability Of The Myenergybalance Iphone App And Website

The powerful technical capabilities of smartphones offer unprecedented opportunities for collecting dietary information. We have developed an enhanced smartphone application called MyEnergyBalance, which permits imaged-based self-monitoring of all foods consumed, and links to a convenient and user-friendly web-based dietary assessment tool. The primary objective of this pilot study was to determine if the MyEnergyBalance app (with use of images) in combination of the associated website improves dietary recall compared to diet analysis on the MyEnergyBalance website alone. We also generated preliminary data on the usability of the MyEnergyBalance iPhone app and website. This pilot study was a crossover study design of healthy, college students. Participants were randomly assigned to two groups. Both groups consumed their normal diet for the first day with one group recording their food intake with image functions of the MyEnergyBalance app, while the other group did not use the app. On the second day, all participants logged into the MyEnergyBalance website to record their food intake from the previous day; one group using the images from the app to assist in recalling what they ate, while the other group recalled what they ate from memory. The diet analysis results were compared to those obtained using the ASA24 website. The groups were then crossed over to the opposite vs no-image assisted recalls. Ten participants (seven females and three males) aged 20 to 22 years completed this study. The average BMI of all participants was 23.12 kg/m2 (ranging from 18.95 to 32.28 kg/m2). There was no statistically significant differences in the estimates of the energy intake between the MyEnergyBalance app and website compared to ASA24. The SUS mean score for the MyEnergyBalance app and website was 86 and 69.5, respectively. A strong, negative correlation was found between the system usability scale scores and the absolute differences in energy intake of the MyEnergyBalance app and ASA24. Although we were not able to demonstrate a significant benefit of the images from the iPhone app at improving food recall (perhaps due to the small study sample size), we were able to demonstrate a high usability score for the iPhone app, average usability score for the website, and a significant correlation between subjects\u27 usability scores and relative accuracy of the subjects\u27 food recall using the images from the iPhone app. A future study with a larger sample size will hopefully provide more information on the efficacy of image-based food recalls