Real-time food intake classification and energy expenditure estimation on a mobile device

© 2015 IEEE.Assessment of food intake has a wide range of applications in public health and life-style related chronic disease management. In this paper, we propose a real-time food recognition platform combined with daily activity and energy expenditure estimation. In the proposed method, food recognition is based on hierarchical classification using multiple visual cues, supported by efficient software implementation suitable for realtime mobile device execution. A Fischer Vector representation together with a set of linear classifiers are used to categorize food intake. Daily energy expenditure estimation is achieved by using the built-in inertial motion sensors of the mobile device. The performance of the vision-based food recognition algorithm is compared to the current state-of-the-art, showing improved accuracy and high computational efficiency suitable for realtime feedback. Detailed user studies have also been performed to demonstrate the practical value of the software environment

Validation of a recommender system for prompting omitted foods in online dietary assessment surveys

Recall assistance methods are among the key aspects that improve the accuracy of online dietary assessment surveys. These methods still mainly rely on experience of trained interviewers with nutritional background, but data driven approaches could improve cost-efficiency and scalability of automated dietary assessment. We evaluated the effectiveness of a recommender algorithm developed for an online dietary assessment system called Intake24, that automates the multiple-pass 24-hour recall method. The recommender builds a model of eating behavior from recalls collected in past surveys. Based on foods they have already selected, the model is used to remind respondents of associated foods that they may have omitted to report. The performance of prompts generated by the model was compared to that of prompts hand-coded by nutritionists in two dietary studies. The results of our studies demonstrate that the recommender system is able to capture a higher number of foods omitted by respondents of online dietary surveys than prompts hand-coded by nutritionists. However, the considerably lower precision of generated prompts indicates an opportunity for further improvement of the system

Technology-assisted dietary assessment

Dietary intake provides valuable insights for mounting intervention programs for prevention of disease. With growing concern for adolescent obesity, the need to accurately measure diet becomes imperative. Assessment among adolescents is problematic as this group has irregular eating patterns and have less enthusiasm for recording food intake. Preliminary studies among adolescents suggest that innovative use of technology may improve the accuracy of diet information from young people. In this paper, we propose a novel food record method using a mobile device that will provide an accurate account of daily food and nutrient intake among adolescents. Our approach includes the use of image analysis tools for identification and quantification of food consumption. Images obtained before and after food is consumed can be used to estimate the diet of an individual. In this paper we describe our initial results and indicate the potential of the proposed system

Validation Study of a Passive Image-Assisted Dietary Assessment Method with Automated Image Analysis Process

Background: Image-assisted dietary assessment is being developed to enhance accuracy of dietary assessment. This study validated a passive image-assisted dietary assessment method, with an emphasis on examining if food shape and complexity influenced results.Methods: A 2x2x2x2x3 mixed factorial design was used, with a between-subject factor of meal orders, and within-subject factors of food shapes, food complexities, meals, and methods of measurement, to validate the passive image-assisted dietary assessment method. Thirty men and women (22.7 ± 1.6 kg/m2, 25.1 ± 6.6 years, 46.7% White) wore the Sony Smarteyeglass that automatically took images while two meals containing four foods representing four food categories were consumed. Images from the first 5 minutes of each meal were coded and then compared to DietCam for food identification. The comparison produced four outcomes: DietCam identifying food correctly in image (True Positive), DietCam incorrectly identifying food in image (False Positive), DietCam not identifying food in image (False Negative), or DietCam correctly identifying that the food is not in the image (True Negative). Participants’ feedback about the Sony Smarteyeglass was obtained by a survey.Results: A total of 36,412 images were coded by raters and analyzed by DietCam, with raters coding that 92.4% of images contained foods and DietCam coding that 76.3% of images contained foods. Mixed factorial analysis of covariance revealed a significant main effect of percent agreement between DietCam and rater’s coded images [(F (3,48) = 8.5, p \u3c 0.0001]. The overall mean of True Positive was 22.2 ± 3.6 %, False Positive was 1.2 ± 0.4%, False Negative was 19.6 ± 5.0%, and True Negative was 56.8 ± 7.2%. True Negative was significantly (p \u3c 0.0001) different from all other percent agreement categories. No main effects of food shape or complexity were found. Participants reported that they were not willing to wear the Sony Smarteyeglass under different types of dining experiences.Conclusion: DietCam is most accurate in identifying images that do not contain food. The platform from which the images are collected needs to be modified to enhance consumer acceptance

Systems and WBANs for Controlling Obesity

According to World Health Organization (WHO) estimations, one out of five adults worldwide will be obese by 2025. Worldwide obesity has doubled since 1980. In fact, more than 1.9 billion adults (39%) of 18 years and older were overweight and over 600 million (13%) of these were obese in 2014. 42 million children under the age of five were overweight or obese in 2014. Obesity is a top public health problem due to its associated morbidity and mortality. This paper reviews the main techniques to measure the level of obesity and body fat percentage, and explains the complications that can carry to the individual's quality of life, longevity and the significant cost of healthcare systems. Researchers and developers are adapting the existing technology, as intelligent phones or some wearable gadgets to be used for controlling obesity. They include the promoting of healthy eating culture and adopting the physical activity lifestyle. The paper also shows a comprehensive study of the most used mobile applications and Wireless Body Area Networks focused on controlling the obesity and overweight. Finally, this paper proposes an intelligent architecture that takes into account both, physiological and cognitive aspects to reduce the degree of obesity and overweight

Quantification of energy intake using food image analysis

Obtaining real-time and accurate estimates of energy intake while people reside in their natural environment is technically and methodologically challenging. The goal of this project is to estimate energy intake accurately in real-time and free-living conditions. In this study, we propose a computer vision based system to estimate energy intake based on food pictures taken and emailed by subjects participating in the experiment. The system introduces a reference card inclusion procedure, which is used for geometric and photometric corrections. Image classification and segmentation methods are also incorporated into the system to have fully-automated decision making

Mobile monitoring application to support sustainable behavioural change towards healthy lifestyle

We describe the development of body area networks (BANs) incorporating sensors and other devices to provide intelligent mobile services in healthcare and well-being. The first BAN applications were designed to simply transmit biosignals and display them remotely. Further developments include analysis and interpretation of biosignals in the light of context data. By including feedback loops, BAN telemonitoring was also augmented with teletreatment services. Recent developments include incorporation of clinical decision support by applying techniques from artificial intelligence. These developments represent a movement towards smart healthcare, making health BAN applications more intelligent by incorporating feedback, context awareness, personalization, and decision support.\ud The element of decision support was first introduced into the BAN health and well-being applications in the Food Valley Eating Advisor (FOVEA) project. Obesity and overweight represent a growing threat to health and well-being in modern society. Physical inactivity has been shown to contribute significantly to morbidity and mortality rates, and this is now a global trend bringing huge costs in terms of human suffering and reduction in life expectancy as well as uncontrolled growth in demand on healthcare services. Part of the solution is to foster healthier lifestyle. A major challenge however is that exercise and dietary programs may work for the individual in the short term, but adherence in the medium and long term is difficult to sustain, making weight management a continuing struggle for individuals and a growing problem for society, governments, and health services. Using ICT to support sustainable behavioral change in relation to healthy exercise and diet is the goal of the FOVEA monitoring and feedback application. We strive to design and develop intelligent BAN-based applications that support motivation and adherence in the long term. We present this healthy lifestyle application and report results of an evaluation conducted by surveying professionals in related disciplines