Smartphone-based Calorie Estimation From Food Image Using Distance Information

Personal assistive systems for diet control can play a vital role to combat obesity. As smartphones have become inseparable companions for a large number of people around the world, designing smartphone-based system is perhaps the best choice at the moment. Using this system people can take an image of their food right before eating, know the calorie content based on the food items on the plate. In this paper, we propose a simple method that ensures both user flexibility and high accuracy at the same time. The proposed system employs capturing food images with a fixed posture and estimating the volume of the food using simple geometry. The real world experiments on different food items chosen arbitrarily show that the proposed system can work well for both regular and liquid food items

A Survey on Automated Food Monitoring and Dietary Management Systems

Healthy diet with balanced nutrition is key to the prevention of life-threatening diseases such as obesity, cardiovascular disease, and cancer. Recent advances in smartphone and wearable sensor technologies have led to a proliferation of food monitoring applications based on automated food image processing and eating episode detection, with the goal to conquer drawbacks of the traditional manual food journaling that is time consuming, inaccurate, underreporting, and low adherent. In order to provide users feedback with nutritional information accompanied by insightful dietary advice, various techniques in light of the key computational learning principles have been explored. This survey presents a variety of methodologies and resources on this topic, along with unsolved problems, and closes with a perspective and boarder implications of this field

CaloriNet: From silhouettes to calorie estimation in private environments

We propose a novel deep fusion architecture, CaloriNet, for the online estimation of energy expenditure for free living monitoring in private environments, where RGB data is discarded and replaced by silhouettes. Our fused convolutional neural network architecture is trainable end-to-end, to estimate calorie expenditure, using temporal foreground silhouettes alongside accelerometer data. The network is trained and cross-validated on a publicly available dataset, SPHERE_RGBD + Inertial_calorie. Results show state-of-the-art minimum error on the estimation of energy expenditure (calories per minute), outperforming alternative, standard and single-modal techniques.Comment: 11 pages, 7 figure

An Improved Encoder-Decoder Framework for Food Energy Estimation

Dietary assessment is essential to maintaining a healthy lifestyle. Automatic image-based dietary assessment is a growing field of research due to the increasing prevalence of image capturing devices (e.g. mobile phones). In this work, we estimate food energy from a single monocular image, a difficult task due to the limited hard-to-extract amount of energy information present in an image. To do so, we employ an improved encoder-decoder framework for energy estimation; the encoder transforms the image into a representation embedded with food energy information in an easier-to-extract format, which the decoder then extracts the energy information from. To implement our method, we compile a high-quality food image dataset verified by registered dietitians containing eating scene images, food-item segmentation masks, and ground truth calorie values. Our method improves upon previous caloric estimation methods by over 10\% and 30 kCal in terms of MAPE and MAE respectively.Comment: Accepted for Madima'23 in ACM Multimedi

CuisineNet: Food Attributes Classification using Multi-scale Convolution Network

Diversity of food and its attributes represents the culinary habits of peoples from different countries. Thus, this paper addresses the problem of identifying food culture of people around the world and its flavor by classifying two main food attributes, cuisine and flavor. A deep learning model based on multi-scale convotuional networks is proposed for extracting more accurate features from input images. The aggregation of multi-scale convolution layers with different kernel size is also used for weighting the features results from different scales. In addition, a joint loss function based on Negative Log Likelihood (NLL) is used to fit the model probability to multi labeled classes for multi-modal classification task. Furthermore, this work provides a new dataset for food attributes, so-called Yummly48K, extracted from the popular food website, Yummly. Our model is assessed on the constructed Yummly48K dataset. The experimental results show that our proposed method yields 65% and 62% average F1 score on validation and test set which outperforming the state-of-the-art models.Comment: 8 pages, Submitted in CCIA 201

Perspectives and Preferences of Adult Smartphone Users Regarding Nutrition and Diet Apps: Web-Based Survey Study

BACKGROUND: Digital technologies have evolved dramatically in the recent years finding applications in a variety of aspects of everyday life. Smartphones and mobile apps are steadily used for more and more tasks including health monitoring. A large amount of "Nutrition and Diet" apps are available with some of them being very popular in terms of user downloads highlighting a trend towards diet monitoring and assessment. OBJECTIVE: We sought to explore the perspectives of end-users on the features, current use, and acceptance of "Nutrition and Diet" mHealth apps with a survey. We expect that such a study can provide user insights, assisting researchers and developers towards innovative dietary assessment. METHODS: A multidisciplinary team designed and compiled the survey. Before its release, it has been pilot-tested by 18 end-users. A 19-question survey was finally developed which has been translated into six languages: EN, DE, FR, ES, IT, EL. The participants were mainly recruited via social media and mailing lists of universities, university hospitals and patient associations. RESULTS: Respondents (n=2382) (79.4% female, 19.9% male, 0.7% neither) with a mean age of 27.2 (SD: 8.5) completed the survey. Around half of the participants (51.5%, 1227 out of 2382) have used a "Nutrition and Diet" app. The primary criteria for selecting such an app were to be easy to use (65.9%, 1570 out of 2382), free of charge (59.3%, 1413 out of 2382) and also produce automatic readings of caloric (51.7%, 1231 out of 2382) and macronutrient content (46.9%, 1117 out of 2382) (i.e., food type and/or the portion size are estimated by the system without any contribution by the user). An app is less likely to be selected if it incorrectly estimates portion size, calories or nutrient content (33.5%, 798 out of 2382). Moreover, other important limitations include the use of a database that comprises of non-local foods (27.5%, 655 out of 2382) and which may omit major foods (41%, 977 out of 2382). CONCLUSIONS: This comprehensive study in a mostly European population assessed the preferences and perspectives of (potential) "Nutrition and Diet" app users. Understanding user needs will benefit both researchers who work on tools for innovative dietary assessment, as well as those who assist research on behavioural changes related to nutrition