A Framework to Estimate the Nutritional Value of Food in Real Time Using Deep Learning Techniques

There has been a rapid increase in dietary ailments during last few decades, caused by unhealthy food routine. Mobile-based dietary assessment systems that can record real time images of meal and analyze it for nutritional content can be very handy and improve the dietary habits, and therefore, result in healthy life. This paper proposes a novel system to automatically estimate food attributes such as ingredients and nutritional value by classifying the input image of food. Our method employs different deep learning models for accurate food identification. In addition to image analysis, attributes and ingredients are estimated by extracting semantically related words from a huge corpus of text, collected over the Internet. We performed experiments with a dataset comprising 100 classes, averaging 1000 images for each class to acquire top 1 classification rate of up to 85 percent. An extension of a benchmark dataset Food-101 is also created to include sub-continental foods. Results show that our proposed system is equally efficient on basic Food- 101 dataset and its extension for sub-continental foods. The proposed system is implemented as a mobile app that has its application in healthcare sector

Hierarchical Attention Network for Visually-aware Food Recommendation

Food recommender systems play an important role in assisting users to identify the desired food to eat. Deciding what food to eat is a complex and multi-faceted process, which is influenced by many factors such as the ingredients, appearance of the recipe, the user's personal preference on food, and various contexts like what had been eaten in the past meals. In this work, we formulate the food recommendation problem as predicting user preference on recipes based on three key factors that determine a user's choice on food, namely, 1) the user's (and other users') history; 2) the ingredients of a recipe; and 3) the descriptive image of a recipe. To address this challenging problem, we develop a dedicated neural network based solution Hierarchical Attention based Food Recommendation (HAFR) which is capable of: 1) capturing the collaborative filtering effect like what similar users tend to eat; 2) inferring a user's preference at the ingredient level; and 3) learning user preference from the recipe's visual images. To evaluate our proposed method, we construct a large-scale dataset consisting of millions of ratings from AllRecipes.com. Extensive experiments show that our method outperforms several competing recommender solutions like Factorization Machine and Visual Bayesian Personalized Ranking with an average improvement of 12%, offering promising results in predicting user preference for food. Codes and dataset will be released upon acceptance

Innovative food recommendation systems: a machine learning approach

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonRecommendation systems employ users history data records to predict their preference, and have been widely used in diverse fields including biology, e-commerce, and healthcare. Traditional recommendation techniques include content-based, collaborative-based and hybrid methods but not all real-world problems can be best addressed by these classical recommendation techniques. Food recommendation is one such challenging problem where there is an urgent need to use novel recommendation systems in assisting people to select healthy, balanced and personalized food plans. In this thesis, we make several advances in food recommendation systems using innovative machine learning methods. First, a novel recommendation approach is proposed by transforming an original recommendation problem into a many-objective optimisation one that contains several different objectives resulting in more balanced recommendations. Second, a unified approach to designing sequence-based personalised food recommendation systems is investigated to accommodate dynamic user behaviours. Third, a new food recommendation approach is developed with a temporal dependent graph neural network and data augmentation techniques leading to more accurate and robust recommendations. The experimental results show that these proposed approaches have not only provided a more balanced and accurate way of recommending food than the traditional methods but also led to promising areas for future research

Image-based food classification and volume estimation for dietary assessment: a review.

A daily dietary assessment method named 24-hour dietary recall has commonly been used in nutritional epidemiology studies to capture detailed information of the food eaten by the participants to help understand their dietary behaviour. However, in this self-reporting technique, the food types and the portion size reported highly depends on users' subjective judgement which may lead to a biased and inaccurate dietary analysis result. As a result, a variety of visual-based dietary assessment approaches have been proposed recently. While these methods show promises in tackling issues in nutritional epidemiology studies, several challenges and forthcoming opportunities, as detailed in this study, still exist. This study provides an overview of computing algorithms, mathematical models and methodologies used in the field of image-based dietary assessment. It also provides a comprehensive comparison of the state of the art approaches in food recognition and volume/weight estimation in terms of their processing speed, model accuracy, efficiency and constraints. It will be followed by a discussion on deep learning method and its efficacy in dietary assessment. After a comprehensive exploration, we found that integrated dietary assessment systems combining with different approaches could be the potential solution to tackling the challenges in accurate dietary intake assessment