Smartphone-based Calorie Estimation From Food Image Using Distance Information

Personal assistive systems for diet control can play a vital role to combat obesity. As smartphones have become inseparable companions for a large number of people around the world, designing smartphone-based system is perhaps the best choice at the moment. Using this system people can take an image of their food right before eating, know the calorie content based on the food items on the plate. In this paper, we propose a simple method that ensures both user flexibility and high accuracy at the same time. The proposed system employs capturing food images with a fixed posture and estimating the volume of the food using simple geometry. The real world experiments on different food items chosen arbitrarily show that the proposed system can work well for both regular and liquid food items

Pipeline for Calculating Calories for Print Recipes with Minimal User Intervention

The thesis will provide a pipeline to estimate calorie counts from print recipes. The pipeline takes scanned recipes from cookbooks and uses Optical Character Recognition (OCR) to convert the scanned images of recipes to text. Several OCR tools were tested for their accuracy on fractions using a sample of the data, and the most accurate tool is used on the data. Next, a specially trained named entity recognition model is used to identify ingredients, quantities and units. These ingredients are used to search a database of values from the FDA to compute a calorie count for the recipe. The thesis tests the effectiveness of search by examining performance over 100 of the most common ingredients in the corpus of recipes. Finally, the thesis tests the performance of the model on a set of recipes, and found to estimate the calorie count at least as accurately as other automated approaches, such as those based on image recognition

Feasibility of Food Recognition and Calorie Estimation of Fast Food and Healthy Meals Available in the Philippines

This paper presents the design and development of a food recognition smartphone application which can also display the estimated calorie/s of the food itself. It is intended for people who would like to monitor their diet through food calorie intake measurement (i.e. user’s daily calorie intake record). It is equipped with a food database consisting of typical fruits and vegetables commonly found in the Philippines. As part of the study, it also includes some of the meals in food chains (i.e. McDonald's, and The Healthy Corner) found in the Philippines where the calorie information is readily available. The result shows 82.86 % accuracy for the top-1 category, and 99.29 % for the top-5 category. The algorithm being used in this project is Artificial Neural Network (ANN) wherein the recognition process must properly be achieved. Furthermore, the aforementioned database is supported by TensorFlow which is an open-source software library for Machine Intelligence

Learning from Unstructured Data to Monitor Human Health

The integration of mobile devices into our daily lives has created unique opportunities to improve human health and well-being. Many of these devices such as smartphones and smartwatches allow the users to enter unstructured data such as speech. This research is focused on utilizing such data for health monitoring through development of computational algorithms and optimization strategies that process unstructured data, compute health-related markers, and provide recommendations for improved health. The applications of this research include nutrition monitoring, dietary recommendation, personality assessment, and commonsense reasoning. Diet is known as an important lifestyle factor in self-management and prevention of chronic diseases. Although mobile and wearable sensors have been used to estimate eating context, accurate monitoring of dietary intake has remained a challenging problem. New approaches based on mobile devices have been proposed to facilitate the process of food intake recording. These technologies require individuals to use mobile devices such as smartphones to record nutrition intake by either entering text or taking images of the food. These technologies are prone to measurement errors related to challenges of human memory and bias. In order to address these limitations, we introduced development and validation of two nutrition monitoring frameworks, Speech2Health and EZNutriPal, which use unstructured data along with speech processing, natural language processing (NLP), and text mining techniques to facilitate dietary assessment.Implementing strategies that improve dietary intake is also very important. A general diet behavior change framework for joint nutrition monitoring and diet planning allows continuous diet recommendations for achieving a diet goal. This research introduces a diet planning framework, called iTell-uEat, to provide diet recommendations continuously based on user's diet habits. Two approaches are proposed including a reinforcement-learning-based and a greedy-based diet planning. An optimization algorithm is proposed to construct a meaningful action space for training reinforcement learning algorithms. Moreover, a linear optimization approach is developed forgreedy diet planning. To demonstrate the potential of utilizing unstructured data in applications beyond dietary assessment, a computational framework is proposed to analyze human personality traits based on expressed texts and to use these personalities for behavior and commonsense reasoning analysis

Designing Augmented Reality Applications for Personal Health Decision-Making

Augmented reality (AR) is a technology that can assist with our daily decision-making tasks by presenting information that extends the physical world. However, little work has been done to understand the effect of the layout of AR interfaces on decision-making. In this paper, we present PHARA, an AR-based personal assistant that supports decision-making for healthier food products. In a controlled user study (n=28), we explored the use of four different AR layouts on two different devices: Microsoft HoloLens and smartphone. Using subjective and objective means, we measured their effects on decision-making tasks that occur when people hold food products in their hands. We found that pie and grid layouts perform better on the smartphone, whereas a stacked layout works better on the reduced field-of-view of the Microsoft HoloLens, potentially at the cost of some affordances such as time spent and actions