A Survey on Automated Food Monitoring and Dietary Management Systems

Healthy diet with balanced nutrition is key to the prevention of life-threatening diseases such as obesity, cardiovascular disease, and cancer. Recent advances in smartphone and wearable sensor technologies have led to a proliferation of food monitoring applications based on automated food image processing and eating episode detection, with the goal to conquer drawbacks of the traditional manual food journaling that is time consuming, inaccurate, underreporting, and low adherent. In order to provide users feedback with nutritional information accompanied by insightful dietary advice, various techniques in light of the key computational learning principles have been explored. This survey presents a variety of methodologies and resources on this topic, along with unsolved problems, and closes with a perspective and boarder implications of this field

An Improved Encoder-Decoder Framework for Food Energy Estimation

Dietary assessment is essential to maintaining a healthy lifestyle. Automatic image-based dietary assessment is a growing field of research due to the increasing prevalence of image capturing devices (e.g. mobile phones). In this work, we estimate food energy from a single monocular image, a difficult task due to the limited hard-to-extract amount of energy information present in an image. To do so, we employ an improved encoder-decoder framework for energy estimation; the encoder transforms the image into a representation embedded with food energy information in an easier-to-extract format, which the decoder then extracts the energy information from. To implement our method, we compile a high-quality food image dataset verified by registered dietitians containing eating scene images, food-item segmentation masks, and ground truth calorie values. Our method improves upon previous caloric estimation methods by over 10\% and 30 kCal in terms of MAPE and MAE respectively.Comment: Accepted for Madima'23 in ACM Multimedi

Bariatric surgery and brain health: A longitudinal observational study investigating the effect of surgery on cognitive function and gray matter volume

Dietary modifications leading to weight loss have been suggested as a means to improve brain health. In morbid obesity, bariatric surgery (BARS)—including different procedures, such as vertical sleeve gastrectomy (VSG), gastric banding (GB), or Roux-en-Y gastric bypass (RYGB) surgery—is performed to induce rapid weight loss. Combining reduced food intake and malabsorption of nutrients, RYGB might be most effective, but requires life-long follow-up treatment. Here, we tested 40 patients before and six months after surgery (BARS group) using a neuropsychological test battery and compared them with a waiting list control group. Subsamples of both groups underwent structural MRI and were examined for differences between surgical procedures. No substantial differences between BARS and control group emerged with regard to cognition. However, larger gray matter volume in fronto-temporal brain areas accompanied by smaller volume in the ventral striatum was seen in the BARS group compared to controls. RYGB patients compared to patients with restrictive treatment alone (VSG/GB) had higher weight loss, but did not benefit more in cognitive outcomes. In sum, the data of our study suggest that BARS might lead to brain structure reorganization at long-term follow-up, while the type of surgical procedure does not differentially modulate cognitive performance

Validation of a recommender system for prompting omitted foods in online dietary assessment surveys

Recall assistance methods are among the key aspects that improve the accuracy of online dietary assessment surveys. These methods still mainly rely on experience of trained interviewers with nutritional background, but data driven approaches could improve cost-efficiency and scalability of automated dietary assessment. We evaluated the effectiveness of a recommender algorithm developed for an online dietary assessment system called Intake24, that automates the multiple-pass 24-hour recall method. The recommender builds a model of eating behavior from recalls collected in past surveys. Based on foods they have already selected, the model is used to remind respondents of associated foods that they may have omitted to report. The performance of prompts generated by the model was compared to that of prompts hand-coded by nutritionists in two dietary studies. The results of our studies demonstrate that the recommender system is able to capture a higher number of foods omitted by respondents of online dietary surveys than prompts hand-coded by nutritionists. However, the considerably lower precision of generated prompts indicates an opportunity for further improvement of the system

GourmetNet: Food Segmentation Using Multi-Scale Waterfall Features With Spatial and Channel Attention

Deep learning and Computer vision are extensively used to solve problems in wide range of domains from automotive and manufacturing to healthcare and surveillance. Research in deep learning for food images is mainly limited to food identification and detection. Food segmentation is an important problem as the first step for nutrition monitoring, food volume and calorie estimation. This research is intended to expand the horizons of deep learning and semantic segmentation by proposing a novel single-pass, end-to-end trainable network for food segmentation. Our novel architecture incorporates both channel attention and spatial attention information in an expanded multi-scale feature representation using the WASPv2 module. The refined features will be processed with the advanced multi-scale waterfall module that combines the benefits of cascade filtering and pyramid representations without requiring a separate decoder or postprocessing

Visual analytics methods for shape analysis of biomedical images exemplified on rodent skull morphology

In morphometrics and its application fields like medicine and biology experts are interested in causal relations of variation in organismic shape to phylogenetic, ecological, geographical, epidemiological or disease factors - or put more succinctly by Fred L. Bookstein, morphometrics is "the study of covariances of biological form". In order to reveal causes for shape variability, targeted statistical analysis correlating shape features against external and internal factors is necessary but due to the complexity of the problem often not feasible in an automated way. Therefore, a visual analytics approach is proposed in this thesis that couples interactive visualizations with automated statistical analyses in order to stimulate generation and qualitative assessment of hypotheses on relevant shape features and their potentially affecting factors. To this end long established morphometric techniques are combined with recent shape modeling approaches from geometry processing and medical imaging, leading to novel visual analytics methods for shape analysis. When used in concert these methods facilitate targeted analysis of characteristic shape differences between groups, co-variation between different structures on the same anatomy and correlation of shape to extrinsic attributes. Here a special focus is put on accurate modeling and interactive rendering of image deformations at high spatial resolution, because that allows for faithful representation and communication of diminutive shape features, large shape differences and volumetric structures. The utility of the presented methods is demonstrated in case studies conducted together with a collaborating morphometrics expert. As exemplary model structure serves the rodent skull and its mandible that are assessed via computed tomography scans