1 research outputs found
Scraping Social Media Photos Posted in Kenya and Elsewhere to Detect and Analyze Food Types
Monitoring population-level changes in diet could be useful for education and
for implementing interventions to improve health. Research has shown that data
from social media sources can be used for monitoring dietary behavior. We
propose a scrape-by-location methodology to create food image datasets from
Instagram posts. We used it to collect 3.56 million images over a period of 20
days in March 2019. We also propose a scrape-by-keywords methodology and used
it to scrape ~30,000 images and their captions of 38 Kenyan food types. We
publish two datasets of 104,000 and 8,174 image/caption pairs, respectively.
With the first dataset, Kenya104K, we train a Kenyan Food Classifier, called
KenyanFC, to distinguish Kenyan food from non-food images posted in Kenya. We
used the second dataset, KenyanFood13, to train a classifier KenyanFTR, short
for Kenyan Food Type Recognizer, to recognize 13 popular food types in Kenya.
The KenyanFTR is a multimodal deep neural network that can identify 13 types of
Kenyan foods using both images and their corresponding captions. Experiments
show that the average top-1 accuracy of KenyanFC is 99% over 10,400 tested
Instagram images and of KenyanFTR is 81% over 8,174 tested data points.
Ablation studies show that three of the 13 food types are particularly
difficult to categorize based on image content only and that adding analysis of
captions to the image analysis yields a classifier that is 9 percent points
more accurate than a classifier that relies only on images. Our food trend
analysis revealed that cakes and roasted meats were the most popular foods in
photographs on Instagram in Kenya in March 2019.Comment: Another version of the paper was submitted to the ACM International
Conference on Multimedia (ACMMM2019