Hierarchical Attention Network for Visually-aware Food Recommendation

Food recommender systems play an important role in assisting users to identify the desired food to eat. Deciding what food to eat is a complex and multi-faceted process, which is influenced by many factors such as the ingredients, appearance of the recipe, the user's personal preference on food, and various contexts like what had been eaten in the past meals. In this work, we formulate the food recommendation problem as predicting user preference on recipes based on three key factors that determine a user's choice on food, namely, 1) the user's (and other users') history; 2) the ingredients of a recipe; and 3) the descriptive image of a recipe. To address this challenging problem, we develop a dedicated neural network based solution Hierarchical Attention based Food Recommendation (HAFR) which is capable of: 1) capturing the collaborative filtering effect like what similar users tend to eat; 2) inferring a user's preference at the ingredient level; and 3) learning user preference from the recipe's visual images. To evaluate our proposed method, we construct a large-scale dataset consisting of millions of ratings from AllRecipes.com. Extensive experiments show that our method outperforms several competing recommender solutions like Factorization Machine and Visual Bayesian Personalized Ranking with an average improvement of 12%, offering promising results in predicting user preference for food. Codes and dataset will be released upon acceptance

Towards Multi-Language Recipe Personalisation and Recommendation

Multi-language recipe personalisation and recommendation is an under-explored field of information retrieval in academic and production systems. The existing gaps in our current understanding are numerous, even on fundamental questions such as whether consistent and high-quality recipe recommendation can be delivered across languages. In this paper, we introduce the multi-language recipe recommendation setting and present grounding results that will help to establish the potential and absolute value of future work in this area. Our work draws on several billion events from millions of recipes and users from Arabic, English, Indonesian, Russian, and Spanish. We represent recipes using a combination of normalised ingredients, standardised skills and image embeddings obtained without human intervention. In modelling, we take a classical approach based on optimising an embedded bi-linear user-item metric space towards the interactions that most strongly elicit cooking intent. For users without interaction histories, a bespoke content-based cold-start model that predicts context and recipe affinity is introduced. We show that our approach to personalisation is stable and easily scales to new languages. A robust cross-validation campaign is employed and consistently rejects baseline models and representations, strongly favouring those we propose. Our results are presented in a language-oriented (as opposed to model-oriented) fashion to emphasise the language-based goals of this work. We believe that this is the first large-scale work that comprehensively considers the value and potential of multi-language recipe recommendation and personalisation as well as delivering scalable and reliable models.Comment: 5 table

Towards Automated Recipe Genre Classification using Semi-Supervised Learning

Sharing cooking recipes is a great way to exchange culinary ideas and provide instructions for food preparation. However, categorizing raw recipes found online into appropriate food genres can be challenging due to a lack of adequate labeled data. In this study, we present a dataset named the ``Assorted, Archetypal, and Annotated Two Million Extended (3A2M+) Cooking Recipe Dataset" that contains two million culinary recipes labeled in respective categories with extended named entities extracted from recipe descriptions. This collection of data includes various features such as title, NER, directions, and extended NER, as well as nine different labels representing genres including bakery, drinks, non-veg, vegetables, fast food, cereals, meals, sides, and fusions. The proposed pipeline named 3A2M+ extends the size of the Named Entity Recognition (NER) list to address missing named entities like heat, time or process from the recipe directions using two NER extraction tools. 3A2M+ dataset provides a comprehensive solution to the various challenging recipe-related tasks, including classification, named entity recognition, and recipe generation. Furthermore, we have demonstrated traditional machine learning, deep learning and pre-trained language models to classify the recipes into their corresponding genre and achieved an overall accuracy of 98.6\%. Our investigation indicates that the title feature played a more significant role in classifying the genre

XAIR: A Framework of Explainable AI in Augmented Reality

Explainable AI (XAI) has established itself as an important component of AI-driven interactive systems. With Augmented Reality (AR) becoming more integrated in daily lives, the role of XAI also becomes essential in AR because end-users will frequently interact with intelligent services. However, it is unclear how to design effective XAI experiences for AR. We propose XAIR, a design framework that addresses "when", "what", and "how" to provide explanations of AI output in AR. The framework was based on a multi-disciplinary literature review of XAI and HCI research, a large-scale survey probing 500+ end-users' preferences for AR-based explanations, and three workshops with 12 experts collecting their insights about XAI design in AR. XAIR's utility and effectiveness was verified via a study with 10 designers and another study with 12 end-users. XAIR can provide guidelines for designers, inspiring them to identify new design opportunities and achieve effective XAI designs in AR.Comment: Proceedings of the 2023 CHI Conference on Human Factors in Computing System

意外性のあるレシピを推薦するエージェントの提案

毎日の食事のレシピを考えることは非常に大変なことである．近年では独自のレシピを自由に書き込むことができる投稿型レシピサイトが多数存在しており，サイト内のレシピ数，ユーザ数は年々増加している．投稿型レシピサイトには，普通とは多少異なる食材を用いた意外性のあるレシピが存在するが，通常の検索機能を使って発見するのは困難である．そこで，本研究では投稿型レシピサイトから意外性のあるレシピを抽出するための推薦エージェントを提案する．このレシピ推薦エージェントはTF-IDFの考えを応用したRF-IIF（Recipe Frequency-Inverse Ingredient Frequency）を利用し，ユーザから指定された料理カテゴリーにおける食材の希少度と一般度から意外度を算出する．次にレシピに出現する食材の意外度からレシピの意外度を算出するが，各レシピの料理カテゴリーを誤判定すると，普通のレシピが意外レシピと誤判断されてしまうため，別カテゴリーのレシピをいかに除外するかが重要である．最後に，レシピ間の類似度を計り類似したレシピを除去することで多様性に富んだ意外性のあるレシピを抽出する．アンケートによる評価を実施し，提案するレシピ推薦エージェントの有用性を示した．Many surprising recipes that have different ingredients from normal recipes exist in user-generated recipe sites. However, we cannot find surprising recipes by using the search function. In this paper, we propose a method of extracting surprising recipes from the user-generated recipe sites. We calculate the surprising value of ingredients and recipes by using RF-IIF. Then, we remove the redundancy of recipes that have high surprising values. Finally, we extract surprising recipes of the dish category specified by the user. In the evaluation experiment, we conducted a questionnaire about each surprising recipe. As a result, we showed the usefulness of our proposed method