Deep learning in food category recognition

Integrating artificial intelligence with food category recognition has been a field of interest for research for the past few decades. It is potentially one of the next steps in revolutionizing human interaction with food. The modern advent of big data and the development of data-oriented fields like deep learning have provided advancements in food category recognition. With increasing computational power and ever-larger food datasets, the approach’s potential has yet to be realized. This survey provides an overview of methods that can be applied to various food category recognition tasks, including detecting type, ingredients, quality, and quantity. We survey the core components for constructing a machine learning system for food category recognition, including datasets, data augmentation, hand-crafted feature extraction, and machine learning algorithms. We place a particular focus on the field of deep learning, including the utilization of convolutional neural networks, transfer learning, and semi-supervised learning. We provide an overview of relevant studies to promote further developments in food category recognition for research and industrial applicationsMRC (MC_PC_17171)Royal Society (RP202G0230)BHF (AA/18/3/34220)Hope Foundation for Cancer Research (RM60G0680)GCRF (P202PF11)Sino-UK Industrial Fund (RP202G0289)LIAS (P202ED10Data Science Enhancement Fund (P202RE237)Fight for Sight (24NN201);Sino-UK Education Fund (OP202006)BBSRC (RM32G0178B8

Making the best use of new technologies in the National Diet and Nutrition Survey: a review

.Background Dietary assessment is of paramount importance for public health monitoring. Currently in the UK, the population’s diets are examined by the National Diet and Nutrition Survey Rolling Programme (NDNS RP). In the survey, diet is assessed by a four-day paper-based dietary diary, with accompanying interviews, anthropometric measurements and blood and urine sampling. However, there is growing interest worldwide in the potential for new technologies to assist in data collection for assessment of dietary intake. Published literature reviews have identified the potential of new technologies to improve accuracy, reduce costs, and reduce respondent and researcher burden by automating data capture and the nutritional coding process. However, this is a fast-moving field of research, with technologies developing at a rapid pace, and an updated review of the potential application of new technologies in dietary assessment is warranted. This review was commissioned to identify the new technologies employed in dietary assessment and critically appraise their strengths and limitations in order to recommend which technologies, if any, might be suitable to develop for use in the NDNS RP and other UK population surveys. Objectives The overall aim of the project was to inform the Department of Health of the range of new technologies currently available and in development internationally that have potential to improve, complement or replace the methods used in the NDNS RP. The specific aims were: to generate an itinerary of new and emerging technologies that may be suitable; to systematically review the literature and critically appraise new technologies; and to recommend which of these new technologies, if any, would be appropriate for future use in the NDNS RP. To meet these aims, the project comprised two main facets, a literature review and qualitative research. Literature review data sources The literature review incorporated an extensive search of peer-reviewed and grey literature. The following sources were searched: Cochrane Database of Systematic Reviews (CDSR), Database of Abstracts of Reviews of Effectiveness (DARE), Web of Science Core Collection, Ovid MEDLINE, Ovid MEDLINE In-Process, Embase, NHS EED (Economic Evaluation Database), National Cancer Institute (NCI) Dietary Assessment Calibration/Validation Register, OpenGrey, EPPI Centre (TRoPHI), conference proceedings (ICDAM 2012, ISBNPA 2013, IEEE Xplore, Nutrition Society Irish Section and Summer Meetings 2014), recent issues of journals (Journal of Medical Internet Research, International Journal of Medical Informatics), grants registries (ClinicalTrials.gov, BBSRC, report), national surveys, and mobile phone application stores. In addition, hand-searching of relevant citations was performed. The search also included solicitation of key authors in the field to enquire about Making the best use of new technologies in the NDNS: a review 4 as-yet unpublished articles or reports, and a Bristol Online Survey publicised via social media, society newsletters and meetings. Literature review eligibility criteria Records were screened for eligibility using a three-stage process. Firstly, keyword searches identified obviously irrelevant titles. Secondly, titles and abstracts were screened against the eligibility criteria, following which full-text copies of papers were obtained and, in the third stage of screening, examined against the criteria. Two independent reviewers screened each record at each stage, with discrepancies referred to a third reviewer. Eligibility criteria were pre-specified and agreed by the project Steering Group (Section 1.6). Eligible records included: studies involving technologies, new to the NDNS RP, which can be used to automate or assist the collection of food consumption data and the coding of foods and portion sizes, currently available or beta versions, public domain or commercial; studies that address the development, features, or evaluation of new technology; technologies appropriate for the requirements of the NDNS RP in terms of nutritional analysis, with capacity to collect quantifiable consumption data at the food level; primary sources of information on a particular technology; and journal articles published since the year 2000 or grey literature available from 2011 onwards. The literature search was not limited to Englishlanguage publications, which are included in the itinerary, although data were not extracted from non-English studies. Literature synthesis and appraisal New technologies were categorised into eleven types of technology, and an itinerary was generated of tools falling under each category type. Due to the volume of eligible studies identified by the literature searches, data extraction was limited to the literature focussing on selected exemplar tools of five technology categories (web-based diet diary, web-based 24- hour recall, handheld devices (personal digital assistants and mobile phones), nonautomated cameras to complement traditional methods, and non-automated cameras to replace traditional methods). For each category, at least two exemplars were chosen, and all studies involving the exemplar were included in data extraction and synthesis. Exemplars were selected on the basis of breadth of evidence available, using pre-specified criteria agreed by the Steering Group. Data were extracted by a single reviewer and an evidence summary collated for each exemplar. A quality appraisal checklist was developed to assess the quality of validation studies. The checklist was piloted and applied by two independent reviewers. Studies were not excluded on the basis of quality, but study quality was taken into account when judging the strength of evidence. Due to the heterogeneity of the literature, meta-analyses were not performed. References were managed and screened using the EPPI Reviewer 4 systematic review software. EPPI Reviewer was also used to extract data

Advancement in Dietary Assessment and Self-Monitoring Using Technology

Although methods to assess or self-monitor intake may be considered similar, the intended function of each is quite distinct. For the assessment of dietary intake, methods aim to measure food and nutrient intake and/or to derive dietary patterns for determining diet-disease relationships, population surveillance or the effectiveness of interventions. In comparison, dietary self-monitoring primarily aims to create awareness of and reinforce individual eating behaviours, in addition to tracking foods consumed. Advancements in the capabilities of technologies, such as smartphones and wearable devices, have enhanced the collection, analysis and interpretation of dietary intake data in both contexts. This Special Issue invites submissions on the use of novel technology-based approaches for the assessment of food and/or nutrient intake and for self-monitoring eating behaviours. Submissions may document any part of the development and evaluation of the technology-based approaches. Examples may include: web adaption of existing dietary assessment or self-monitoring tools (e.g., food frequency questionnaires, screeners) image-based or image-assisted methods mobile/smartphone applications for capturing intake for assessment or self-monitoring wearable cameras to record dietary intake or eating behaviours body sensors to measure eating behaviours and/or dietary intake use of technology-based methods to complement aspects of traditional dietary assessment or self-monitoring, such as portion size estimation

Sustaining dairy

Dairy in Europe has undergone many changes in the last few years—the abolition of milk production quotas being a fundamental one. This study explores these changes in relation to the sustained social and environmental viability of the sector and how dairy processors' sustainability programs are a part of that. Regime change as outlined in transition theory enhanced through a sociological approach on actors informed this research. More specifically, the notion of obligatory passage points was used to explore the mechanisms through which dominant actors make certain actions mandatory and reify their status as indispensable. The thesis consists of three case studies: the dairy sectors in the Netherlands, Ireland and the United Kingdom. The cases trace the evolution of all sectors since the post-war era, outlining the dominant logic that has guided its development. The sustainability programs of three dairy processors—located in each of the case countries—are also part of the analysis. Data was collected through document analysis and semi-structured interviews. The analysis shows that the post-war logic based on the increase of scale and intensification of dairying has continued to shape the development of the sector through today. While the visible impacts of intensive dairy have led to adaptations to the dominant rules and practices, these changes have not been fundamental in nature. The analysis of dairy processors and their sustainability programs revealed that these programs can be an additional tool for compliance to legal standards and the alleviation of pressing societal concerns. However, processors address social and environmentally relevant dairy-related challenges when an effective link to profit can be established. These programs have been unable to ensure that the dairy sector operates within established environmental limits and societal expectations, while providing a stable livelihood for farmers. This research contributes to the understanding of sustainability (agri-food) transitions by identifying the mechanisms through which the regime adapts to the shifting environment and dominant actors strive for their own continuity. It also adds to the debate about the role that incumbent actors can have in sustainability transitions—their involvement is important but they are unable to guide such processes. This study advances the empirical ground in sustainability transition studies by focusing on systems in which change is less likely to be technologically driven and where social change plays a larger role. Finally, this thesis connects past development, current challenges, and present engagement in a discussion about the future development of the dairy sector; this adds to the further conceptualization of the complexity and co-evolutionary nature of sustainability transitions.</p

Exploring the potential of using mobile applications in diabetes management

Background Diabetes mellitus is a common chronic disease and a leading cause of morbidity, complications and mortality worldwide. The number of people living with diabetes is projected to rise sharply over the forthcoming decades. Diabetes care is complex and can overburden clinicians and nurses. There is a need for innovative, flexible and cost-effective technologies to enable successful diabetes management. This thesis explores the opportunities and challenges of the mobile application (app) technology as a potential tool to support diabetes care and management. Purpose The purpose was to develop and evaluate a mobile app that supports healthcare professionals (HCPs) in clinical decision-making. Methods A mixed-methods approach was used following the user-centred design (UCD) framework for the design and implementation of all studies. Quantitative and qualitative systematic reviews of studies reporting the use of mobile apps to support diabetes management were undertaken to identify, appraise and summarise available research evidence. An interview study was carried out with diabetes specialist nurses (DSNs), to explore their experiences and views, and to identify user requirements for apps. Lastly, a guidelines-based mobile clinical decision-support app was developed and tested with junior doctors and DSNs in a controlled environment to evaluate its usability and impact on adherence to clinical guidelines, and to explore how participants experienced the app and their suggestions for improvements. Results Both reviews found that the existing evidence base for mobile apps is weak and inadequate to draw conclusions about the impact of their use as interventions in diabetes management. The interview study identified that nurses lack experience in using apps in clinical practice, even though they believed it could facilitate and support their work. ‘Diabetes & CKD’, a simple mobile decision-support app, has been designed and built for the study to assist HCPs in management of patients with diabetes and kidney disease and was tested by 39 junior doctors and 3 DSNs. It had no impact on the accuracy of decisions. Feedback from participants after the pilot session and usability testing indicated a wish to integrate such apps into their clinical practice with a strong willingness to use them in the future. Conclusions Application of UCD methods was efficient as the app was well-accepted by both DSNs and junior doctors. Despite the positive views and the strong willingness to use such apps, they are not widely used. There is a need to regulate the use of medical apps in clinical practice. Further research with rigorous methodology is required upon which policymakers and practitioners can base their decision-making

Pervasive Quantied-Self using Multiple Sensors

abstract: The advent of commercial inexpensive sensors and the advances in information and communication technology (ICT) have brought forth the era of pervasive Quantified-Self. Automatic diet monitoring is one of the most important aspects for Quantified-Self because it is vital for ensuring the well-being of patients suffering from chronic diseases as well as for providing a low cost means for maintaining the health for everyone else. Automatic dietary monitoring consists of: a) Determining the type and amount of food intake, and b) Monitoring eating behavior, i.e., time, frequency, and speed of eating. Although there are some existing techniques towards these ends, they suffer from issues of low accuracy and low adherence. To overcome these issues, multiple sensors were utilized because the availability of affordable sensors that can capture the different aspect information has the potential for increasing the available knowledge for Quantified-Self. For a), I envision an intelligent dietary monitoring system that automatically identifies food items by using the knowledge obtained from visible spectrum camera and infrared spectrum camera. This system is able to outperform the state-of-the-art systems for cooked food recognition by 25% while also minimizing user intervention. For b), I propose a novel methodology, IDEA that performs accurate eating action identification within eating episodes with an average F1-score of 0.92. This is an improvement of 0.11 for precision and 0.15 for recall for the worst-case users as compared to the state-of-the-art. IDEA uses only a single wrist-band which includes four sensors and provides feedback on eating speed every 2 minutes without obtaining any manual input from the user.Dissertation/ThesisDoctoral Dissertation Computer Engineering 201

Frequency statistics of words used in Japanese food records of FoodLog

All in-text references underlined in blue are linked to publications on ResearchGate, letting you access and read them immediately

The effect of telemonitoring interventions on health outcomes in individuals with type 2 diabetes, heart failure and coronary artery disease : Results from three randomized controlled trials

Kroonisten sairauksien hoitoon vaaditaan tehokkaampia ratkaisuja. Informaatio- ja kommunikaatioteknologiat tarjoavat mahdollisuuksia pitkäaikaissairaiden potilaiden hoidon kehittämiseksi ja paremman hoitotasapainon saavuttamiseksi. Tässä väitöskirjassa esitellään kolme satunnaistettua kontrolloitua tutkimusta, joissa arvioitiin etämittausinterventioiden vaikuttavuutta terveyteen liittyviin muuttujiin tyypin 2 diabeetikoilla, sydämen vajaatoimintapotilailla sekä sepelvaltimotautipotilailla. Kaikissa tutkimuksissa etämittausinterventio käsitti pitkäaikaissairauteen liittyvien terveysparametrien viikoittaista seurantaa sekä mittausarvojen välittämisen terveydenhuollon ammattilaisten käyttöön matkapuhelimen avulla. Etämittausten tueksi jokaisessa tutkimuksessa interventioon sisältyi erilainen potilaan päätöksentukikomponentti. Tutkimus I, Mobile Sipoo -tutkimus, toteutettiin Sipoon terveyskeskuksessa. Tutkimuksessa oli mukana 51 tyypin 2 diabeetikkoa, joita seurattiin 10 kuukauden ajan. Etämittausryhmässä potilaat ohjeistettiin mittaamaan painoa, verenpainetta, askelia ja/tai verensokeria. Mittaustuloksiin perustuen potilaat saivat reaaliaikaisia palauteviestejä omahoidon tukemiseksi. Käyttäytymisteoriaan pohjautuvat palauteviestit sisälsivät tiivistettyä tietoa potilaan pitkäaikaismittauksista, motivoivat muutokseen ja tarjosivat käytännön keinoja muutoksen saavuttamiseksi. Tutkimus I:n primääritavoite oli potilaiden glykohemoglobiinin ja verenpaineen alentaminen. Tutkimus II, Heart at Home -tutkimus, toteutettiin sydän ja keuhkokeskuksen poliklinikalla Helsingin yliopistollisessa sairaalassa. Tutkimukseen osallistui 94 sydämen vajaatoimintapotilasta. Etämittausryhmän potilaat ohjeistettiin viikoittain mittaamaan painoa, verenpainetta, sykettä sekä sydämen vajaatoimintaan liittyviä oireita. Etämittauksiin perustuen potilaat saivat reaaliaikaisen palauteviestin, joka sisälsi lyhyen palautteen suhteessa asetettuihin tavoitearvoihin sekä kuvaajan mittaushistoriasta. Tutkimuksen II primääritavoite oli sydämen vajaatoimintaan liittyvien sairaalakäyntien määrän vähentäminen. Tutkimuksen kesto oli kuusi kuukautta. Tutkimus III, Renewing Health -tutkimus, toteutettiin Etelä-Karjalan sairaanhoitopiirin terveyskeskuksissa. Tutkimukseenosallistui 519 potilasta; 250 tyypin 2 diabeetikkoa, 227 sepelvaltimotautipotilasta ja 42 sydämen vajaatoimintapotilasta. Potilaat ohjeistettiin mittamaan painoa, verenpainetta, askelia ja/tai verensokeria viikoittain. Etämittausten lisäksi osallistujat saivat henkilökohtaisen terveysvalmentajan, joka oli heihin puhelimitse yhteydessä 4–6 viikon välein. Terveysvalmennuksen avulla pyrittiin voimaannuttamaan potilasta sekä tarjoamaan keinoja pitkäaikaissairauden paremman hoitotasapainon saavuttamiseksi. Primääritavoite tutkimus III:ssa oli parantaa potilaiden elämänlaatua sekä tyypin 2 diabeetikoiden osalta madaltaa veren glykohemoglobiinia, jota mitattiin HbA1c:n avulla. Kaikissa kolmessa tutkimuksessa kontrolliryhmä sai tavanomaista hoitoa. Koska terveysvaikutusten saavuttaminen edellyttää potilaiden sitoutumista interventioon, potilaiden sitoutumista etämittausinterventioihin tutkittiin yksityiskohtaisesti lokitietojen avulla. Sitoutuminen määriteltiin niiden viikkojen prosentuaalisena osuutena, jotka sisälsivät vähintään yhden etämittauksen. Tilastolliset analyysit toteutettiin intention-to-treat periaatteiden mukaisesti. Sitoutuminen viikoittaisiin etämittauksiin vaihteli tutkimusten välillä kohtuullisesta (III) korkeaan (I ja II). Potilaat toteuttivat etämittauksia keskimäärin 93 % (I), 90 % (II) ja 66 % (III) viikoista. Sitoutumisen aste ei huomattavasti laskenut seuranta-ajan kuluessa. Etämittausinterventio tutkimuksessa I johti glykohemoglobiinin tilastollisesti merkitsevään vähenemiseen. HbA1c oli 0,44 prosenttiyksikköä matalampi etämittausryhmässä. Verenpaineessa ei ollut eroa ryhmien välillä. Tutkimuksessa II etämittausinterventio ei vähentänyt sydämen vajaatoimintaan liittyvien sairaalapäivien määrää. Itse asiassa terveydenhuollon resurssien käyttö lisääntyi; sydänhoitajan vastanottoaikoja ja puhelinkontakteja oli 2–5 kertaa enemmän etämittausryhmässä. Etämittausryhmässä oli myös enemmän ennakoimattomia poliklinikkakäyntejä. Etämittausinterventio tutkimuksessa III, jossa yhdistettiin etämittaukset sekä henkilökohtainen terveysvalmennus, ei johtanut elämänlaadun parantamiseen. Myöskään glykohemoglobiini ei vähentynyt tyypin 2 diabeetikoilla. Kaikissa kolmessa tutkimuksessa potilaiden sitoutuminen viikoittaisten etämittausten tekemiseen oli suhteellisen korkea eikä se vähentynyt ajan kuluessa. Nämä havainnot viittaavat siihen, että viikoittainen etämittaaminen on toteutettavissa. Korkea sitoutuminen ei kuitenkaan johtanut positiivisiin terveysvaikutuksiin; kahdessa tutkimuksista ei havaittu positiivisia terveysvaikutuksia, vaikka potilaat tekivät aktiivisesti etämittauksia. Etämittaaminen itsessään ei välttämättä ole riittävää kroonisten sairauksien hoitotulosten parantamiseksi. Positiivisia terveysvaikutuksia saavutettiin ainoastaan tutkimus I:ssä, jossa osallistujat saivat etämittaustensa pohjalta käyttäytymisteoriaan perustuvia palauteviestejä. Palauteviestit sisälsivät tiivistettyä tietoa potilaan pitkäaikaismittauksista sekä motivoivat muutokseen ja tarjosivat keinoja muutoksen saavuttamiseksi. Tämän työn tulokset ovat linjassa aiempien tutkimustulosten kanssa, jotka korostavat käyttäytymisteorioihin perustuvien interventioiden tärkeyttä. Väitöskirjan yhteenvetona todetaan, että etämittauksista saatavalla palautteella on keskeinen rooli. On tärkeää antaa reaaliaikaisuutta ja sisältörikasta palautetta, jotta voidaan tukea pitkäaikaissairaiden potilaiden omahoitoa ja saavuttaa positiivisia terveysvaikutuksia. Etämittausinterventiot saattavat kasvattaa terveydenhuollon resurssien käyttöä, erityisesti sairaanhoitajien työkuormaa, joten tarvittavat resurssit tulisi varmistaa ja etämittausintervention mahdolliset hyödyt punnita muiden tulosten valossa. Lisäksi on tärkeää suunnitella etämittausinterventiot huolellisesti ja kohdistaa ne potilaille, jotka sitoutuvat käyttöön ja todennäköisesti hyötyvät interventiosta.The increasing burden of chronic conditions creates a need to develop more effective approaches to improve management and health outcomes of chronic conditions. Information- and communication technologies provide tools to promote the management of chronic conditions. This thesis presents three randomized controlled trials that assessed the effect of telemonitoring interventions on health outcomes in individuals with type 2 diabetes (T2D), heart failure (HF) and coronary artery diseases (CAD). In all studies, telemonitoring involved self-monitoring of chronic condition related health parameters on a weekly basis, and sharing these data with healthcare professionals using a mobile phone. In addition, each study had a specific patient decision support component linked to the telemonitoring data. Study I, the Mobile Sipoo study, was conducted at the healthcare center of Sipoo, and included 51 patients with T2D who were followed for 10 months. The participants in the intervention arm recorded their weight, blood pressure, steps and/or blood glucose. These data were further linked with an automatic feedback system that provided patients with real-time, behavioral theory-based feedback messages that summarized the telemonitoring data, and motivated patients and provided them with behavioral skills to strengthen their self-management practices. The primary aim of Study I was to improve glycemic control, measured as HbA1c, and decrease blood pressure. Study II, the Heart at Home study, was conducted at the Cardiology Outpatient Clinic of Helsinki University Central Hospital and included 94 patients with systolic heart failure. The intervention participants were instructed to monitor their weight, blood pressure, heart rate and symptoms, and the data were linked with real-time, short feedback messages that summarized the data in relation to pre-specified individual target values. The primary aim of Study II was to reduce HFrelated hospitalizations during the 6-month follow-up. Study III, the Renewing Health study, was conducted at healthcare centers in South Karelia and included 519 patients with T2D (n=250), CAD (n=227) or HF (n=42). Participants monitored their weight, blood pressure, steps and/or blood glucose. In addition, each participant received individual health coaching calls every 4 to 6 weeks to empower the patients and to teach them appropriate self-management skills tailored for each condition. The primary aim of Study III was to improve the health-related quality of life in all patients, and to reduce HbA1c in patients with T2D. In all studies the control group consisted of patients receiving standard care. As adherence is a prerequisite for achieving the intervention effects, adherence to the telemonitoring interventions was investigated in detail using the log files, and was defined as a percentage of weeks including at least one health parameter recorded. Analyses were performed according to the intention-to-treat principle. In all studies, adherence to the weekly telemonitoring was moderate (Study III) to high (Study I and II) with the median percentage of adherent weeks being 93%, 90% and 66% in Study I, II and III, respectively, without major attrition in time. The telemonitoring intervention in Study I demonstrated a statistically significant improvement in glycemic control by reducing HbA1c by 0.44 percentage points. However, the blood pressure levels did not differ between the treatment arms. In study II, the telemonitoring intervention did not significantly reduce HF-related hospital admissions but, in fact, the utilization of the healthcare resources increased with the number of appointments and calls to the HFnurse being 2–5 times higher in the telemonitoring arm, and more unplanned visits to the cardiology clinic. A combination of telemonitoring and health coaching in Study III did not improve the health-related quality of life in patients with T2D, HF or CAD. Neither did it reduce HbA1c in patients with T2D. Sustained, fairly high adherence seen in all studies suggests that weekly telemonitoring of health-related parameters is feasible. Nevertheless, high adherence does not guarantee positive health effects. Two of the studies showed no improvement in health outcomes although participants were actively involved with telemonitoring. This indicates that telemonitoring as such might not be effective in improving chronic disease outcomes. Positive health effects were seen only in study I, where the individuals received real-time, behavioral-theory based feedback messages that summarized the TM data, and motivated and guided patients to take actions to promote self-management. Putting the results together, the findings of this work support earlier research findings on the importance of grounding interventions on behavioral theory and providing timely feedback with enriched content to promote self-management and further improve the health outcomes of individuals with chronic conditions. However, telemonitoring interventions might increase the use of healthcare resources, especially personnel resources by requiring more time of the responsible nurse. Thus, sufficient resources should be ensured and the benefits gained evaluated in the light of other findings. Telemonitoring interventions should be carefully designed to target patients who are likely to adhere to them and likely to benefit from such interventions