In Switzerland, 2.8 million tons of food are lost or wasted across all stages of food production - every year. This equates to approximately 330 kg of food waste per person. By analysing and classifying discarded food with a smart waste analysis system combined with machine learning, valuable insights can be gained and the amount of wasted food can be significantly reduced. In this paper, we present how we have developed an embedded system which helps to solve this task.
The embedded system operates in a decentralized manner: It captures an image every time food is thrown into a bin. The discarded food is identified and classified with machine learning algorithms. This provides a detailed insight into the structure of food waste for customers, e.g. restaurants or canteens.
We implemented the machine learning algorithm directly on the embedded systems control unit. We found that running machine learning directly on embedded devices has many advantages compared to running them in the cloud: We saved significant amounts of cloud storage and reduced power consumption by up to a factor 100. In addition, privacy was increased and required bandwidth reduced because only the machine learning results are forwarded to the cloud, not the full data

Andermatt, Philipp

Mazloumian, Amin

Rosenthal, Matthias

Zingg, Raphael

ZHAW digitalcollection

Smart Food Waste Management - Embedded MachineLearning vs Cloud Based SolutionsRaphael Zingg1, Philipp Andermatt2, Amin Mazloumian1 and Matthias Rosenthal11Institute of Embedded Systems, Zurich University of Applied Sciences (ZHAW), Winterthur, Switzerland2KITRO SA, Zürich, SwitzerlandAbstractIn Switzerland, 2.8 million tons of food are lost or wasted across all stages of food production - everyyear. This equates to approximately 330 kg of food waste per person. By analysing and classifyingdiscarded food with a smart waste analysis system combined with machine learning, valuable insightscan be gained and the amount of wasted food can be significantly reduced. In this paper, we present howwe have developed an embedded system which helps to solve this task. The embedded system operatesin a decentralized manner: It captures an image every time food is thrown into a bin. The discardedfood is identified and classified with machine learning algorithms. This provides a detailed insight intothe structure of food waste for customers, e.g. restaurants or canteens. We implemented the machinelearning algorithm directly on the embedded systems control unit. We found that running machinelearning directly on embedded devices has many advantages compared to running them in the cloud:We saved significant amounts of cloud storage and reduced power consumption by up to a factor 100. Inaddition, privacy was increased and required bandwidth reduced because only the machine learningresults are forwarded to the cloud, not the full data.KeywordsFood Waste, IoT, Embedded Machine Learning, TensorRT, Embedded Inference1. IntroductionIn Switzerland, 2.8 million tons of edible food is wasted across all stages of the food industry.This equates to approximately 330kg of avoidable food waste per person and year, or 37%of agricultural production, i.e. food produced in Switzerland and abroad for consumption inSwitzerland [1]. To produce all this wasted food, half of Switzerland’s total farm land would berequired. Moreover, this huge amount of wasted food is responsible for 27% of Switzerland’senvironmental impact [2]. The sectors retail, gastronomy and households contribute a totalof 1.39 million tons of wasted food per year [2]. Most of this food waste is avoidable andcomes from cooking surplus and consumer preferences (aesthetics) [3]. In addition to theenvironmental costs, it is estimated that the gastronomy sector in Switzerland loses 1 BillionSwiss Francs due to wasted food per year [2].FTAL 2021, October 28–29, 2021, Lugano, SwitzerlandEnvelope-Open zing@zhaw.ch (R. Zingg); philipp.andermatt@kitro.ch (P. Andermatt); mazl@zhaw.ch (A. Mazloumian);rosn@zhaw.ch (M. Rosenthal)GLOBE https://www.kitro.ch/ (P. Andermatt); https://blog.zhaw.ch/high-performance/ (M. Rosenthal)© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).CEURWorkshopProceedingshttp://ceur-ws.orgISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)Figure 1: IoT device. The key hardware consists of a scale, a control unit and a camera.To achieve the target number 12.3 of the Sustainable Development Goals (SDG) it is crucialto: ”By 2030, halve per capita global food waste at the retail and consumer levels and reducefood losses along production and supply chains, including post-harvest losses” [4]. There existdifferent interventions which can lead to a reduction in food waste, such as size or type ofplates or changing nutritional guidelines [5]. Interestingly, a recent study has found that byonly providing food waste weight data to consumers and charging them accordingly, their foodwaste was reduced by 33% [6]. However, without detailed information about the wasted food itis difficult to make evidence-based decisions to prevent consummation stage food waste in acost-effective manner [5]. Evidence-based decisions are only possible if there is informationand data available about the subject. Regarding consumer based food waste it is crucial to haveinformation about the kind and quantity of thrown away food items. Smart garbage systems,which collect waste data in a decentralized manner and transfer data to the cloud, can be usedto collect this required data. Current IoT smart garbage systems are primarily used to ensurethat no garbage bins are overloaded [7, 8]. However, there exist some smart garbage systemswhich are specialized in food waste [9, 10].This paper shows how we have developed a smart IoT food waste monitoring system, shown infigure 1. It takes images and measures the weight of food waste in commercial kitchens anduses embedded machine learning (ML) to analyse the data. A summary of the collected data isprovided to the customers via an online dashboard. The customers can track their waste overtime and take data-driven decisions to reduce food waste.Using machine learning directly on the embedded device offers many benefits and has a hugepotential due to the omnipresence of embedded devices [11]. Moreover, running machinelearning algorithms directly on the embedded system enables real-time data processing, increasesprivacy because only results and not the raw data is transmitted to the cloud and reduces requiredbandwidth [12]. However, embedded systems have limited resources, such as memory or CPUpower. Therefore, it is important that the ML algorithms are optimized for the embedded system.There are several tools and frameworks which can be used for this task [13, 11, 14]. In thispaper we will report how much cloud storage, cloud computing cost and energy can be savedby running the ML algorithms directly on the embedded systems control unit, compared torunning them on a cloud based solution.2. Food Waste Management with Machine LearningKITRO’s IoT device is shown in figure 1. The key components are: a scale, a camera and anembedded control unit. The scale registers the increased weight when food gets thrown intothe bin. The control unit activates the camera and stores the image locally on the device. Thoseimages can then later be analysed using the embedded ML algorithms. Figure 2a and 2b showsome example images.As the sample images indicate, food waste classification is a difficult computer vision taskfor several reasons: a) Low inter-class variance: Based on visual features only, it is difficult todifferentiate between certain classes, i.e. different types of salad, b) High intra-class variance:Some food classes come in a wide variety of textures and colors if cooked or processed, c) Liquidfood waste, e.g. soups and stews as well as soft food waste, e.g. chopped vegetables and salads,can hide visual features of other food classes [15]. ML algorithms, in particular deep learningapproaches, seem most suitable for this challenging image processing task because they are ableto generalize well [16] and can be fine-tuned to specific data of selected canteens or additionalfood classes [17]. However, the foundation of every deep learning algorithm is labeled data.KITRO has collected and labeled a large amount of food waste images that can be used as deeplearning training data. The labels consist of food classes and the food weight. The labeledimages are split into different food classes such as: bread, banana, herb etc. The images cancontain one or more food or waste items. As an example the label of figure 2a is herb and thelabel of 2b is banana because those are the last added items. In addition, it can happen thatthe kitchen staff moves the bin or covers part or all of the bin. This makes food recognitionadditionally challenging. The ML algorithm has to detect and tag those error images such thatactions can be taken to reduce the number of errors.2.1. Machine Learning for Food Waste AnalysisThe ML algorithm has to solve two different tasks in order to analyse food waste. First it hasto perform change detection, meaning that it has to detect the newly added food and maskthat region in the input image. Then it has to classify the items within the masked region.Figure 2a resp. figure 2b show two example input images. Figure 2c and figure 2d show thecorresponding change masks. The final pre-processed images are shown in figure 2e and figure2f. The predicted labels for the food items are shown in the image captions.(a)ML example input image with newly addedfood.(b)ML example input image with newly addedfood.(c) Detected newly added food item with changemask and bounding box.(d) Detected newly added food item with changemask and bounding box.(e) Final pre-processed image. ML label prediction:herb.(f) Final pre-processed image. ML label prediction:banana.Figure 2: Different machine learning processing steps.3. TensorRT - Embedded InferenceWe implemented the ML algorithms directly on the embedded device in order to obtain real-time predictions, increase privacy and reduce the required bandwidth and energy consumption.Running the ML algorithms in the cloud also offers advantages such as: scaling, online dataanalysis, data and processing in one place, simpler access to the data. We implemented theembedded ML models using the TensorRT (TRT) framework. To compare the performanceto another widely used framework, they were also implemented with TensorFlow (TF). Bothframeworks use the GPU of the embedded device to run ML inference. TensorRT includes adeep learning inference optimizer, a runtime code that delivers low latency and high throughputinference [14]. Those optimizations are required in embedded applications because embeddedhardware has limited resources compared to cloud or desktop solutions. We have compareddifferent metrics such as tera floating point operations per second (TFLOPS), gpu memory(GPUMEM), mean processing time (PT) (time required for the ML to process one image) and itsstandard derivation (std) across three frameworks and hardware platforms.Table 1Platform and framework performance comparison results. PT measurements were repeated for 100times.Platform TFLOPs GPUMEM PT mean PT stdTesla P40 TF 11.76 24GB 0.15s 0.005Embedded TF 0.472 4GB 1.65s 0.15Embedded TRT 0.472 4GB 0.27s 0.0028Using the TensorRT framework on the embedded device speeds up the inference time by afactor of 6 compared to standard TensorFlow. This speedup is crucial for real-time applicationsand decision making on the embedded system. Moreover, the embedded device has less FLOPsand memory available and is still able to run the food waste analysis ML in less than one second,which is comparable to the inference time on a Tesla P40. Furthermore, by only forwarding thelabels to the cloud, a large amount of cloud resources can be saved. Table 2 shows the requiredmemory for a typical image taken by the camera and the required memory for a .json text filewhich contains the results of the embedded ML algorithm.Table 2Example memory required to store results.Data type Required memory.jpg image 1’881’180bytes.json text 1’435bytesIf we assume that we process 1000 images per day, the following amount of cloud storagecan be saved:(1881180𝑏𝑦𝑡𝑒𝑠 − 1435𝑏𝑦𝑡𝑒𝑠) ∗ 1000 = 1.81𝐺𝐵 (1)1.81GB of cloud storage is not that expensive today, with prices ranging from 0.005 to 0.021$/GB/Month[18]. However, those costs accumulate over time.The last measurement was to estimate and compare the power consumption. The cloudimplementation runs the ML algorithms on a Tesla K80 which has a power consumption of300𝑊 [19]. Not every processing step in the cloud is done on the GPU and the Tesla K80 isnot always running at 100%. Therefore, the average power was reduced by a factor of 2. If weagain assume that we need to process 1000 images per day, the cloud implementation of the MLalgorithms runs for 0.21ℎ (including an estimation of time required to upload the data). Theenergy consumption per day on the cloud can then be estimated with:300𝑊 ∗ 0.5 ∗ 0.21ℎ = 31.5𝑊ℎ (2)This energy consumption is only an estimation because we ignored the power consumption ofuploading images and storing images etc. However, the estimation makes it possible to comparethe cloud power consumption to the power consumption of the embedded device.The embedded device had an average power consumption of 4.47𝑊 and 0.075ℎ to process1000 images. It’s energy consumption (EC) can be estimated with:4.47𝑊 ∗ 0.075ℎ = 0.34𝑊ℎ (3)Table 3Estimated energy consumption of running the ML algorithms in the cloud vs running them directly onthe embedded system.Platform Estimated EC per dayCloud 31.5WhEmbedded 0.34WhAlthough the results of table 3 are estimated, they show the significant difference betweenthe two solutions. By running the ML algorithms on an embedded device, energy consumptioncan be reduced by a factor of ∼ 100. From a cost perspective the embedded device also offersadvantages: Costs are significantly lower because the embedded device is already used in mostof the IoT applications anyway and therefore only the power consumption adds additional costs.On the other hand, cloud instances have the advantage that they can be turned on and off ondemand. The embedded system has to run all day long which again adds to power consumptionand costs.When designing an IoT solution, both methods - running ML in the cloud and running MLon the embedded system - should be considered, because both have different advantages anddisadvantages. Depending on the specific use case, also a combination of the two approachescould be considered.4. Discussion and OutlookIn this paper we presented how a smart IoT device can be used to analyse and classify food waste.ML algorithms enable the solution to detect and classify a broad range of different food items.By showing customers which kind and how much food they wasted, they can take measuresto reduce the amount of wasted food and save costs. Furthermore, we showed the benefits ofrunning ML algorithms directly on the embedded IoT device. Many IoT control devices todayare capable of running small ML algorithms and are present in IoT solutions. Therefore, theonly additional cost is the energy consumption which is significantly lower than running thesame algorithms in the cloud. Another benefit is the real-time capability, the increased privacy,and the reduced memory requirement. Thus, depending on the use case, an embedded MLimplementation is an interesting alternative to a cloud implementation.AcknowledgmentsThis research was funded through an Innosuisse grant by the Swiss Innovation Agency (2019-2021).References[1] C. Beretta, S. Hellweg, Lebensmittelverluste in der schweiz: Umweltbelastung und ver-meidungspotenzial, ETH Zürich, commissioned by the Swiss Federal Office for the Envi-ronment FOEN (2019). URL: https://www.newsd.admin.ch/newsd/message/attachments/58769.pdf.[2] Auswirkungen von vermeidbaren lebensmittelverlusten auf die umwelt, https://www.bafu.admin.ch/bafu/de/home/themen/abfall/abfallwegweiser-a-z/biogene-abfaelle/abfallarten/lebensmittelabfaelle.html, 2021. Accessed: 2021-05-17.[3] Nahrungsmittelverluste im detailhandel und in der gastronomie in der schweiz, Bericht inErfüllung des Postulates 12.3907 von Isabelle Chevalley vom 28.9.2012, Accessed: 2021-05-17.[4] K. Flanagan, A. Clowes, B. Lipinski, L. Goodwin, R. Swannell, Sdg target 12.3on food loss and waste: 2018 progress report, An annual update on behalfof Champions 12 (2018). URL: https://champions123.org/sites/default/files/2020-09/champions-12-3-2019-progress-report.pdf.[5] C. Reynolds, L. Goucher, T. Quested, S. Bromley, S. Gillick, V. K. Wells, D. Evans, L. Koh,A. Carlsson Kanyama, C. Katzeff, Åsa Svenfelt, P. Jackson, Review: Consumption-stagefood waste reduction interventions – what works and how to design better interven-tions, Food Policy 83 (2019) 7–27. URL: https://www.sciencedirect.com/science/article/pii/S030691921830318X. doi:https://doi.org/10.1016/j.foodpol.2019.01.009.[6] I. Hong, S. Park, B. Lee, J. Lee, D. Jeong, S. Park, Iot-based smart garbage system forefficient food waste management, The Scientific World Journal 2014 (2014). URL: https://www.hindawi.com/journals/tswj/2014/646953/.[7] M. U. Sohag, A. K. Podder, Smart garbage management system for a sustainable ur-ban life: An iot based application, Internet of Things 11 (2020) 100255. URL: https://www.sciencedirect.com/science/article/pii/S2542660520300901. doi:https://doi.org/10.1016/j.iot.2020.100255.[8] S. Navghane, M. Killedar, V. Rohokale, Iot based smart garbage and waste collectionbin, International Journal of Advanced Research in Electronics and CommunicationEngineering (IJARECE) 5 (2016) 1576–1578. URL: https://www.technoarete.org/common_abstract/pdf/IJERCSE/v5/i4/Ext_04798.pdf.[9] Winnow vision, https://www.winnowsolutions.com/de/vision, Accessed: 2021-05-19.[10] The leanpath food waste prevention platform, https://www.leanpath.com/, Accessed:2021-05-19.[11] R. David, J. Duke, A. Jain, V. J. Reddi, N. Jeffries, J. Li, N. Kreeger, I. Nappier, M. Natraj,S. Regev, R. Rhodes, T. Wang, P. Warden, Tensorflow lite micro: Embedded machinelearning on tinyml systems, 2021. arXiv:2010.08678.[12] R. Zingg, M. Rosenthal, Artificial intelligence on microcontrollers, in: Em-bedded World Conference 2020, Nürnberg, 25.-27. Februar 2020, 2020. URL:https://raw.githubusercontent.com/InES-HPMM/Artificial_Intelligence_on_Microcontrollers/master/Artificial_Intelligence_on_Microcontrollers.pdf.[13] Ai expansion pack for stm32cubemx, https://www.st.com/en/embedded-software/x-cube-ai.html, Accessed: 2021-05-17.[14] Nvidia tensorrt, https://developer.nvidia.com/tensorrt, Accessed: 2021-05-17.[15] A. Mazloumian, M. Rosenthal, H. Gelke, Deep learning for classifying food waste, arXivpreprint arXiv:2002.03786 (2020). URL: https://arxiv.org/abs/2002.03786.[16] K. Kawaguchi, L. P. Kaelbling, Y. Bengio, Generalization in deep learning, 2020.arXiv:1710.05468.[17] C. Käding, E. Rodner, A. Freytag, J. Denzler, Fine-tuning deep neural networks in contin-uous learning scenarios, in: Asian Conference on Computer Vision, Springer, 2016, pp.588–605. URL: https://link.springer.com/chapter/10.1007/978-3-319-54526-4_43.[18] Cloud storage pricing comparison, https://www.backblaze.com/b2/cloud-storage-pricing.html, 2021. Accessed: 2021-05-19.[19] NVIDIA, Tesla k80 gpu accelerator board specification, https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/tesla-product-literature/Tesla-K80-BoardSpec-07317-001-v05.pdf, 2021. Accessed: 2021-05-19.A. Online ResourcesAn example workflow for custom TensorRT models is available on• GitHub

Smart food waste management : embedded machine learning vs cloud based solutions

https://digitalcollection.zhaw.ch/bitstream/11475/23847/3/2021_Zingg-etal_Smart-food-waste-management_FTAL.pdf

Smart food waste management : embedded machine learning vs cloud based solutions

Abstract

Similar works

Full text

Available Versions

ZHAW digitalcollection