Fruit sizing using AI: A review of methods and challenges

Fruit size at harvest is an economically important variable for high-quality table fruit production in orchards and vineyards. In addition, knowing the number and size of the fruit on the tree is essential in the framework of precise production, harvest, and postharvest management. A prerequisite for analysis of fruit in a real-world environment is the detection and segmentation from background signal. In the last five years, deep learning convolutional neural network have become the standard method for automatic fruit detection, achieving F1-scores higher than 90 %, as well as real-time processing speeds. At the same time, different methods have been developed for, mainly, fruit size and, more rarely, fruit maturity estimation from 2D images and 3D point clouds. These sizing methods are focused on a few species like grape, apple, citrus, and mango, resulting in mean absolute error values of less than 4 mm in apple fruit. This review provides an overview of the most recent methodologies developed for in-field fruit detection/counting and sizing as well as few upcoming examples of maturity estimation. Challenges, such as sensor fusion, highly varying lighting conditions, occlusions in the canopy, shortage of public fruit datasets, and opportunities for research transfer, are discussed.This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646 and 2021 LLAV 00088) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / FEDER (grants RTI2018-094222-B-I00 [PAgFRUIT project] and PID2021-126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.info:eu-repo/semantics/publishedVersio

Multispectral Image Analysis of Remotely Sensed Crops

The range in topography, biodiversity, and agricultural technology has led to the emergence of precision agriculture. Precision agriculture is a farming management concept based on monitoring, measuring, and responding to crop variability. Computer vision, image analysis, and image processing are gaining considerable traction. For this paper, image analysis involves recognizing individual objects and providing insights from vegetation indices. The data acquired was remote-sensed multispectral images from blueberry, maguey, and pineapple. After computing vegetation indices, histograms were analyzed to choose thresholds. The masking of vegetation indices with threshold allowed the removal of areas with shadows and soil. The four leading vegetation indices used were the Normalized Difference Vegetation Index (NDVI), the Normalized Difference Red Edge (NDRE), the Simple Ratio, the Red Edge Chlorophyll Index, and the Visible Atmospherically Resistant Index (SAVI). This research reviews literature for acquiring, preprocessing, and analyzing remote-sensed multispectral images in precision agriculture. It compiles the theoretical framework for analyzing multispectral data. Also, it describes and implements radiometric calibration and image alignment using the custom code from the MicaSense repository. As a result, it was possible to segment the blueberry, tequila agave, and pineapple plants from the background regardless of the noisy images. Non-plant pixels were excluded and shown as transparent by masking areas with shadows and low NDVI pixels, which sometimes removed plant pixels. The NDVI and NDRE helped identify crop pixels. On the other hand, it was possible to identify the pineapple fruits from the agave plantation using the SAVI vegetation index and the thresholding method. Finally, the work identifies the problems associated with an incorrect data acquisition methodology and provides suggestions.ITESO, A. C

Apple (Malus domestica) and pear (Pyrus communis) yield prediction after tree image analysis

Yield forecasting depends on accurate tree fruit counts and mean size estimation. This information is generally obtained manually, requiring many hours of work. Artificial vision emerges as an interesting alternative to obtaining more information in less time. This study aimed to test and train YOLO pre-trained models based on neural networks for the detection and count of pears and apples on trees after image analysis; while also estimating fruit size. Images of trees were taken during the day and at night in apple and pear trees while fruits were manually counted. Trained models were evaluated according to recall, precision and F1score. The correlation between detected and counted fruits was calculated while fruit size estimation was made after drawing straight lines on each fruit and using reference elements. The precision, recall and F1score achieved by the models were up to 0.86, 0.83 and 0.84, respectively. Correlation coefficients between fruit sizes measured manually and by images were 0.73 for apples and 0.80 for pears. The proposed methodologies showed promising results, allowing forecasters to make less time consuming and accurate estimates compared to manual measurements. Highlights The number of fruits in apple and pear trees, could be estimated from images with promising results. The possibility of estimating the fruit numbers from images could reduce the time spent on this task, and above all, the costs. This allow growers to increase the number of trees sampled to make yield forecasts.Yield forecasting depends on accurate tree fruit counts and mean size estimation. This information is generally obtained manually, requiring many hours of work. Artificial vision emerges as an interesting alternative to obtaining more information in less time. This study aimed to test and train YOLO pre-trained models based on neural networks for the detection and count of pears and apples on trees after image analysis; while also estimating fruit size. Images of trees were taken during the day and at night in apple and pear trees while fruits were manually counted. Trained models were evaluated according to recall, precision and F1score. The correlation between detected and counted fruits was calculated while fruit size estimation was made after drawing straight lines on each fruit and using reference elements. The precision, recall and F1score achieved by the models were up to 0.86, 0.83 and 0.84, respectively. Correlation coefficients between fruit sizes measured manually and by images were 0.73 for apples and 0.80 for pears. The proposed methodologies showed promising results, allowing forecasters to make less time consuming and accurate estimates compared to manual measurements. Highlights The number of fruits in apple and pear trees, could be estimated from images with promising results. The possibility of estimating the fruit numbers from images could reduce the time spent on this task, and above all, the costs. This allow growers to increase the number of trees sampled to make yield forecasts

Advanced Image Acquisition, Processing Techniques and Applications

"Advanced Image Acquisition, Processing Techniques and Applications" is the first book of a series that provides image processing principles and practical software implementation on a broad range of applications. The book integrates material from leading researchers on Applied Digital Image Acquisition and Processing. An important feature of the book is its emphasis on software tools and scientific computing in order to enhance results and arrive at problem solution

Computer Vision System for Non-Destructive and Contactless Evaluation of Quality Traits in Fresh Rocket Leaves (Diplotaxis Tenuifolia L.)

La tesi di dottorato è incentrata sull'analisi di tecnologie non distruttive per il controllo della qualità dei prodotti agroalimentari, lungo l'intera filiera agroalimentare. In particolare, la tesi riguarda l'applicazione del sistema di visione artificiale per valutare la qualità delle foglie di rucola fresh-cut. La tesi è strutturata in tre parti (introduzione, applicazioni sperimentali e conclusioni) e in cinque capitoli, rispettivamente il primo e il secondo incentrati sulle tecnologie non distruttive e in particolare sui sistemi di computer vision per il monitoraggio della qualità dei prodotti agroalimentari. Il terzo, quarto e quinto capitolo mirano a valutare le foglie di rucola sulla base della stima di parametri qualitativi, considerando diversi aspetti: (i) la variabilità dovuta alle diverse pratiche agricole, (ii) la senescenza dei prodotti confezionati e non, e (iii) lo sviluppo e sfruttamento dei vantaggi di nuovi modelli più semplici rispetto al machine learning utilizzato negli esperimenti precedenti. Il lavoro di ricerca di questa tesi di dottorato è stato svolto dall'Università di Foggia, dall'Istituto di Scienze delle Produzioni Alimentari (ISPA) e dall'Istituto di Tecnologie e Sistemi Industriali Intelligenti per le Manifatture Avanzate (STIIMA) del Consiglio Nazionale delle Ricerche (CNR). L’attività di ricerca è stata condotta nell'ambito del Progetto SUS&LOW (Sustaining Low-impact Practices in Horticulture through Non-destructive Approach to Provide More Information on Fresh Produce History & Quality), finanziato dal MUR-PRIN 2017, e volto a sostenere la qualità della produzione e dell'ambiente utilizzando pratiche agricole a basso input e la valutazione non distruttiva della qualità di prodotti ortofrutticoli.The doctoral thesis focused on the analysis of non-destructive technologies available for the control quality of agri-food products, along the whole supply chain. In particular, the thesis concerns the application of computer vision system to evaluate the quality of fresh rocket leaves. The thesis is structured in three parts (introduction, experimental applications and conclusions) and in 5 chapters, the first and second focused on non-destructive technologies and in particular on computer vision systems for monitoring the quality of agri-food products, respectively. The third, quarter, and fifth chapters aim to assess the rocket leaves based on the estimation of quality aspects, considering different aspects: (i) the variability due to the different agricultural practices, (ii) the senescence of packed and unpacked products, and (iii) development and exploitation of the advantages of new models simpler than the machine learning used in the previous experiments. The research work of this doctoral thesis was carried out by the University of Foggia, the Institute of Science of Food Production (ISPA) and the Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing (STIIMA) of National Research Council (CNR). It was conducted within the Project SUS&LOW (Sustaining Low-impact Practices in Horticulture through Non-destructive Approach to Provide More Information on Fresh Produce History & Quality), funded by MUR- PRIN 2017, and aimed at sustaining quality of production and of the environment using low input agricultural practices and non-destructive quality evaluation

Biological responses and control of California red scale Aonidiella aurantii (Maskell) (Hemiptera: Diaspididae)

In many citrus areas around the world and within citrus-producing regions of Australia, the California red scale (CRS), Aonidiella aurantii (Maskell) (Hemiptera: Diaspididae), is considered the most important pests of citrus. The main biological control agents of Ao. aurantii in this zone are the parasitoid Aphytis melinus DeBach (Hymenoptera: Aphelinidae). In order to improve the biological control of Ao. aurantii several biotic and abiotic factors were studied, that may affect the efficiency of A. melinus in the laboratory and the field. More concretely, reproductive potential and age-specific fecundity schedules of Ao. aurantii were studied in the laboratory at constant temperatures (20, 23 and 27°C), while the biological parameters of its parasitoid A. melinus were conducted at 27°C. Results revealed that the net reproduction rate (Ro) was considerably higher for Ao. aurantii than A. melinus, which reached 28.14 at 27°C, indicating its high reproductive capacity. Moreover, the net reproduction rate obtained for A. melinus indicates a low substitution potential for each female having Ao. aurantii as a host under laboratory conditions. The intrinsic rate of increase (rm) of A. melinus (0.188 ♀/♀/day) was significantly greater than that of Ao. aurantii (0.080) at 27°C. Plants produce volatile organic compounds (VOCs) in response to herbivore attack, and these VOCs can be exploited by parasitoids of the herbivore as host location cues. The VOCs from non-infested and Ao. aurantii-infested citrus fruit were investigated using headspace solid-phase microextraction (HS-SPME) coupled with gas chromatography-mass spectrometry (GC-MS). The data showed that more than 52 different compounds were identified, and different emissions associated attributed to herbivore activity were found for all fruit species (lemon, orange, mandarin and Tahitian lime). However, a single compound was exclusively produced by infested lemon fruit, while two compounds were significantly increased, and two compounds were only present in non-infested lemon. Five compounds were significantly increased in infested mandarins. For orange, five compounds were increased, and five compounds were exclusively presented in infested fruit. For lime fruit, eighteen of these compounds were increased, one was decreased, whereas five compounds were produced exclusively from infested lime fruit. Two putative herbivores-induced plant volatiles, d-limonene and β-ocimene, were significantly increased by Ao. aurantii infestation in all infested fruit, regardless of the citrus species. Subsequently, the preferences of female parasitoid on infested or healthy fruit in olfactometer bioassays were evaluated. Then in order to understand the magnitude of volatile attractiveness, the innate attractiveness of VOCs to A. melinus females in varying densities were tested in the laboratory. The results of the olfactometer assays that tested the behaviour of A. melinus to the different compounds emitted from infested and non-infested citrus fruit showed no such preference when compared between non-infested and infested oranges, mandarins and lime fruit; whilst, there were significant preferences for lemon fruit infested with Ao. aurantii over non-infested ones. For assessment, the attraction of synthetic Herbivore induced plant volatiles (HIPVs), four different concentrations (5,10, 15 and 20 μl/ml) of d-l-limonene and β-ocimene were investigated. However, mated A. melinus females preferred the reward-associated VOC more than hexane control in the case of d-limonene at the tested dosages of 15 and 20 μl/ml, β-ocimene at tested dosages of 10, 15 and 20 μl/ml. Finally, this study evaluated the dispersal ability of released A. melinus adults and their effect on the parasitism percentage, using d-limonene and β-ocimene with yellow sticky traps and scoring percentage parasitism on infested fruit. Under field conditions, the natural enemies’ effectiveness in controlling pests is largely correlated with their capability to spread towards infested crops. In this study, d-limonene and β-ocimene were examined for their attractiveness to California red scale parasitoid A. melinus in the field after augmentative releases. Field experiments demonstrated that lures baited with isolates of d-limonene and\or β-ocimene, which significantly attracted some species of natural enemies but had no significant impact on others. The number of A. melinus captured during the whole trial was greater in the traps treated with volatiles than the control. Finally, the overall parasitism rates were not increased by synthetic HIPV lures, but there was evidence that lures may increase parasitism of California red scale when there is a decrease in the amount of volatile organic compounds due to lack of healthy and infested fruit. In conclusion, HIPVs can potentially play important roles in attracting and exploiting natural enemies to reduce pest infestations

New strategies for row-crop management based on cost-effective remote sensors

Agricultural technology can be an excellent antidote to resource scarcity. Its growth has led to the extensive study of spatial and temporal in-field variability. The challenge of accurate management has been addressed in recent years through the use of accurate high-cost measurement instruments by researchers. However, low rates of technological adoption by farmers motivate the development of alternative technologies based on affordable sensors, in order to improve the sustainability of agricultural biosystems. This doctoral thesis has as main objective the development and evaluation of systems based on affordable sensors, in order to address two of the main aspects affecting the producers: the need of an accurate plant water status characterization to perform a proper irrigation management and the precise weed control. To address the first objective, two data acquisition methodologies based on aerial platforms have been developed, seeking to compare the use of infrared thermometry and thermal imaging to determine the water status of two most relevant row-crops in the region, sugar beet and super high-density olive orchards. From the data obtained, the use of an airborne low-cost infrared sensor to determine the canopy temperature has been validated. Also the reliability of sugar beet canopy temperature as an indicator its of water status has been confirmed. The empirical development of the Crop Water Stress Index (CWSI) has also been carried out from aerial thermal imaging combined with infrared temperature sensors and ground measurements of factors such as water potential or stomatal conductance, validating its usefulness as an indicator of water status in super high-density olive orchards. To contribute to the development of precise weed control systems, a system for detecting tomato plants and measuring the space between them has been developed, aiming to perform intra-row treatments in a localized and precise way. To this end, low cost optical sensors have been used and compared with a commercial LiDAR laser scanner. Correct detection results close to 95% show that the implementation of these sensors can lead to promising advances in the automation of weed control. The micro-level field data collected from the evaluated affordable sensors can help farmers to target operations precisely before plant stress sets in or weeds infestation occurs, paving the path to increase the adoption of Precision Agriculture techniques

Deep learning in food category recognition

Integrating artificial intelligence with food category recognition has been a field of interest for research for the past few decades. It is potentially one of the next steps in revolutionizing human interaction with food. The modern advent of big data and the development of data-oriented fields like deep learning have provided advancements in food category recognition. With increasing computational power and ever-larger food datasets, the approach’s potential has yet to be realized. This survey provides an overview of methods that can be applied to various food category recognition tasks, including detecting type, ingredients, quality, and quantity. We survey the core components for constructing a machine learning system for food category recognition, including datasets, data augmentation, hand-crafted feature extraction, and machine learning algorithms. We place a particular focus on the field of deep learning, including the utilization of convolutional neural networks, transfer learning, and semi-supervised learning. We provide an overview of relevant studies to promote further developments in food category recognition for research and industrial applicationsMRC (MC_PC_17171)Royal Society (RP202G0230)BHF (AA/18/3/34220)Hope Foundation for Cancer Research (RM60G0680)GCRF (P202PF11)Sino-UK Industrial Fund (RP202G0289)LIAS (P202ED10Data Science Enhancement Fund (P202RE237)Fight for Sight (24NN201);Sino-UK Education Fund (OP202006)BBSRC (RM32G0178B8

Sustainable Agriculture and Advances of Remote Sensing (Volume 2)

Agriculture, as the main source of alimentation and the most important economic activity globally, is being affected by the impacts of climate change. To maintain and increase our global food system production, to reduce biodiversity loss and preserve our natural ecosystem, new practices and technologies are required. This book focuses on the latest advances in remote sensing technology and agricultural engineering leading to the sustainable agriculture practices. Earth observation data, in situ and proxy-remote sensing data are the main source of information for monitoring and analyzing agriculture activities. Particular attention is given to earth observation satellites and the Internet of Things for data collection, to multispectral and hyperspectral data analysis using machine learning and deep learning, to WebGIS and the Internet of Things for sharing and publication of the results, among others