Fruit Detection and Tree Segmentation for Yield Mapping in Orchards

Accurate information gathering and processing is critical for precision horticulture, as growers aim to optimise their farm management practices. An accurate inventory of the crop that details its spatial distribution along with health and maturity, can help farmers efficiently target processes such as chemical and fertiliser spraying, crop thinning, harvest management, labour planning and marketing. Growers have traditionally obtained this information by using manual sampling techniques, which tend to be labour intensive, spatially sparse, expensive, inaccurate and prone to subjective biases. Recent advances in sensing and automation for field robotics allow for key measurements to be made for individual plants throughout an orchard in a timely and accurate manner. Farmer operated machines or unmanned robotic platforms can be equipped with a range of sensors to capture a detailed representation over large areas. Robust and accurate data processing techniques are therefore required to extract high level information needed by the grower to support precision farming. This thesis focuses on yield mapping in orchards using image and light detection and ranging (LiDAR) data captured using an unmanned ground vehicle (UGV). The contribution is the framework and algorithmic components for orchard mapping and yield estimation that is applicable to different fruit types and orchard configurations. The framework includes detection of fruits in individual images and tracking them over subsequent frames. The fruit counts are then associated to individual trees, which are segmented from image and LiDAR data, resulting in a structured spatial representation of yield. The first contribution of this thesis is the development of a generic and robust fruit detection algorithm. Images captured in the outdoor environment are susceptible to highly variable external factors that lead to significant appearance variations. Specifically in orchards, variability is caused by changes in illumination, target pose, tree types, etc. The proposed techniques address these issues by using state-of-the-art feature learning approaches for image classification, while investigating the utility of orchard domain knowledge for fruit detection. Detection is performed using both pixel-wise classification of images followed instance segmentation, and bounding-box regression approaches. The experimental results illustrate the versatility of complex deep learning approaches over a multitude of fruit types. The second contribution of this thesis is a tree segmentation approach to detect the individual trees that serve as a standard unit for structured orchard information systems. The work focuses on trellised trees, which present unique challenges for segmentation algorithms due to their intertwined nature. LiDAR data are used to segment the trellis face, and to generate proposals for individual trees trunks. Additional trunk proposals are provided using pixel-wise classification of the image data. The multi-modal observations are fine-tuned by modelling trunk locations using a hidden semi-Markov model (HSMM), within which prior knowledge of tree spacing is incorporated. The final component of this thesis addresses the visual occlusion of fruit within geometrically complex canopies by using a multi-view detection and tracking approach. Single image fruit detections are tracked over a sequence of images, and associated to individual trees or farm rows, with the spatial distribution of the fruit counting forming a yield map over the farm. The results show the advantage of using multi-view imagery (instead of single view analysis) for fruit counting and yield mapping. This thesis includes extensive experimentation in almond, apple and mango orchards, with data captured by a UGV spanning a total of 5 hectares of farm area, over 30 km of vehicle traversal and more than 7,000 trees. The validation of the different processes is performed using manual annotations, which includes fruit and tree locations in image and LiDAR data respectively. Additional evaluation of yield mapping is performed by comparison against fruit counts on trees at the farm and counts made by the growers post-harvest. The framework developed in this thesis is demonstrated to be accurate compared to ground truth at all scales of the pipeline, including fruit detection and tree mapping, leading to accurate yield estimation, per tree and per row, for the different crops. Through the multitude of field experiments conducted over multiple seasons and years, the thesis presents key practical insights necessary for commercial development of an information gathering system in orchards

Fruit sizing using AI: A review of methods and challenges

Fruit size at harvest is an economically important variable for high-quality table fruit production in orchards and vineyards. In addition, knowing the number and size of the fruit on the tree is essential in the framework of precise production, harvest, and postharvest management. A prerequisite for analysis of fruit in a real-world environment is the detection and segmentation from background signal. In the last five years, deep learning convolutional neural network have become the standard method for automatic fruit detection, achieving F1-scores higher than 90 %, as well as real-time processing speeds. At the same time, different methods have been developed for, mainly, fruit size and, more rarely, fruit maturity estimation from 2D images and 3D point clouds. These sizing methods are focused on a few species like grape, apple, citrus, and mango, resulting in mean absolute error values of less than 4 mm in apple fruit. This review provides an overview of the most recent methodologies developed for in-field fruit detection/counting and sizing as well as few upcoming examples of maturity estimation. Challenges, such as sensor fusion, highly varying lighting conditions, occlusions in the canopy, shortage of public fruit datasets, and opportunities for research transfer, are discussed.This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646 and 2021 LLAV 00088) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / FEDER (grants RTI2018-094222-B-I00 [PAgFRUIT project] and PID2021-126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.info:eu-repo/semantics/publishedVersio

An integrated study of earth resources in the State of California based on ERTS-1 and supporting aircraft data, volume 1

There are no author-identified significant results in this report

Geographic Information Science (GIScience) and Geospatial Approaches for the Analysis of Historical Visual Sources and Cartographic Material

This book focuses on the use of GIScience in conjunction with historical visual sources to resolve past scenarios. The themes, knowledge gained and methodologies conducted might be of interest to a variety of scholars from the social science and humanities disciplines

Geoscience and remote sensing on horticulture as support for management and planning

The importance of horticulture around the large cities, called green belt (GB), or proximity food production area is related to its contribution to the provision of food as well as its role on social, cultural and ecological aspects. Geoscience and Remote sensing (GRS) are tools that should aid in gathering and updating the information to develop science-based management plans of this areas. Recently, the improvement in terms of spatial, temporal and radiometric resolutions has changed the performance and the approach to the horticulture remote sensing. In this work, we make a brief review on the literature exploring the use of GRS techniques in horticulture, and future trends in order to exploit the available techniques for efficient crop management in the way to improve territorial planning and management. Specifically we found a lack of academic production in this area. In addition we examine the importance of this landscape areas from different points of view (food security, health, ecology, etc.). A systematic revision of published studies on remote sensing on horticulture including different platforms, sensors and methodologies are briefly presented. Finally some aspect related with future trends are discussed.EEA ManfrediFil: Marinelli, María Victoria. Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET); ArgentinaFil: Marinelli, María Victoria. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria Manfredi. Agencia de Extensión Rural Córdoba; ArgentinaFil: Marinelli, María Victoria. Comisión Nacional de Actividades Espaciales (CONAE). Instituto de altos estudios espaciales "Mario Gulich"; ArgentinaFil: Marinelli, María Victoria. Universidad Nacional de Córdoba. Instituto de altos estudios espaciales "Mario Gulich"; ArgentinaFil: Scavuzzo, Carlos Matías. Universidad Nacional de Córdoba Facultad de Ciencias Médicas. Escuela de Nutrición; ArgentinaFil: Giobellina, Beatriz Liliana. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria Manfredi. Observatorio de Agricultura Urbana, Periurbana y Agroecología (O-AUPA); ArgentinaFil: Scavuzzo, Carlos Marcelo. Comisión Nacional de Actividades Espaciales (CONAE); Argentin

Yield sensing technologies for perennial and annual horticultural crops: a review

Yield maps provide a detailed account of crop production and potential revenue of a farm. This level of details enables a range of possibilities from improving input management, conducting on-farm experimentation, or generating profitability map, thus creating value for farmers. While this technology is widely available for field crops such as maize, soybean and grain, few yield sensing systems exist for horticultural crops such as berries, field vegetable or orchards. Nevertheless, a wide range of techniques and technologies have been investigated as potential means of sensing crop yield for horticultural crops. This paper reviews yield monitoring approaches that can be divided into proximal, either direct or indirect, and remote measurement principles. It reviews remote sensing as a way to estimate and forecast yield prior to harvest. For each approach, basic principles are explained as well as examples of application in horticultural crops and success rate. The different approaches provide whether a deterministic (direct measurement of weight for instance) or an empirical (capacitance measurements correlated to weight for instance) result, which may impact transferability. The discussion also covers the level of precision required for different tasks and the trend and future perspectives. This review demonstrated the need for more commercial solutions to map yield of horticultural crops. It also showed that several approaches have demonstrated high success rate and that combining technologies may be the best way to provide enough accuracy and robustness for future commercial systems

Geoscience and Remote Sensing on Horticulture as Support for Management and Planning

The importance of horticulture around the large cities, called green belt (GB), or proximity food production area is related to its contribution to the provision of food as well as its role on social, cultural and ecological aspects. Geoscience and Remote sensing (GRS) are tools that should aid in gathering and updating the information to develop science-based management plans of this areas. Recently, the improvement in terms of spatial, temporal and radiometric resolutions has changed the performance and the approach to the horticulture remote sensing. In this work, we make a brief review on the literature exploring the use of GRS techniques in horticulture, and future trends in order to exploit the available techniques for efficient crop management in the way to improve territorial planning and management. Specifically we found a lack of academic production in this area. In addition we examine the importance of this landscape areas from different points of view (food security, health, ecology, etc.). A systematic revision of published studies on remote sensing on horticulture including different platforms, sensors and methodologies are briefly presented. Finally some aspect related with future trends are discussed.Fil: Marinelli, María Victoria. Comision Nacional de Actividades Espaciales. Instituto de Altos Estudios Espaciales "Mario Gulich"; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; ArgentinaFil: Scavuzzo, Matías. Universidad Nacional de Córdoba. Facultad de Medicina. Escuela de Nutrición; ArgentinaFil: Giobellina, Beatriz. Instituto Nacional de Tecnologia Agropecuaria. Centro Regional Cordoba. Estacion Experimental Agropecuaria Manfredi. Agencia de Extension Rural Cordoba.; ArgentinaFil: Scavuzzo, Marcelo. Comision Nacional de Actividades Espaciales. Instituto de Altos Estudios Espaciales "Mario Gulich"; Argentin